gavo.rsc.dumping module

DaCHS supports dump/restore operations on tables or set of tables. This module implements the underlying file format and some utilities.

The file format itself is a tar.gz file with an index.txt consisting of lines

file_name table-id

followed by the files. Table id is DaCHS’s usual rd-id#table-id. The files contain binary dump material.

gavo.rsc.dumping.create(args)

gavo.rsc.dumping.createDump(tableIds, destFile, binary=True)

writes a DaCHS dump of tableIds to destFile.

tableIds is a list of rd-id#table-id identifiers (all must resolve), destFile is a file object opened for writing.

gavo.rsc.dumping.getTablesForIds(tableIds, connection)

returns a list of validated dbtables of all tableIds.

This will raise an exception if any table id or database table doesn’t exist, or if the on-disk schema doesn’t match the definition.

For convenience in internal use, tableIds that already are table instances will not be touched. That’s a bit tricky, though, because you can have data from different transactions when you do that.

gavo.rsc.dumping.iterDbTables(objectId, connection)

iterates over dbtable objects referenced by objectId.

objectId can reference a table def or an RD (in which case all onDisk tables from it are returned). Or it can be a table already.

gavo.rsc.dumping.iterTableInfos(dumpFile)

iterates over table info tuples from an open dump file.

Each tuple has the member name, the table id, a boolean whether the table definion is accessible, the UTC unix time the dump was made, and the size of the dump.

gavo.rsc.dumping.load(args)

gavo.rsc.dumping.ls(args)

gavo.rsc.dumping.main()

does the user interaction.

gavo.rsc.dumping.parseIndexFile(srcFile)

parses our index.txt file format and returns (member-name, table-id) tuples.

srcFile is an open binary file, probably from TarFile.extractfile.

If you change the index file format, you’ll have to change this and the corresponding code in createDump.

gavo.rsc.dumping.restoreDump(dumpFile)

restores a dump.

dumpFile is an open file object containing a file created by createDump.

This comprises recrating all mentioned tables, copying in the associated data, and re-creating all indices.

Each table is handled in a separate transaction, we do not stop if a single restore has failed.