Package gavo :: Package rsc :: Module dumping

Module dumping

DaCHS supports dump/restore operations on tables or set of tables. This module implements the underlying file format and some utilities.

The file format itself is a tar.gz file with an index.txt consisting of lines

file_name table-id

followed by the files. Table id is DaCHS's usual rd-id#table-id. The files contain binary dump material.

Functions

parseIndexFile(srcFile)
parses our index.txt file format and returns (member-name, table-id) tuples.

source code

iterDbTables(objectId, connection)
iterates over dbtable objects referenced by objectId.

source code

getTablesForIds(tableIds, connection)
returns a list of validated dbtables of all tableIds.

source code

createDump(tableIds, destFile)
writes a DaCHS dump of tableIds to destFile.

source code

iterTableInfos(dumpFile)
iterates over table info tuples from an open dump file.

source code

restoreDump(dumpFile)
restores a dump.

source code

create(args)

source code

load(args)

source code

ls(args)

source code

main()
does the user interaction.

source code

Variables
	__package__ = `'gavo.rsc'`

Function Details

parseIndexFile(srcFile)

source code

parses our index.txt file format and returns (member-name, table-id) tuples.

srcFile is an open file, probably from TarFile.extractfile.

If you change the index file format, you'll have to change this and the corresponding code in createDump.

iterDbTables(objectId, connection)

source code

iterates over dbtable objects referenced by objectId.

objectId can reference a table def or an RD (in which case all onDisk tables from it are returned). Or it can be a table already.

getTablesForIds(tableIds, connection)

source code

returns a list of validated dbtables of all tableIds.

This will raise an exception if any table id or database table doesn't exist, or if the on-disk schema doesn't match the definition.

For convenience in internal use, tableIds that already are table instances will not be touched. That's a bit tricky, though, because you can have data from different transactions when you do that.

createDump(tableIds, destFile)

source code

writes a DaCHS dump of tableIds to destFile.

tableIds is a list of rd-id#table-id identifiers (all must resolve), destFile is a file object opened for writing.

iterTableInfos(dumpFile)

source code

iterates over table info tuples from an open dump file.

Each tuple has the member name, the table id, a boolean whether the table definion is accessible, the UTC unix time the dump was made, and the size of the dump.

restoreDump(dumpFile)

source code

restores a dump.

dumpFile is an open file object containing a file crated by createDump.

This comprises recrating all mentioned tables, copying in the associated data, and re-creating all indices.

Each table is handled in a separate transaction, we do not stop if a single restore has failed.

create(args)

source code

Decorators:

@utils.exposedFunction([utils.Arg("dumpFile", help= "Name of a file to write the dump to; use - to" " dump to stderr."), utils.Arg("ids", help= "ids of table definitions (as in myres/q#main)" " or RDs to dump.", nargs= "+")], help= "Dump one or more tables to DaCHS' dump format.")

load(args)

source code

Decorators:

@utils.exposedFunction([utils.Arg("source", help= "File to restore from. Use - to restore from stdin.")], help= "Restore one or more table(s) from a file created by the create" " subcommand before")

ls(args)

source code

Decorators:

@utils.exposedFunction([utils.Arg("source", help= "File to list")], help= "List tables and dump metadata from a DaCHS dump.")