Package gavo :: Package rsc :: Module data

Module data

Making data out of descriptors and sources.

Classes
	Data A collection of tables.
	DDDependencyGraph a graph giving the dependency structure between DDs.

Functions

processSource(data, source, feeder, opts, connection=None)
ingests source into the Data instance data.

source code

makeData(dd, parseOptions=<ParseOptions validateRows=False maxRows=None keepGoing=False>, forceSource=None, connection=None, data=None, runCommit=True)
returns a data instance built from ``dd``.

source code

makeDependentsFor(dds, parseOptions, connection)
rebuilds all data dependent on one of the DDs in the dds sequence.

source code

makeDataById(ddId, parseOptions=<ParseOptions validateRows=False maxRows=None keepGoing=False>, connection=None, inRD=None)
returns the data set built from the DD with ddId (which must be fully qualified).

source code

wrapTable(table, rdSource=None)
returns a Data instance containing only table (or table if it's already a data instance).

source code

makeCombinedData(baseDD, tablesForRoles)
returns a Data instance containing all of tablesForRoles.

source code

Variables
	__package__ = `'gavo.rsc'`

Function Details

processSource(data, source, feeder, opts, connection=None)

source code

ingests source into the Data instance data.

If this builds database tables, you must pass in a connection object.

If opts.keepGoing is True,the system will continue importing even if a particular source has caused an error. In that case, everything contributed by the bad source is rolled back (this will only work when filling database tables).

makeData(dd, parseOptions=<ParseOptions validateRows=False maxRows=None keepGoing=False>, forceSource=None, connection=None, data=None, runCommit=True)

source code

returns a data instance built from ``dd``.

It will arrange for the parsing of all tables generated from dd's grammar.

If database tables are being made, you *must* pass in a connection. The entire operation will then run within a single transaction within this connection (except for building dependents; they will be built in separate transactions).

The connection will be rolled back or committed depending on the success of the operation (unless you pass ``runCommit=False``, in which case even a successful import will not be committed)..

You can pass in a data instance created by yourself in data. This makes sense if you want to, e.g., add some meta information up front.

wrapTable(table, rdSource=None)

source code

returns a Data instance containing only table (or table if it's already a data instance).

If table has no rd, you must pass rdSource, which must be an object having and rd attribute (rds, tabledefs, etc, work).

This will grab info meta from the table.

makeCombinedData(baseDD, tablesForRoles)

source code

returns a Data instance containing all of tablesForRoles.

A DD is being generated based on baseDD; if baseDD has any tables, they are discarded.

tablesForRoles is a mapping from strings (one of which should be "primary") to tables; the strings end up as roles.