Package: z3c.dobbin
ImportantThis project has been retired; another projectdobbinhas taken its place.OverviewDobbin is an object database implemented on top of SQLAlchemy. It’s
designed to mimick the behavior of the Zope object database (ZODB)
while providing greater flexibility and control of the storage.It supports strong typing with native SQL columns by utilizing the
declarative field definitions from zope.schema. Weak typing is
supported using the Python pickle protocol. Attributes are
automatically persisted with the exception of those starting with the
characters “_v_” (volatile attributes).Tables to support the strongly typed attributes are created on-the-fly
with a 1:1 correspondence to interfaces with no inheritance (base
interface). As such, objects are modelled as a join between the
interfaces they implement plus a table that maintains object metadata
and weakly typed instance attributes.AuthorsThis package was designed and implemented by Malthe Borch and Stefan
Eletzhofer with parts contributed by Kapil Thangavelu and Laurence
Rowe. It’s licensed as ZPL.Developer documentationObjects are mapped by their specification. Polymorphic attributes are
declared as interface attributes; strong typing may be declared using
schema fields; Attributes that are not declared in a schema or
interface are considered volatile.Unique identifiers (UUID)A 16-byte unique identification number is used.RelationsPolymorphic attributes are always stored using foreign key
relations. This is handled transparently by the framework.The target of a relation may be a basic type such as a string,
integer, tuple or list, or it may be a mapped object.The following fields allow polymorphic relations of any kind with the
type declared on assignment.zope.schema.Objectzope.interface.AttributeCollections are instrumented objects and may be declared using the
sequence fields:zope.schema.Listzope.schema.Dictzope.schema.SetA note on dictionaries: Dictionaries are keyed by (unicode)
string. Mapped instances may be used as keys in which case a string
representation of the unique instance identifier is used. Dictionaries
support polymorphic values with type set on assignment.Walk-through of the frameworkThis section demonstrates the functionality of the package.IntroductionDobbin uses SQLAlchemy’s object relational mapper to transparently
store objects in the database. Objects are persisted in two levels:Attributes may correspond directly to a table column in which case we
say that the attribute is strongly typed. This is the most optimal
way to store data.We may also store attributes that are not mapped directly to a column;
in this case, the value of the attribute is stored as a Python
pickle. This allows weak typing, but also persistence of amorphic
data, e.g. data which does not fit naturally in a relational database.A universally unique id (UUID) is automatically assigned to all
objects.We begin with a new database session.>>> import z3c.saconfig
>>> session = z3c.saconfig.Session()Declarative configurationWe can map attributes to table columns using zope.schema. Instead of
using SQL column definitions, we rely on the declarative properties of
schema fields.We start out with an interface decribing a recorded album.>>> class IAlbum(interface.Interface):
...     artist = schema.TextLine(
...         title=u"Artist",
...         default=u"")
...
...     title = schema.TextLine(
...         title=u"Title",
...         default=u"")We can now fabricate instances that implement this interface by using
thecreatemethod. This is a shorthand for setting up a mapper and
creating an instance by calling it.>>> from z3c.dobbin.factory import create
>>> album = create(IAlbum)Set attributes.>>> album.artist = "The Beach Boys"
>>> album.title = u"Pet Sounds"Interface inheritance is supported. For instance, a vinyl record is a
particular type of album.>>> class IVinyl(IAlbum):
...     rpm = schema.Int(
...         title=u"RPM")>>> vinyl = create(IVinyl)
>>> vinyl.artist = "Diana Ross and The Supremes"
>>> vinyl.title = "Taking Care of Business"
>>> vinyl.rpm = 45The attributes are instrumented by SQLAlchemy and map directly to a
column in a table.>>> IVinyl.__mapper__.artist
<sqlalchemy.orm.attributes.InstrumentedAttribute object at ...>A compact disc is another kind of album.>>> class ICompactDisc(IAlbum):
...     year = schema.Int(title=u"Year")Let’s pick a more recent Diana Ross, to fit the format.>>> cd = create(ICompactDisc)
>>> cd.artist = "Diana Ross"
>>> cd.title = "The Great American Songbook"
>>> cd.year = 2005To verify that we’ve actually inserted objects to the database, we
commit the transacation, thus flushing the current session.>>> session.save(album)
>>> session.save(vinyl)
>>> session.save(cd)We must actually query the database once before proceeding; this seems
to be a bug inzope.sqlalchemy.>>> results = session.query(album.__class__).all()Proceed with the transaction.>>> import transaction
>>> transaction.commit()We get a reference to the database metadata object, to locate each
underlying table.>>> engine = session.bind
>>> metadata = engine.metadataTables are given a name based on the dotted path of the interface they
describe. A utility method is provided to create a proper table name
for an interface.>>> from z3c.dobbin.mapper import encodeVerify tables forIVinyl,IAlbumandICompactDisc.>>> session.bind = metadata.bind
>>> session.execute(metadata.tables[encode(IVinyl)].select()).fetchall()
[(2, 45)]>>> session.execute(metadata.tables[encode(IAlbum)].select()).fetchall()
[(1, u'Pet Sounds', u'The Beach Boys'),
 (2, u'Taking Care of Business', u'Diana Ross and The Supremes'),
 (3, u'The Great American Songbook', u'Diana Ross')]>>> session.execute(metadata.tables[encode(ICompactDisc)].select()).fetchall()
[(3, 2005)]Mapping concrete classesNow we’ll create a mapper based on a concrete class. We’ll let the
class implement the interface that describes the attributes we want to
store, but also provides a custom method.>>> class Vinyl(object):
...     interface.implements(IVinyl)
...
...     def __repr__(self):
...         return "<Vinyl %s: %s (@ %d RPM)>" % \
...                (self.artist, self.title, self.rpm)Although the symbols we define in this test report that they’re
available from the__builtin__module, they really aren’t.We’ll manually add these symbols.>>> import __builtin__
>>> __builtin__.IVinyl = IVinyl
>>> __builtin__.IAlbum = IAlbum
>>> __builtin__.Vinyl = VinylCreate an instance using thecreatefactory.>>> vinyl = create(Vinyl)Verify that we’ve instantiated and instance of our class.>>> isinstance(vinyl, Vinyl)
TrueCopy the attributes from the Diana Ross vinyl record.>>> diana = session.query(IVinyl.__mapper__).filter_by(
...     artist=u"Diana Ross and The Supremes")[0]
>>> vinyl.artist = diana.artist
>>> vinyl.title = diana.title
>>> vinyl.rpm = diana.rpmVerify that the methods on ourVinyl-class are available on the mapper.>>> repr(vinyl)
'<Vinyl Diana Ross and The Supremes: Taking Care of Business (@ 45 RPM)>'When mapping a class we may run into properties that should take the
place of a column (a read-only value). As an example, consider this
experimental record class where rotation speed is a function of the
title and artist.>>> class Experimental(Vinyl):
...     @property
...     def rpm(self):
...         return len(self.title+self.artist)>>> experimental = create(Experimental)XXX: There’s currently an issue with SQLAlchemy that hinders this
behavior; it specifically won’t work if a default value is set on the
column that we’re overriding.>>> # session.save(experimental)>>> experimental.artist = vinyl.artist
>>> experimental.title = vinyl.titleLet’s see how fast this record should be played back.>>> experimental.rpm
50RelationsRelations are columns that act as references to other objects. They’re
declared using thezope.schema.Objectfield.Note that we needn’t declare the relation target type in advance,
although it may be useful in general to specialize theschemakeyword parameter.>>> class IFavorite(interface.Interface):
...     item = schema.Object(
...         title=u"Item",
...         schema=interface.Interface)>>> __builtin__.IFavorite = IFavoriteLet’s make our Diana Ross record a favorite.>>> favorite = create(IFavorite)
>>> favorite.item = vinyl
>>> favorite.item
<Vinyl Diana Ross and The Supremes: Taking Care of Business (@ 45 RPM)>>>> session.save(favorite)We’ll commit the transaction and lookup the object by its unique id.>>> transaction.commit()>>> from z3c.dobbin.soup import lookup
>>> favorite = lookup(favorite.uuid)When we retrieve the related items, it’s automatically reconstructed
to match the specification to which it was associated.>>> favorite.item
<Vinyl Diana Ross and The Supremes: Taking Care of Business (@ 45 RPM)>We can create relations to objects that are not mapped. Let’s model an
accessory item.>>> class IAccessory(interface.Interface):
...     name = schema.TextLine(title=u"Name of accessory")>>> class Accessory(object):
...     interface.implements(IAccessory)
...
...     def __repr__(self):
...          return "<Accessory '%s'>" % self.nameIf we now instantiate an accessory and assign it as a favorite item,
we’ll implicitly create a mapper from the class specification and
insert it into the database.>>> cleaner = Accessory()
>>> cleaner.name = u"Record cleaner"Set up relation.>>> favorite.item = cleanerLet’s try and get back our record cleaner item.>>> __builtin__.Accessory = Accessory
>>> favorite.item
<Accessory 'Record cleaner'>Within the same transaction, the relation will return the original
object, maintaining integrity.>>> favorite.item is cleaner
TrueThe session keeps a copy of the pending object until the transaction
is ended.>>> cleaner in session._d_pending.values()
TrueHowever, once we commit the transaction, the relation is no longer
attached to the relation source, and the correct data will be
persisted in the database.>>> cleaner.name = u"CD cleaner">>> session.flush()
>>> session.update(favorite)>>> favorite.item.name
u'CD cleaner'This behavior should work well in a request-response type environment,
where the request will typically end with a commit.CollectionsWe can instrument properties that behave like collections by using the
sequence and mapping schema fields.Let’s set up a record collection as an ordered list.>>> class ICollection(interface.Interface):
...     records = schema.List(
...         title=u"Records",
...         value_type=schema.Object(schema=IAlbum)
...         )>>> __builtin__.ICollection = ICollection>>> collection = create(ICollection)
>>> collection.records
[]Add the Diana Ross record, and save the collection to the session.>>> collection.records.append(diana)
>>> session.save(collection)We can get our collection back.>>> collection = lookup(collection.uuid)Let’s verify that we’ve stored the Diana Ross record.>>> record = collection.records[0]>>> record.artist, record.title
(u'Diana Ross and The Supremes', u'Taking Care of Business')>>> session.flush()When we create a new, transient object and append it to a list, it’s
automatically saved on the session.>>> collection = lookup(collection.uuid)>>> kool = create(IVinyl)
>>> kool.artist = u"Kool & the Gang"
>>> kool.title = u"Music Is the Message"
>>> kool.rpm = 33>>> collection.records.append(kool)
>>> [record.artist for record in collection.records]
[u'Diana Ross and The Supremes', u'Kool & the Gang']>>> session.flush()
>>> session.update(collection)We can remove items.>>> collection.records.remove(kool)
>>> len(collection.records) == 1
TrueAnd extend.>>> collection.records.extend((kool,))
>>> len(collection.records) == 2
TrueItems can appear twice in the list.>>> collection.records.append(kool)
>>> len(collection.records) == 3
TrueWe can add concrete instances to collections.>>> marvin = Vinyl()
>>> marvin.artist = u"Marvin Gaye"
>>> marvin.title = u"Let's get it on"
>>> marvin.rpm = 33>>> collection.records.append(marvin)
>>> len(collection.records) == 4
TrueAnd remove them, too.>>> collection.records.remove(marvin)
>>> len(collection.records) == 3
TrueThe standard list methods are available.>>> collection.records = [marvin, vinyl]
>>> collection.records.sort(key=lambda record: record.artist)
>>> collection.records
[<Vinyl Diana Ross and The Supremes: Taking Care of Business (@ 45 RPM)>,
 <Vinyl Marvin Gaye: Let's get it on (@ 33 RPM)>]>>> collection.records.reverse()
>>> collection.records
[<Vinyl Marvin Gaye: Let's get it on (@ 33 RPM)>,
 <Vinyl Diana Ross and The Supremes: Taking Care of Business (@ 45 RPM)>]>>> collection.records.index(vinyl)
1>>> collection.records.pop()
<Vinyl Diana Ross and The Supremes: Taking Care of Business (@ 45 RPM)>>>> collection.records.insert(0, vinyl)
>>> collection.records
[<Vinyl Diana Ross and The Supremes: Taking Care of Business (@ 45 RPM)>,
 <Vinyl Marvin Gaye: Let's get it on (@ 33 RPM)>]>>> collection.records.count(vinyl)
1>>> collection.records[1] = vinyl
>>> collection.records.count(vinyl)
2For good measure, let’s create a new instance without adding any
elements to its list.>>> empty_collection = create(ICollection)
>>> session.save(empty_collection)To demonstrate the mapping implementation, let’s set up a catalog for
our record collection. We’ll index the records by their ASIN string.>>> class ICatalog(interface.Interface):
...     index = schema.Dict(
...         title=u"Record index")>>> catalog = create(ICatalog)
>>> session.save(catalog)Add a record to the index.>>> catalog.index[u"B00004WZ5Z"] = diana
>>> catalog.index[u"B00004WZ5Z"]
<Mapper (__builtin__.IVinyl) at ...>Verify state after commit.>>> transaction.commit()
>>> catalog.index[u"B00004WZ5Z"]
<Mapper (__builtin__.IVinyl) at ...>Let’s check that the standard dict methods are supported.>>> catalog.index.values()
[<Mapper (__builtin__.IVinyl) at ...>]>>> tuple(catalog.index.itervalues())
(<Mapper (__builtin__.IVinyl) at ...>,)>>> catalog.index.setdefault(u"B00004WZ5Z", None)
<Mapper (__builtin__.IVinyl) at ...>>>> catalog.index.pop(u"B00004WZ5Z")
<Mapper (__builtin__.IVinyl) at ...>>>> len(catalog.index)
0Concrete instances are supported.>>> vinyl = Vinyl()
>>> vinyl.artist = diana.artist
>>> vinyl.title = diana.title
>>> vinyl.rpm = diana.rpm>>> catalog.index[u"B00004WZ5Z"] = vinyl
>>> len(catalog.index)
1>>> catalog.index.popitem()
(u'B00004WZ5Z',
 <Vinyl Diana Ross and The Supremes: Taking Care of Business (@ 45 RPM)>)>>> catalog.index = {u"B00004WZ5Z": vinyl}
>>> len(catalog.index)
1>>> catalog.index.clear()
>>> len(catalog.index)
0We may use a mapped object as index.>>> catalog.index[diana] = diana
>>> catalog.index.keys()[0] == diana.uuid
True>>> transaction.commit()>>> catalog.index[diana]
<Mapper (__builtin__.IVinyl) at ...>>>> class IDiscography(ICatalog):
...     records = schema.Dict(
...         title=u"Discographies by artist",
...         value_type=schema.List())Amorphic objectsWe can set and retrieve attributes that aren’t declared in an
interface.>>> record = create(interface.Interface)>>> record.publisher = u"Columbia records"
>>> record.publisher
u'Columbia records'>>> session.save(record)
>>> session.query(record.__class__).filter_by(
...     uuid=record.uuid)[0].publisher
u'Columbia records'Using this kind of weak we can store (almost) any kind of
structure. Values are kept as Python pickles.>>> favorite = create(interface.Interface)
>>> session.save(favorite)A transaction hook makes sure that assigned values are transient
during a session.>>> obj = object()
>>> favorite.item = obj
>>> favorite.item is obj
TrueIntegers, floats and unicode strings are straight-forward.>>> favorite.item = 42; transaction.commit()
>>> favorite.item
42>>> favorite.item = 42.01; transaction.commit()
>>> 42 < favorite.item <= 42.01
True>>> favorite.item = u"My favorite number is 42."; transaction.commit()
>>> favorite.item
u'My favorite number is 42.'Normal strings need explicit coercing tostr.>>> favorite.item = "My favorite number is 42."; transaction.commit()
>>> str(favorite.item)
'My favorite number is 42.'Or sequences of items.>>> favorite.item = (u"green", u"blue", u"red"); transaction.commit()
>>> favorite.item
(u'green', u'blue', u'red')Dictionaries.>>> favorite.item = {u"green": 0x00FF00, u"blue": 0x0000FF, u"red": 0xFF0000}
>>> transaction.commit()
>>> favorite.item
{u'blue': 255, u'green': 65280, u'red': 16711680}>>> favorite.item[u"black"] = 0x000000
>>> sorted(favorite.item.items())
[(u'black', 0), (u'blue', 255), (u'green', 65280), (u'red', 16711680)]We do need explicitly set the dirty bit of this instance.>>> favorite.item = favorite.item
>>> transaction.commit()>>> sorted(favorite.item.items())
[(u'black', 0), (u'blue', 255), (u'green', 65280), (u'red', 16711680)]When we create relations to mutable objects, a hook is made into the
transaction machinery to keep track of the pending state.>>> some_list = [u"green", u"blue"]
>>> favorite.item = some_list
>>> some_list.append(u"red"); transaction.commit()
>>> favorite.item
[u'green', u'blue', u'red']Amorphic structures.>>> favorite.item = ((1, u"green"), (2, u"blue"), (3, u"red")); transaction.commit()
>>> favorite.item
((1, u'green'), (2, u'blue'), (3, u'red'))Structures involving relations to other instances.>>> favorite.item = vinyl; transaction.commit()
>>> favorite.item
<Vinyl Diana Ross and The Supremes: Taking Care of Business (@ 45 RPM)>Self-referencing works because polymorphic attributes are lazy.>>> favorite.item = favorite; transaction.commit()
>>> favorite.item
<z3c.dobbin.bootstrap.Soup object at ...>SecurityThe security model from Zope is applied to mappers.>>> from zope.security.checker import getCheckerForInstancesOfOurVinylclass does not have a security checker defined.>>> from z3c.dobbin.mapper import getMapper
>>> mapper = getMapper(Vinyl)>>> getCheckerForInstancesOf(mapper) is None
TrueLet’s set a checker and regenerate the mapper.>>> from zope.security.checker import defineChecker, CheckerPublic
>>> defineChecker(Vinyl, CheckerPublic)>>> from z3c.dobbin.mapper import createMapper
>>> mapper = createMapper(Vinyl)
>>> getCheckerForInstancesOf(mapper) is CheckerPublic
TrueKnown limitationsCertain names are disallowed, and will be ignored when constructing
the mapper.>>> class IKnownLimitations(interface.Interface):
...     __name__ = schema.TextLine()>>> from z3c.dobbin.interfaces import IMapper
>>> mapper = IMapper(IKnownLimitations)
>>> mapper.__name__
'Mapper'CleanupCommit session.>>> transaction.commit()Change log0.4dev0.4.2Compatibility with SQLAlchemy 0.5rc1.0.4.1Fixed version issue.0.4.0Added patch to support old-style mixin classes in the inheritance
tree of a mapper class.All attributes that are not declared as interface names are now
persisted automatically using the Pickle-protocol. The exception to
this rule is attributes starting with the characters “_v_” (volatile
attributes).Changed target to SQLAlchemy 0.5-series.0.3.2Use pickles to store polymorphic attributes; there’s no benefit in
using native columns for amorphic data.Dobbin now useszope.sqlalchemyfor transaction and session
glue.0.3.1Use native UUID column type (available on PostgreSQL); compatibility
with SQLite is preserved due to its weak typing.Basic type factories are now registered as components.0.3.0Implemented rest of list methods.Refactoring of table bootstrapping; internal tables now using a
naming convention less likely to clash with existing tables.Added support forschema.Dict(including polymorphic dictionary
relation).Implemented polymorphic relations for a subset of the basic types
(int, str, unicode, tuple and list).0.2.9Tables are now only created once per minimal interface; this fixes
issue on both SQLite and Postgres when we create mappers with an
explicit polymorphic class.First entry in change-log.