Package: savage
savageA library built on top of the SQLAlchemy ORM for versioning row changes to PostgreSQL tables.Based onversionalchemyAuthor:Jeremy LewisWhy not use versionalchemy?versionalchemyexecutes four SQL statements for every versioned row that is inserted/updated/deleted:INSERT|UPDATE|DELETE: Insert/update/delete of the versioned rowSELECT max(va_version) ...: Selects the current maxva_versionfrom archive table based on rowINSERT ...: Inserts a new row into the archive table, withva_versionincremented from previous resultUPDATE ... SET va_id = ...: Update versioned row withva_id, returned after the previous result executesPostgreSQL has a couple of features that allow for a simpler implementation:RETURNING: PostgreSQL allows you to return server generated column values onINSERT/UPDATEtxid_current(): System function that returns a monotonically increasing 64-bit int ID for current transactionUtilizing these two features allows for a much simpler implementation. Instead of storingva_idon the archived
table, we storeversion_id(generated server-side usingtxid_current()) on both the archived and archive tables.
As a result, we don't need to select the max version (b/c it's handled server-side), and we don't need to update
the archive row witharchive_id.Getting StartedSample Usageimportsqlalchemyassafromsqlalchemyimportcreate_enginefromsqlalchemy.ext.declarativeimportdeclarative_basefromsqlalchemy.schemaimportUniqueConstraintfromsavageimportinitfromsavage.modelsimportSavageLogMixin,SavageModelMixinPOSTGRESQL_URL='<insert postgresql url here>'engine=create_engine(POSTGRESQL_URL)Base=declarative_base(bind=engine)classExample(Base,SavageModelMixin):__tablename__='example'version_columns=['id']id=sa.Column(sa.Integer,primary_key=True)value=sa.Column(sa.String(128))classExampleArchive(Base,SavageLogMixin):__tablename__='example_archive'__table_args__=(UniqueConstraint('id','version_id'),)id=sa.Column(sa.Integer)user_id=sa.Column(sa.Integer)init()# Only call this onceExample.register(ExampleArchive,engine)# Call this once per engine, AFTER init()LatencyWe compared the results ofbenchmark.pyto
a comparablebenchmark.pywritten for Savage. It times the performance of inserts using SQLAlchemy core, ORM
with and without version tracking, and (for Savage only) bulk inserts with versioning.The below stats were generated for 100,000 records using local Docker containers with MySQL and Postgres (average of 3 runs).Core InsertsORM InsertsVersioned ORMBulk VersioningVersionAlchemy/MySQL 5.6135 s.203 s.489 s.unsupportedSavage/Postgres 9.6154 s. (-12%)177 s. (+15%)283 s. (+73%)17.7 s. (+2,658%)VersionAlchemy: ~5 ms./recordSavage: ~3 ms./recordBulk insert/archive with Savage: ~180 µs./record (!!)Caveatstxid_current()depends on executing within a single transaction context.frommodelsimportdb,Exampleexample=Example(value='foo')withdb.session.begin():db.session.add(example)db.session.commit()example.value='bar'db.session.add(example)db.session.commit()# This will raise an IntegrityError because `txid_current()` hasn't changedNote that this is only an issue if you try to commit the same archived row multiple times within a single transaction.The following would work just fine:frommodelsimportdb,Exampleexample=Example(value='foo')db.session.add(example)db.session.commit()example.value='bar'db.session.add(example)db.session.commit()Why is it called Savage?SQLAlchemyVersionAlchemyPostgresStyleFollow PEP8 with a line length of 100 charactersPrefer parenthesis to\for line breaksLicenseMIT License