Package: audite
Audite: instant change feeds for SQLiteAudite uses SQL triggers to add a transactional change feed to any SQLite
database. It gives you a totally-ordered audit history of all changes to your
data, without touching your application code or running an extra process.Use casesTrack who changed what, whenRestore previous versions of changed rowsReplicate data to external systems by streaming the change feedQuick startLet's add a changefeed totodo.db, a SQLite database with the following schema:sqlite3todo.db"CREATE TABLE project (id INTEGER PRIMARY KEY, name TEXT)"sqlite3todo.db"CREATE TABLE task (name TEXT PRIMARY KEY,project_id INTEGER REFERENCES project (project_id),done BOOLEAN NOT NULL DEFAULT FALSE)"Install audite on your sytempython3-mpipinstallauditeEnable audite on your databasepython3-mauditetodo.dbDone! From now on, any process canINSERT,UPDATE, andDELETEfrom your
database as usual, and audite's triggers will store the results aschange
eventsin theaudite_changefeedtable. All (and only)
committed transactions will appear in the change feed.Modfying data and querying the change feedWe'll add a project and two tasks...sqlite3todo.db"INSERT INTO project (id, name) VALUES (1, 'goals')"sqlite3todo.db"INSERT INTO task (project_id, name) VALUES (1, 'try audite'), (1, 'profit')"cross one task off the list...sqlite3todo.db"UPDATE task SET done = TRUE WHERE name = 'try audite'"and cancel the other:sqlite3todo.db"DELETE FROM task WHERE name = 'profit'"Now let's see what changed:sqlite3todo.db"SELECT * FROM audite_changefeed ORDER BY id"You should get back something like this:id  source   subject     type             time        specversion  data                                                                                                     
--  -------  ----------  ---------------  ----------  -----------  ---------------------------------------------------------------------------------------------------------
1   project  1           project.created  1669730365  1.0          {"new":{"name":"goals","id":1}}                                                                          
2   task     try audite  task.created     1669730374  1.0          {"new":{"project_id":1,"done":0,"name":"try audite"}}                                                    
3   task     profit      task.created     1669730374  1.0          {"new":{"project_id":1,"done":0,"name":"profit"}}                                                        
4   task     try audite  task.updated     1669730381  1.0          {"new":{"project_id":1,"done":1,"name":"try audite"},"old":{"project_id":1,"done":0,"name":"try audite"}}
5   task     profit      task.deleted     1669730386  1.0          {"old":{"project_id":1,"done":0,"name":"profit"}}Event SchemaThe event schema follows theCloudEvents
specso that other systems can easily
handle events from yours.iduniquely identifies the event with a monotonically increasing integer.sourceis name of the database table that changed.subjectis the primary key of the database row that changed.typedescribes the type of change:*.created,*.updated, or*.deleted.timeis the Unix time when the change was committed.specversionis the verion of theCloudEvents specin use, currently1.0.datais a JSON snapshot of the row that changed. Thedata.newobject holds the post-change values and is present for*.createdand*.updatedevents. Thedata.oldobject holds pre-change values and is present for*.updatedand*.deletedevents.Note:Audite storesidandtimeas integers so that SQLite can store
and sort them efficiently, but the CloudEvents spec mandates strings. To query
events that conform exactly to theCloudEvents JSON
spec,
select from theaudite_cloudeventview instead of the underlyingaudite_changefeedtable:sqlite3todo.db"SELECT cloudevent FROM audite_cloudevent ORDER BY id"cloudevent                                                                                                                                                                                                                                                                                                            
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
{"id":"1","sequence":"00000000000000000001","source":"project","subject":"1","type":"project.created","time":"2022-11-29T13:59:25+00:00","specversion":"1.0","datacontenttype":"application/json","data":{"new":{"name":"goals","id":1}}}                                                                             
{"id":"2","sequence":"00000000000000000002","source":"task","subject":"try audite","type":"task.created","time":"2022-11-29T13:59:34+00:00","specversion":"1.0","datacontenttype":"application/json","data":{"new":{"project_id":1,"done":0,"name":"try audite"}}}                                                    
{"id":"3","sequence":"00000000000000000003","source":"task","subject":"profit","type":"task.created","time":"2022-11-29T13:59:34+00:00","specversion":"1.0","datacontenttype":"application/json","data":{"new":{"project_id":1,"done":0,"name":"profit"}}}                                                            
{"id":"4","sequence":"00000000000000000004","source":"task","subject":"try audite","type":"task.updated","time":"2022-11-29T13:59:41+00:00","specversion":"1.0","datacontenttype":"application/json","data":{"new":{"project_id":1,"done":1,"name":"try audite"},"old":{"project_id":1,"done":0,"name":"try audite"}}}
{"id":"5","sequence":"00000000000000000005","source":"task","subject":"profit","type":"task.deleted","time":"2022-11-29T13:59:46+00:00","specversion":"1.0","datacontenttype":"application/json","data":{"old":{"project_id":1,"done":0,"name":"profit"}}}Handling database schema changesWhen your database schema changes, you need to re-run audite for the triggers to
pick up the latest fields. It's safe to re-run audite multiple times, including
as part of your schema migration scripts or even on app startup.When your database schema hasn't changed, then re-running audite does nothing.
When your schemahaschanged, then re-running audite rebuilds the triggers to
write to the change feed with the latest schema.Auditing only particular tablesBy default, audite tracks all tables in the target database. But you can specify
tables to track via the--tableargument:python3-mauditeblog.db--tablepost--tablecommentDependenciesAudite is a python package with no dependencies. You need Python >= 3.7 to
enable audite on a database, but because "enable audite on a database" just
means "add some SQL triggers," you don't need Python after the triggers are
installed.Prior Artlitestreamby
@benbjohnson makes a convincing case for using SQLite in production.supa_auditby @supabase
demonstrates how easy change feeds can be in Postgres.marmotby @maxpert uses schema
introspection and triggers that directly inspired the approach here.