Why Copy-on-Write Semantics and Node-Level-Versioning are Key to Efficient Snapshots
By: Johannes Lichtenberger
Tags: Versioning, Xml, Database, Temporal, Json

Life is subdued to constant evolution. So is our data, be it in research, business or personal information management. As such it is surprising that databases usually just keep the current state.

Introduction

Usually database systems simply either overwrite data in-place or do a copy-on-write operation followed by a removal of the outdated data (the latter maybe some time later from a background process). Data however naturally evolves over time. It is often times of great value to keep the history. We for instance might record the payroll of an employee at the first of march in 2019. Let us say it is 5000€ / month. Then as of 15th April we notice, that the recorded payroll was wrong and correct it to 5300€. Now, what is the answer to what the payroll was on the 1st of march in 2019? Database Systems, which only preserve the most recent version do not even know that the payroll wasn’t right. Our answer to this question depends on what source we consider most authoritative: the record or reality? The fact that they disagree effectively splits this payroll event into two tracks of time, rendering neither source entirely accurate. Questions such as these might be easily answered by temporal database systems such as Sirix. We provide at all times the system/ transaction time, which is set, once a transaction commits (when is a fact valid in the database / the record). Application or valid time has to be set by the application itself (when is a fact valid in the real world/reality?). We first want to motivate the usage of Sirix through some example use cases and afterwards dive into technical aspects and concepts.

Data Audits

Thus, one usage scenario for Sirix is data auditing. We keep track of past revisions in our specialized index structure for (bi)temporal data and our novel sliding snapshot versioning algorithm, which balances read- and write-performance while avoiding any peaks (our system is very space efficient and depending on the versioning algorithm we only copy changed records plus maybe a few more during writes). Thus we for instance usually do not simply copy a whole database page if only a single record has changed; we also do not have to cluster data during every commit as in B+-trees and LSM-Trees), Furthermore, we can simply keep track of changes through hooks and our user defined secondary index structures to avoid potentially expensive diffing of whole resources (even though we can utilize hashes, which can optionally be built during update operations for integrity checks, and thus skip the comparison of whole sub-tree if the hashes and stable, unique node identifiers match). As already mention we can use an ID-based diff algorithm if we do not want to keep track of what changed in an additional index. We never allow to override or delete old revisions (that said we might look into how to prune old revisions if that is ever needed). A single read/write transaction appends data at all times. For sure you can revert to a specific revision and commit the new revision, but all revisions in-between will be accessible for data audits. Thus, we are able to help to answer who changed what and when.

Time travel queries

Data audits are about how specific records/nodes have changed. Time Travel queries can answer questions as these, but also allows to reconstruct a whole sub-tree as it looked at a specific time or during a specific time span, or how the whole document/resource changed over time. You might want to analyze the past to predict the future. Through additional temporal XPath axis and XQuery functions we encourage you to investigate how your data has evolved.

An example query might look like:

This opens the database mycol.jn and therein the document/resource mydoc.jn as it was seen by the database at a specific time, thus it loads a snapshot/specific revision. This example opens and queries a JSON-database/resource, but you can also create native XML-databases. We then search for statuses with a datetime, which is newer / greater than the 1st February of 2018, whereby the status didn’t exist in the previous revision. We’re then outputting the revision-number and the text of the found status (projection). You’re also able to open a resource in between a given time-span and receive a sequence of JSON items as a result, which represents a list of revisions in this time-span. This is just a simple example: We provide many more temporal functions.

Fixing application or human errors / simple undo/redo operations / rollbacks / reproduce experiments

You can simple revert to a specific revision/point in time where everything was in a known good state and commit the revision again, or you might simply select a specific record/node and correct the error and commit the new revision.

How are all these usage scenarios possible? Let’s turn our focus to the question why historical data hasn’t been retained in the past and how new storage advances in recent years made it possible, to build sophisticated solutions to help answer these questions without the hurdle, state-of-the-art systems bring.

Why storing historical data becomes feasible nowadays

As Marc Kramis points out in his paper “Growing Persistent Trees into the 21st Century”:

The switch to flash drives keenly motivates to shift from the “current state” paradigm towards remembering the evolutionary steps leading to this state.

From Marc Kramis: Modifications evolving the state of your data.

The main insight is that flash drives as for instance SSDs, which are common nowadays have zero seek time while not being able to do in-place modifications of the data. Flash drives are organized into pages and blocks. Due to their characteristics they are able to read data on a fine-granular page-level, but can only erase data at the coarser block-level. Furthermore, blocks first have to be erased, before they can be updated. Thus, updated data is written to another place. A garbage collector marks the data, which has been rewritten to the new place as erased at the previous block location, such that new data can be stored in the future. Metadata to find the data at the new location are updated.

Thus, it is possible to store fine granular changes nowadays which naturally evolve the state of the data.

Search/insertion/deletion-operations of a temporal database should be in logarithmic time (O(log(n)), to compete with commonly used index structures.

The temporal storage system Sirix.io

Sirix.io is a temporal storage system, which has been inspired by the file system ZFS and the versioning system Git amongst others. It is based on copy-on-write semantics and node-level versioning and thus never overwrites data.

Let’s first define what a temporal database system is all about.

A temporal database is capable of storing and retrieving past states of your data efficiently. Typically it stores the transaction time, when the data actually is committed to the database. If we also store the valid time, that is how long a fact is true in the real world, we have a bitemporal relation, that is two time-axis, the transaction time and the valid time.

Sirix has been developed to answer questions such as: Give me last month’s history of the Dollar-Pound Euro exchange rate. What was the customers address on February 12th in 2019 as it was recorded back in the day? Did they move or did we correct an error? Did we have errors in the database, which were corrected later on?

Concepts / architecture of Sirix.io

We support N read-only transactions, which are bound to a single revision (each transaction might be started on any past revision) concurrently to one write transaction on a single resource. Our system thus is based on snapshot isolation. The write-transaction can revert the most recent revision to any past revision. Changes to this past revision can then be commit to create a new snapshot and thus a new revision. The revisions in-between are never lost. We wrote our storage manager from scratch, thus let us now shift our focus to the internals of Sirix.

The page-structure for one revision is depicted in the following figure. Note that during each transaction commit a new revision root page with its sub-tree is created whereas unchanged pages are simply referenced at their original file-offset positions.