voldemort/voldemort — pull request #54 by jroper (2011-10-14 12:53:46+00:00)
Title: Added BDB native backup capabalities

This is a reissue of https://github.com/voldemort/voldemort/pull/51, which I've closed, rebased, and fixed some of the problems with the original pull request (such as excess IDE autoformatting), creating a new branch and applying my changes to that branch in a single commit.

Reading google, the recommended method for backing up a BDB voldemort store is to just copy the files while voldemort is running. Because BDB is an append only database, this should work, right? Wrong, BDB's clean up thread can get in the way and corrupt the backups. The Java BDB port provides a tool for assisting with backups, basically it finishes the current BDB file, and then pauses the clean up thread while the backup is running so that a safe backup can be done. This tool must be run within the same Java process for the correct locking to occur. For detailed documentation you can read here:

http://download.oracle.com/docs/cd/E17277_02/html/java/com/sleepycat/je/util/DbBackup.html

So, I've implemented an option into the voldemort admin client to do this, called native backup. Storage engines can declare themselves to support this by implementing NativeBackupable. I've implemented support for this in the BdbStorageEngine, which does fast NIO copies, and also supports incremental backups. Backups are guaranteed to be consistent.

I've written unit tests for the backup itself, there aren't any existing tests for the admin client/protocol, so I didn't write tests for the changes I made to the client.