BaseXdb/basex — Issue #279: Make BaseX Crash-Proof
Author: malamut2
Created: 2011-11-18 10:27:51+00:00
State: open

Ok, I realize that this is much to ask, but I'm filing this request anyway. :)

I have a Java program writing lots of stuff into a BaseX database. If I kill this program the hard way, like 'kill -9' on Linux or by pressing the 'Stop' button in Eclipse, so that my ShutdownHook with the Close() is not executed, I may end up with a broken database.

Such a broken database will not be recognized as 'broken' immediately. Simple queries like an auto-cropped '//*' from the BaseX GUI will usually still work. Complex queries, however, will produce problems like IndexOutOfBounds, NegativeArraySize, or OutOfMemory (the latter probably corresponding to 'RidiculouslyLargeArraySize' ;) ). Such a database will also result in errors when trying to export it.

In other words: there's no way to recover the data - simply hope there's a recent backup somewhere...

Ideally, BaseX would flush its written data to disk in such a sequence that no bigger problem than loss of the changes of the last few seconds can occur. At least, it should be possible to automatically recognize broken databases so that a crashed and restarted service can automatically recover from the latest backup. The lock file discussed in a separate issue can be a first step into this direction since it signals an unclosed database, but it would be great to have more than just that.