heimdal/heimdal — pull request #13 by nicowilliams (2012-02-05 22:22:58+00:00)
Title: Pluggable libheimbase interface for DBs and misc libheimbase enhancement...

...s

[Code reviewed by Love Hörnquist Åstrand ]

Added heim_db_*() entry points for dealing with databases, and
make krb5_aname_to_localname() use it.

The following enhancements to libheimbase are included:

- Add heim_data_t and heim_string_t "reference" variants to
avoid memory copies of potentially large data/strings.

See heim_data_ref_create() and heim_string_ref_create().

- Added enhancements to heim_array_t to allow their use for
queues and stacks, and to improve performance. See
heim_array_insert_value().

- Added XPath-like accessors for heim_object_t. See
heim_path_get(), heim_path_copy(), heim_path_create(), and
heim_path_delete(). These are used extensively in the DB
framework's generic composition of ACID support and in the
test_base program

- Made libheimbase more consistent with Core Foundation naming
conventions. See heim_{dict, array}_{get, copy}_value() and
heim_path_{get, copy}().

- Added functionality to and fixed bugs in base/json.c:
- heim_serialize();
- depth limit for JSON parsing (for DoS protection);
- pretty-printing;
- JSON compliance (see below);
- flag options for parsing and serializing; these are needed
because of impedance mismatches between heim_object_t and
JSON (e.g., heim_dict_t allows non-string keys, but JSON
does not; heimbase supports binary data, while JSON does
not).

- Added heim_error_enomem().

- Enhanced the test_base program to test new functionality and
to use heim_path*() to better test JSON encoding. This
includes some fuzz testing of JSON parsing, and running the
test under valgrind.

- Started to add doxygen documentation for libheimbase (but doc
build for libheimbase is still incomplete).

Note that there's still some incomplete JSON support:

- JSON string quoting is not fully implemented;

- libheimbase lacks support for real numbers, while JSON has
it -- otherwise libheimbase is a superset of JSON,
specifically in that any heim_object_t can be a key for an
associative array.

The following DB backends are supported natively:

- "sorted-text", a binary search of sorted (in C locale), flat
text files;

- "json", a backend that stores DB contents serialized as JSON
(this is intended for configuration-like contents).

The DB framework supports:

- multiple key/value tables per-DB
- ACID transactions

The DB framework also natively implements ACID transactions for
any DB backends that a) do not provide transactions natively, b)
do provide lock/unlock/sync methods (even on Windows). This
includes autocommit of DB updates outside transactions.

Future DB enhancements may include:

- add backends for various DB types (BDB, CDB, MDB, ...);

- make libhdb use heim_db_t;

- add a command-line tool for interfacing to databases via
libheimbase (e.g., to get/set/delete values, create/copy/
backup DBs, inspect history, check integrity);

- framework-level transaction logging (with redo and undo
logging), for generic incremental replication;

- framework-level DB integrity checking.

We could store a MAC of the XOR of a hash function applied to
{key, value} for every entry in the DB, then use this to check
DB integrity incrementally during incremental replication, as
well as for the whole DB.