Package: kvlayer
kvlayer=======``kvlayer`` is a database abstraction layer providing a simple key-value store to applications.  For development purposes this can run against an in-memory implementation or a server such as [Redis](http://redis.io/); in test and production, this can be switched to a relational database such as [PostgreSQL](http://postgresql.org/) or a cluster database such as [Accumulo](http://accumulo.apache.org/) or [Riak](http://basho.com/riak/).Configuration-------------``kvlayer`` depends on the [Yakonfig](https://github.com/diffeo/yakonfig/) library to get its configuration information.  Configuration is in a YAML file passed into the application.  This includes a *storage type*, indicating which backend to use, and an *application name* and a *namespace*, both of which distinguish different applications sharing the same database.```yamlkvlayer:storage_type: local  # in-memory data storeapp_name: kvlayernamespace: kvlayer```kvlayer API-----------Applications see multiple kvlayer *tables*, which may be implemented as database-native tables for databases that have that concept.  Each row has a key and a value.  The keys are Python tuples, with some consistent set of types; tuple parts may be strings, integers, or UUIDs.  Values are always Python byte strings.There are four basic operations kvlayer makes available.  ``put()`` writes one or more key-value pairs into the database.  ``get()`` retrieves key-value pairs with known fixed keys.  ``scan()`` retrieves key-value pairs within a range of keys.  ``delete()`` removes specific keys.A minimal kvlayer application would look like:```pythonimport argparseimport kvlayerimport yakonfigparser = argparse.ArgumentParser()yakonfig.parse_args(parser, [yakonfig, kvlayer])kvl = kvlayer.client()kvl.setup_namespace({'table': (str,)})# Write valueskvl.put('table', (('foo',), 'one'), (('bar',), 'two'))# Retrieve valuesfor k,v in kvl.get('table', ('foo',)):assert k == 'foo'print v# Scan valuesfor k,v in kvl.scan('table', (('a',), ('e',))):print kprint v# Scan keysfor k in kvl.scan_keys('table', (('e',), ('z',))):print k# Delete valueskvl.delete('table', ('foo',))```Other notes-----------See details of [testing on Accumulo using saltstack](accumulo-tests.md).For throughput testing, see [kvlayer_throughput_tests](https://github.com/diffeo/kvlayer/blob/0.4.5/kvlayer/tests/test_throughput.py).For example, using various single-node EC2 instances, randomreads/writes experiences these rates:| num_workers | storage_type | read MB/sec | write MB/sec |   ||-------------|--------------|-------------|--------------|---|| 100         | redis        | 99.6        | 57.3         |m1.xlarge   || 50          | redis        | 93.7        | 56.5         |m1.xlarge   || 25          | redis        | 66.9        | 33.8         |m1.xlarge   || 80          | postgres     | 34.2        | 14.4         |m1.medium   || 50          | postgres     | 33.1        | 14.1         |m1.medium   || 25          | postgres     | 30.1        | 13.7         |m1.medium   || 100         | accumulo     | 17.2        | 13.6         |m1.large   || 50          | accumulo     | 21.9        | 16.0         |m1.large   || 25          | accumulo     | 24.7        | 16.6         |m1.large   |TODO: gather more stats.