Package: python-tkvdb
python-tkvdbPython-tkvdb is a Cython wrapper fortkvdbtrie key-value
database. Python 3 is required.Code isn't well tested in production environments.InstallationThis is a typical python/cython package that uses setuptools build
system.From PyPiThe most simple way of installing is usingpip:pip install python-tkvdbConsidering that package is using Cython, C compiler may be required
for building if suitable wheel for current platform isn't found.DownloadingOriginal tkvdb sources are included as submodule, so when installing
from git please usegit clone --recursivefor cloning, or init
submodules after withgit submodule update --init --recursive. Custom sources may also be provided intkvdb/subdirectory before build.Source archives that were made withpython setup.py sdistcontain
generated C-code without.pyxfiles as Cython official documentation
recommends. Cython header files (.pxd) still provided though.Package also can be distributed as python wheels. With wheels no
additional post-installation actions are required.Initialization and usage of virtualenv or alternatives aren't
properly described in this manual, so use them by you own discretion.BuildingProject uses Cython to generate C extension for Python. To build
package, you need to have C compiler and build tools.For installation from the source archive, Cython isn't required, but
git versions require it. Source directory also includespyproject.toml(PEP 518), so if your build tool uses it, Cython
would be installed anyway.To make common tasks easier, project contains simple Makefile that may
be used instad of pip/python commands. It isn't a requirement, sopip install .also works. For additional reference, look intoMakefile.Install example with virtualenv:cd python-tkvdb/
python -m venv env
source ./env/bin/activate
makeBoth Makefile and setup.py usesUSE_CYTHONenv variable (int, 0/1)
to determine if Cython (cythonize) would be started before extension
building. Cython needs to be installed in local environment for
this. Default value is 1 for make.Makefile also hasPYTHONenv var that allows overriding python
interpreter path. Default value is justpython.Example usage:USE_CYTHON=1 PYTHON=./env/bin/python make buildMake commands:build-- just build extension withpython setup.py build_ext.install-- runpip install. Extension would be built if needed.no arguments (justmake) -- alias forinstall.dist-- create wheel and sdist archive.test-- run unit testsclean-- remove generated code, compiled objects and distribution
archives.uninstall-- remove previously installed package (throughpip)After installing moduletkvdbmust be importable in the Python environment.UsageOriginaltkvdbuses pretty specific terminology for some actions
(like transaction), so it is recommended to first consult with
original documentation anyway. Thread-safety notes and some caveats
are also described in the original README file.Python-tkvdb provides pythonic-style wrapper for most parts of the original
library.Most object have destructors that callfreeinternally and do memory
management on garbage collection, but don't forget that this is
wrapper for C library.Some code examples may also be found intests code.ModulesProject is splitted into multiple python modules:ctkvdb-- Cython wrapper with C definitions fromtkvdb.h.tkvdb.db-- database object and initialization. Also imported in__init__.py(i.e. maintkvdbmodule).tkvdb.transaction-- transaction (actually main input/output.
interface). Wrapper aroundtkvdb_tr.tkvdb.cursor-- transaction cursors for iteration. Wrapper aroundtkvdb_cursor.tkvdb.iterators-- pythonic iterators fortkvdb.cursor.tkvdb.errors-- all db-related exceptions that code may throw.tkvdb.params-- database and transaction params. Wrapper aroundtkvdb_params.Database initializationDatabase is wrapped into theTkvdbobject fromtkvdbortkvdb.dbmodules. At this time only path to database file is
supported. Parameters (tkvdb.params.Params) optionally may be passed
to constructor.fromtkvdbimportTkvdbdb=Tkvdb(path_to_db_file)# some codedb.close()Context manager (withstatment) that includes auto closing is also
available:withTkvdb(path_to_db_file)asdb:# some codewithdb.transaction()astr:# more codeAttributes (readonly):path: str-- path to database file.is_opened: bool-- shows that database is initialized properly.Methods (may raise exceptions):Tkvdb(path: str, params: tkvdb.params.Params = None)(constructor)
-- create database instance.close()-- close database.transaction(params: tkvdb.params.Params = None) -> tkvdb.transaction.Transaction-- create transaction.There is also Cython methodget_dbthat returnstkvdb_db *pointer.TransactionsTransactions are basic way to do any operation with database. Consult
with original documentation about transaction term, because it doesn't
mean same thing as in other database systems.Input and ouput usesbytestype for everything. Encode and
decode strings if needed.Parameters (tkvdb.params.Params) optionally may be passed to
constructor.Transaction must be created from database instance (described in
previous part):transaction=db.transaction()transaction.begin()transaction.put(b'key',b'value')transaction.commit()# or transaction.rollback()print(transaction.getvalue(b'key'))transaction.free()Pythonicwithstatment also available:withdb.transaction()astr:tr.put(b'key',b'value')tr.commit()print(tr.getvalue(b'key'))Note thatwithstatementdoes not do commit, but rollbacks on
exception. Docommitorrollbackwith your own code, or don't do
anything (implies rollback-like behavior). Transaction is started
(begin) automatically and will be freed (free) at exit fromwithblock though.Transaction also has Python dict-like interface:__getitem__and__setitem__get(key, default=None)keys(),values()anditems()iteratorswithdb.transaction()astr:tr[b'key']=b'value'print(tr.get(b'other-key',b'default'))# prints b'default'tr.commit()print(tr[b'key'])# prints b'value'# Iteratorsforkeyintr:# or tr.keys()print(key)forkey,valueintr.items():print(key,value)forvalueintr.values():print(value)Multiple keys may be deleted bydeletemethod using optionalprefixargument (defaultFalse). PassingTrueallows deleting
keys starting from prefix. Dict-likedeloperator always deletes
only exact key.Attributes (readonly):is_initialized: bool-- shows that transaction underlying
structures are initialized properly.is_started: bool-- shows thatbegin()method was called.is_changed: bool-- shows that transaction had any uncommited
changes (i.e.put()was used).ram_only: bool-- indicates that transaction is RAM-only.Transaction methods. Most of them may raise an exception:Transaction(ram_only=True, params: tkvdb.params.Params = None)(constructor) -- create transaction instance. Must be called
manually only for RAM-only usage, otherwisedb.transaction()must
be used instead.begin()-- starts transaction, calls underlyingtkvdb_tr->begin().getvalue(key: bytes) -> bytes-- get value by key.put(key: bytes)-- insert value into db by key.get(key: bytes, default: bytes = None) -> bytes-- dict-like get with
default value.delete(key: bytes, prefix=False)-- delete value by key. Withprefix=Trueall keys staring withkeywill be deleted.__getitem__,__setitem__,__delitem__-- dict-like methods.__contains__-- allows usage ofinoperator.free()-- free transaction (called inwithstatement
automatically).keys(),values(),items()-- return dict-like iterators.cursor(seek_key=None, seek_type=Seek.EQ)-- return transaction
cursor (seeCursors), with optional seek.RAM-only transactionsTransactions also may be used in RAM-only mode. These transactions don't
require database file and use less memory. They are cleared oncommit()orrollback(). See more about RAM-only transactions in
original documentation.Usetkvdb.Transaction()constructor to create RAM-only transaction
without database. This transaction may also be used withwithstatement, same auto-begin rules apply. Example:withTransaction()astr:tr[b'key']=b'value'print(tr[b'key'])# prints b'value'tr.commit()# clears transactionIteratorsTransaction can be traversed using iterators. It is also the main way
for iterating through database contents.Moduletkvdb.iteratorsprovides three dict-like iterators that usetkvdb.cursor.Cursorinside:tkvdb.iterators.KeysIterator-- iterating over keys.tkvdb.iterators.ValuesIterator-- iterating over values.tkvdb.iterators.ItemsIterator-- iterating over key-value pair.They can be used with transaction:withdb.transaction()astr:forkeyintr:# or tr.keys()print(tr[key])forvalueintr.values():print(value)forkey,valueintr.items():print(key,value)In all loops new instance ofCursoris used.They also can be used with cursor:withdb.transaction()astr:withtr.cursor()asc:forkeyinc:print(c.key(),c.keysize())Notice that cursor iterators use same underlyingCursorobject, so
they would iterate from same place where cursor stopped before:withdb.transaction()astr:withtr.cursor()asc:c.first()# do some iteration with c.next()forkeyinc:print(key)# it wouldn't be first keyifsomething:breakforvalueinc.values():# starts from last iterated keyprint(value)Reverse iteration is available through the standardreversedfunction. Iterators and transaction have required methods for this.
For new cursors iteration will start from the end usingtkvdb.cursor.Cursor.last.withdb.transaction()astr:forkeyinreversed(tr):print(key)# All iterator types allow thisforkey,valueinreversed(tr.items()):print(key,value)withdb.transaction()astr:withtr.cursor()asc:forkeyinreversed(c):print(key)withtr.cursor()asc:forvalueinreversed(c.values()):print(value)Cursors created from transacton can used for searching (as shorthand
fortkvdb.cursor.Cursor.seek()). More information in next section.withdb.transaction()astr:withtr.cursor(seek_key=b'seek-tr-31',seek_type=Seek.GE)asc:print(c.key())CursorsCursors are used to iterate through database contents. They are
defined intkvdb.cursormodule, C implementation is wrapped intkvdb.cursor.Cursorclass.Cursors are attached to transaction and created byTransacion.cursor()method. They also may be created directly.Although cursors are sole way to iterate and seek in tkvdb, it is
better and easier to use python-style iterators for
such purposes.Example usage:withdb.transaction()astr:withtr.cursor()asc:c.first()whileTrue:print(c.key(),c.value())try:c.next()excepttkvdb.errors.NotFoundError:breakCursor also may be used withoutwithstatement, it would be freed
anyway on garbage collection:withdb.transaction()astr:c=tr.cursor():c.first()# ...Notice:firstandnextmethods throwtkvdb.errors.EmptyErroron
empty database, notNotFoundError. Cursors may be iterated by using
iterators (see previous section).Cursor also may be iterated in reverse order usingprev()method. Another method calledlast()moves cursor to last record and
often useful for the reverse iteration.Cursor can be used for search withseek()method. This allows
searching k-v pair by prefix using seek criteria. Criterias are
defined intkvdb.cursor.Seekenum:Seek.EQ-- search for the exact key match.Seek.LE-- search for less (in terms of memcmp()) or equal key.Seek.GE-- search for greater (in terms of memcmp()) or equal key.Seeking is also may be initiated usingTransaction.cursor()method
(see more inTransactionssection).fromtkvdb.cursorimportSeekwithdb.transaction()astr:withtr.cursor()asc:c.seek(b'key',Seek.EQ)key=c.key()# ...c.next()withtr.cursor(seek_key=b'key')asc:# ...Attributes (readonly):is_initialized: bool-- shows that cursor underlying
structures are initialized properly.is_started: bool-- shows thatfirst()method was called.Cursor methods.first()-- move cursor to first item in database.last()-- move cursor to last item in database.next()-- move cursor to next item in database.prev()-- move cursor to previous item in database.key() -> bytes-- get current key.val() -> bytes-- get current value.keysize() -> int-- get current key size.valsize() -> int-- get current value size.free()-- free cursor.__iter__()-- returnstkvdb.iterators.KeysIterator.seek(key: bytes, seek: tkvdb.cursor.Seek)-- search key by
criteria.keys(),values(),items()-- return dict-like iterators.ParamsParams are used to specify different options for database and/or
transactions. They are defined intkvdb.paramsmodule: the C
implementation (tkvdb_paramsstruct) is wrapped bytkvdb.params.Paramsclass, and param values are wrapped by thetkvdb.params.Paramenum.Parameter names transfomed using CamelCase, example:TKVDB_PARAM_TR_DYNALLOC => TrDynalloc
TKVDB_PARAM_CURSOR_STACK_DYNALLOC => CursorStackDynallocAvailable parameters intkvdb.params.Paramenum:TrDynalloc--TKVDB_PARAM_TR_DYNALLOCTrLimit--TKVDB_PARAM_TR_LIMITAlignval--TKVDB_PARAM_ALIGNVALAutobegin--TKVDB_PARAM_AUTOBEGINCursorStackDynalloc--TKVDB_PARAM_CURSOR_STACK_DYNALLOCCursorStackLimit--TKVDB_PARAM_CURSOR_STACK_LIMITCursorKeyDynalloc--TKVDB_PARAM_CURSOR_KEY_DYNALLOCCursorKeyLimit--TKVDB_PARAM_CURSOR_KEY_LIMITDbfileOpenFlags--TKVDB_PARAM_DBFILE_OPEN_FLAGSConsult with originaltkvdbdocumentation for params meaning and
possible values.Params usage:fromtkvdb.paramsimportParams,Param# Passing params to databaseparams=Params({Param.Autobegin:1})# set params at initwithTkvdb(path,params)asdb:# Params also will be passed to db transactionswithdb.transaction()astr:# ...# Transaction may use own paramsparams=Params()params.set(Param.TrLimit,100)withdb.transaction(params)astr:# ...# Setting params after initparams=Params()params.set(Param.Autobegin,1)tr=Transaction(params=params)# RAM-only transactionAs in originaltkvdbcode, params from database are passed to
db-bound transaction, if they aren't overrided directly by passing
anotherParamsinstance to transaction.Python implementation also stores all set params and stores them in
internalvaluesdict. Notice that it tracks only params that were
set directly, so default values aren't known toParamswrapper.Params attributes:is_initialized: bool-- shows that params underlying structures
are initialized properly.Params methods:Params(params=None)(constructor) -- create params
instance. Argumentparamsmay be dict with param names and values.get_values()-- return all set params.get(param: tkvdb.params.Param)-- return single param value.set(param: tkvdb.params.Param, value: int)-- set param value.free()-- free params object.There is also Cython methodget_paramsthat returnstkvdb_params *pointer.ErrorsError classes are defined intkvdb.errorsmodule. Every non-ok
return value from the underlying C code is converted to python
Exception. OnlyTKVDB_RES.TKVDB_OKconsidered as success.Consult with original documentation for error codes meaning.One exception from this rule istkvdb.transaction.Transaction.__getitem__()(dict-like access) that
raisesKeyErrorfor python compatibility.Examples:fromtkvdb.errorsimportEmptyError,NotFoundError,NotStartedError# ...tr=db.transaction()try:print(tr.getvalue(b'key'))except(NotFoundError,EmptyError):print('key not found')exceptNotStartedError:print('transaction not started')withdb.transaction()astr:try:print(tr[b'key'])exceptKeyError:print('key not found')Note that tkvdb raisesEmptyError(TKVDB_RES.TKVDB_EMPTYreturn
code), notNotFoundErrorwhen key is not found in empty database,Errors:Error-- base class for all tkvdb-related errors.IoError--TKVDB_RES.TKVDB_IO_ERRORcode.LockedError--TKVDB_RES.TKVDB_LOCKEDcode.EmptyError--TKVDB_RES.TKVDB_EMPTYcode.NotFoundError--TKVDB_RES.TKVDB_NOT_FOUNDcode.EnomemError--TKVDB_RES.TKVDB_ENOMEMcode.CorruptedError--TKVDB_RES.TKVDB_CORRUPTEDcode.NotStartedError--TKVDB_RES.TKVDB_NOT_STARTEDcode.ModifiedError--TKVDB_RES.TKVDB_MODIFIEDcode.LicensePython-tkvdb is licensed under ISC license as original tkvdb project.