Package: rocksdbserver
A server exposing an RPC interface to RocksDBInstallation(original instructions fromhttp://pyrocksdb.readthedocs.org/en/latest/installation.html)install librocksdb.so$sudoapt-getinstallbuild-essential$sudoapt-getinstallpython-virtualenvpython-dev$sudoapt-getinstalllibsnappy-devzlib1g-devlibbz2-devlibgflags-dev$gitclonehttps://github.com/facebook/rocksdb.git$cdrocksdb$# It is tested with this version$gitcheckout3.9.fb$makeshared_lib$sudomvlibrocksdb.so/usr/lib/$sudomvinclude/*/usr/include/Install RocksDBServerrocksdbserver usesjq.pywhich requires the following$sudoapt-getinstallautoconfautomakebuild-essentiallibtoolpython-dev$sudopipinstallrocksdbserverUsageBasic UsageConsider this simple usage example from the examples/ directory.fromrocksdbserverimportRocksDBServer,RocksDBAPI,TableclassNamesTable(Table):NAME='names'classSimpleDBAPI(RocksDBAPI):defdefine_tables(self):names=NamesTable(self.data_dir,self)return{names.NAME:names}classSimpleDBServer(RocksDBServer):NAME='SimpleDBServer'DESC='Simple DB Server based on RockDB Server'defprepare_api(self):returnSimpleDBAPI(self.args.data_dir)if__name__=='__main__':SimpleDBServer().start()The above code represents a database with just one table called ‘names’.
Run this server by doingpythonexamples/simple_db.pyrun/tmp/dataThe argument /tmp/data points to the data directory where the db files
are stored. You can pick an alternate location too.Now visithttp://locahost:8889to use the web based console and try out
the followingImageThe above exercise demonstrates how to interact with the database via a
web based shell. Now let us try to interact with the database from a
Pythons script using the client API.Consider this sample client code from examples/simple_db_client.pyfromrocksdbserverimportRocksDBClientif__name__=='__main__':db=RocksDBClient('http://localhost:8889')printdb.put('names','prashanth',{'last_name':'ellina','days':10})key=db.put('names',None,{'last_name':'doe','days':12})printkeyprintdb.get('names','prashanth')printdb.get('names',key)Run this by doingpythonexamples/simple_db_client.pyThe output will look something like this$pythonsimple_db_client.pyprashanth238b74b0af8d11e3bcd3d43d7e99b40b{'last_name':'ellina','days':10,'_id':'prashanth'}{'last_name':'doe','days':12,'_id':'238b74b0af8d11e3bcd3d43d7e99b40b'}More details on usageDeletion>api.delete('names','prashanth')>api.get('names','prashanth')# In case you need to delete multiple keys at once, do>api.delete('names',['prashanth','238b74b0af8d11e3bcd3d43d7e99b40b'])# Let us add some data back>api.put('names',None,{'last_name':'ellina'})>api.put('names',None,{'last_name':'ellina1'})>api.put('names',None,{'last_name':'ellina2'})>help(api.delete_all)Helponmethoddelete_allinmodulerocksdbserver.rocksdbserver:delete_all(self,table,*args,**kwargs)methodof__main__.SimpleDBAPIinstanceDeletesallitemsfromthetable.Usewithcaution.Ifthetableisverylarge,thiscouldtakeasignificantamountoftime.>api.delete_all()IterationThe below session in the web-based console demonstrates iteration. Just
like the exercise above the very same API commands used in the web-based
console can be utilized from a client proxy.# Let us first create some records>api.put('names',None,{'city':'London'})'cc38f17ccca311e3aec5d43d7e99b40b'>api.put('names',None,{'city':'New York'})'d0541de0cca311e3aec5d43d7e99b40b'>api.put('names',None,{'city':'Boston'})'d32f14c0cca311e3aec5d43d7e99b40b'>api.put('names',None,{'city':'Frankfurt'})'d5ce57d6cca311e3aec5d43d7e99b40b'>api.put('names',None,{'city':'Singapore'})'d88c8acecca311e3aec5d43d7e99b40b'>help(api.list_keys)Helponmethodlist_keysinmodulerocksdbserver.rocksdbserver:list_keys(self,table,*args,**kwargs)methodof__main__.SimpleDBAPIinstanceListsallthekeysinthetable.ThisismeanttobeusedonlyduringdebuggingindevelopmentandneverinproductionasitloadsallthekeysintableintoRAMwhichmightcausememoryloadissuesforlargetables.>api.list_keys('names')['cc38f17ccca311e3aec5d43d7e99b40b','d0541de0cca311e3aec5d43d7e99b40b','d32f14c0cca311e3aec5d43d7e99b40b','d5ce57d6cca311e3aec5d43d7e99b40b','d88c8acecca311e3aec5d43d7e99b40b']>help(api.list_values)Helponmethodlist_valuesinmodulerocksdbserver.rocksdbserver:list_values(self,table,*args,**kwargs)methodof__main__.SimpleDBAPIinstanceListsallthevaluesinthetable.ThisismeanttobeusedonlyduringdebuggingindevelopmentandneverinproductionasitloadsallthevaluesintableintoRAMwhichmightcausememoryloadissuesforlargetables.>api.list_values('names')[{'city':'London','_id':'cc38f17ccca311e3aec5d43d7e99b40b'},{'city':'New York','_id':'d0541de0cca311e3aec5d43d7e99b40b'},{'city':'Boston','_id':'d32f14c0cca311e3aec5d43d7e99b40b'},{'city':'Frankfurt','_id':'d5ce57d6cca311e3aec5d43d7e99b40b'}, {'city': 'Singapore', '_id': 'd88c8acecca311e3aec5d43d7e99b40b'}]# list_keys and list_values are for usage for testing and development. For production# usage the following is the way to perform iteration.>iterK=api.iter_keys('names')>iterK'NcYAzfks0z'# iterK is a string that represents our current iterator whose state is maintained on the server.>api.tables['names'].iters{'NcYAzfks0z':<rocksdbserver.rocksdbserver.Iteratorobjectat0x7f98a8a03550>}# Before beginning the iteration we need to set the cursor location by seeking.# Let us seek to the beginning.>api.iter_seek_to_first('names',iterK)# Ask the API to send us the first two keys>api.iter_get('names',iterK,num=2)['cc38f17ccca311e3aec5d43d7e99b40b','d0541de0cca311e3aec5d43d7e99b40b']# And two more>api.iter_get('names',iterK,num=2)['d32f14c0cca311e3aec5d43d7e99b40b','d5ce57d6cca311e3aec5d43d7e99b40b']# And the more (only one is left now)>api.iter_get('names',iterK)['d88c8acecca311e3aec5d43d7e99b40b']# There are no more keys left to iterate over.>api.iter_get('names',iterK)[]# Cleanup the iterator state on the server# The server will garbage collect eventually but it is a good# practice to perform this action explicitly.>api.close_iter('names',iterK)>api.tables['names'].iters{}The above exercise shows how to iterate over keys usingiter_keysAPI method. You can useiter_valuesfor iteration over values anditer_itemsfor iterating over key-value item pairs.Client side iteration# Let us first create some records>api.put('names',None,{'city':'London'})'cc38f17ccca311e3aec5d43d7e99b40b'>api.put('names',None,{'city':'New York'})'d0541de0cca311e3aec5d43d7e99b40b'>api.put('names',None,{'city':'Boston'})'d32f14c0cca311e3aec5d43d7e99b40b'>api.put('names',None,{'city':'Frankfurt'})'d5ce57d6cca311e3aec5d43d7e99b40b'>api.put('names',None,{'city':'Singapore'})'d88c8acecca311e3aec5d43d7e99b40b'Now consider this python script that demonstrates client side iteration.fromrocksdbserverimportRocksDBClientif__name__=='__main__':db=RocksDBClient('http://localhost:8889')forkeyindb.iterkeys('names'):printkeyYou could do the same for values and item usingitervaluesanditeritemsrespectively. The client code usesiter_keys,iter_valuesanditer_itemsAPI methods internally.