Package: rocksdict
RocksDict / SpeeDictKey-value storage for Python & Wrapper of Rocksdb and SpeedbInstallationWheels available, justpip install rocksdictfor rocksdb backend, thenfrom rocksdict import Rdictpip install speedictfor speedb backend, thenfrom speedict import RdictIntroductionThis library has two purposes.As an on-disk key-value storage solution for Python.As a RocksDB / Speedict interface.These two purposes operate in different modes:Default mode, which allows storingint,float,bool,str,bytes, and other python objects (withPickle).Raw mode(options=Options(raw_mode=True)),
which allows storing onlybytes.ExamplesA minimal examplefromrocksdictimportRdictimportnumpyasnpimportpandasaspdpath=str("./test_dict")# create a Rdict with default options at `path`db=Rdict(path)db[1.0]=1db["huge integer"]=2343546543243564534233536434567543db["good"]=Truedb["bytes"]=b"bytes"db["this is a list"]=[1,2,3]db["store a dict"]={0:1}db[b"numpy"]=np.array([1,2,3])db["a table"]=pd.DataFrame({"a":[1,2],"b":[2,1]})# reopen Rdict from diskdb.close()db=Rdict(path)assertdb[1.0]==1assertdb["huge integer"]==2343546543243564534233536434567543assertdb["good"]==Trueassertdb["bytes"]==b"bytes"assertdb["this is a list"]==[1,2,3]assertdb["store a dict"]=={0:1}assertnp.all(db[b"numpy"]==np.array([1,2,3]))assertnp.all(db["a table"]==pd.DataFrame({"a":[1,2],"b":[2,1]}))# iterate through all elementsfork,vindb.items():print(f"{k}->{v}")# batch get:print(db[["good","bad",1.0]])# [True, False, 1]# delete Rdict from dictdb.close()Rdict.destroy(path)An Example of Raw ModeThis mode allows only bytes as keys and values.fromrocksdictimportRdict,OptionsPATH_TO_ROCKSDB=str("path")# open raw_mode, which allows only bytesdb=Rdict(path=PATH_TO_ROCKSDB,options=Options(raw_mode=True))db[b'a']=b'a'db[b'b']=b'b'db[b'c']=b'c'db[b'd']=b'd'fork,vindb.items():print(f"{k}->{v}")# close and deletedb.close()Rdict.destroy(PATH_TO_ROCKSDB)New Feature Since v0.3.3Loading Options from RocksDict Path.Load Options and add A New ColumnFamilyfromrocksdictimportOptions,Rdictpath=str("./rocksdict_path")opts,cols=Options.load_latest(path)opts.create_missing_column_families(True)cols["bytes"]=Options()self.test_dict=Rdict(path,options=opts,column_families=cols)Reopening RocksDB Reads DB Options AutomaticallyimportshutilfromrocksdictimportRdict,Options,SliceTransform,PlainTableFactoryOptionsimportosdefdb_options():opt=Options()# create tableopt.create_if_missing(True)# config to more jobsopt.set_max_background_jobs(os.cpu_count())# configure mem-table to a large value (256 MB)opt.set_write_buffer_size(0x10000000)opt.set_level_zero_file_num_compaction_trigger(4)# configure l0 and l1 size, let them have the same size (1 GB)opt.set_max_bytes_for_level_base(0x40000000)# 256 MB file sizeopt.set_target_file_size_base(0x10000000)# use a smaller compaction multiplieropt.set_max_bytes_for_level_multiplier(4.0)# use 8-byte prefix (2 ^ 64 is far enough for transaction counts)opt.set_prefix_extractor(SliceTransform.create_max_len_prefix(8))# set to plain-tableopt.set_plain_table_factory(PlainTableFactoryOptions())returnopt# create DBdb=Rdict("./some_path",db_options())db[0]=1db.close()# automatic reloading all options on reopeningdb=Rdict("./some_path")assertdb[0]==1# destroydb.close()Rdict.destroy("./some_path")More Examples on BatchWrite, SstFileWrite, Snapshot, RocksDB Options, and etc.Go toexamplefolder.LimitationsCurrently, do not support merge operation and custom comparator.Full DocumentationSeerocksdict documentation.