Package: python-store
StoreThis library provides aStoredatatype for Python. Each store looks and feels
like an ORM, but unlike an ORM, there is no database on the other end. Instead,
all data lives in memory, in the form of plain Python dicts and B-tree indices.
Stores supportSQL-likeselectstatementsin the style of SQLAlchemy,atomic transactionsandmultithreading.The source code aims to be rebustly documented, as we encourage open-source
collaboration on this Project.Use-casesLong-running interactive applications, like games.Automated trading systems with complex internal state management requirements.Stream-processing applications that perform fastad hocqueries on stream buffers.An ExampleImagine a system that generates user input events, likemouse clickandkey
press. In the following example, we deleteclick eventscreated after a
specified time and capitalize the character asssociated with eachkey presswithin a transaction.fromstoreimportStoreevents=Store()# insert fictitious "event" recordsevents.create_many([{'event_type':'press','char':'x','time':1},{'event_type':'click','button':'L','position':(5,8),'time':2},{'event_type':'click','button':'R','position':(3,4),'time':3},{'event_type':'press','char':'y','time':4},{'event_type':'press','char':'p','time':5},])withevents.transaction()astransaction:# delete "click" events after specified timetransaction.select().where(events.row.event_type=='click',events.row.time>2).delete()# capitalize the "char" for each selected "press" eventget_press_events=transaction.select().where(x.event_type=='press',x.char.one_of(['x','y','z']))foreventinget_press_events(dtype=list):event['char']=event['char'].upper()State DictsStore methods, likecreateandupdate, returnstate dicts. Unlike regular
dicts, any change to the keys or values of a state dict results in an update to
the store. For example, suppose thatuseris a state dict. As such,user['name'] ='John'generates a call tostore.updateunder the hood. When
this happens, any existing reference to the sameuserimmediately reflect this
change. There is no need to refresh each reference manually (as they are all
actually the same object). The same is true for other methods, likeupdate,setdefault, etc.Let's illustrate with an example:frank_1=store.create({'id':1,'name':'frank'})frank_2=store.get(1)# the store manages a singleton reference to frank's StateDict# in its internal so-called identity set.assertfrank_1isfrank_2# frank_1 and frank_2 are references to the same object,# so they should both reflect the same change.frank_1['name']='Franklin'assertfrank_2['name']=='Franklin'# likewise, any subsequent reference should reflect the same changefrank_3=store.get(1)assertfrank_3['name']=='Franklin'Stateful MethodsHere is a list of eachdictmethod that has been extended to result in an
update to store as a side-effect. On the lefthand side of each arrow is thedictmethod. On the righthand side is the correspondingstorecall.state.update(mapping)➞store.update(state, mapping.keys())state.setdefault(key, default)➞store.update(state, {key})state[key] = value➞store.update(state, {key})del state[key]➞store.delete(state, {key})IndexesBy default, allStateDictkeys are indexed, including those with non-scalar
values -- like lists, sets, dicts, etc. This means that that queries are fast.QueriesYou can query a store like a SQL database, usingselect,where,order_by,limitandoffsetconstraints.SymbolsSelect statements are written with the help of a class calledSymbol. A symbol
is a variable used to express what you want to select and how. Suppose you had a
store ofuserrecords. Then, using a symbol, You could write a query to
selects all users, created after a certain cut-off date.user=user_store.symbol()get_users=user_store.select(user.first_name,user.email).where(user.created_at>cutoff_date)foruseringet_users(dtype=list):send(message=f'Hello,{user["first_name"]}!',email=user['email'])An alternative to instantiating a new symbol for each query is to use a built-in
property,store.row. The following query is identical to the one above:get_users=user_store.select(user_store.row.first_name,user_store.row.email).where(user_store.row.created_at>cutoff_date)SelectBy default, an empty select will select everything, likeselect * from...in
SQL; however, if you're only interested in a subset of fields, you can
explicitly enumerate them.Selecting Everythingquery=store.select()Selecting Specific Fieldsquery=store.select(store.row.name,store.row.email)Where (Filtering)You can constrain queries to select only records whose values match a given
logical predicate. Predicates can be arbitrarily nested in compound boolean
expressions. This is similar to the "where" clause in SQL select statements.Filtering Non-scalars ValuesUnlike a SQL database, with a store, you can apply predicate logic not only to
scalar values, like numbers and strings, but also non-scalar types, like dicts,
lists, and sets.For example, this is possible:# imagine you have a store with user dicts, and each user dict# has a nested dog dict with an "age" value.get_users=store.select().where(store.row.dog<={'age':10})foruseringet_users():assertuser['dog']['age']<=10Using a symbol, here are some example:Conditional Expressionsuser=store.symbol()# equalitypredicate=(user.email=='elon.musk@gmail.com')predicate=(user.email!='elon.musk@gmail.com')# inequalitypredicate=(user.age>=50)# containmentpredicate=(user.favorite_color.in(['red','blue'])# logical conjunction (AND)predicate=(user.scent=='smelly')&(user.income<=20000)# logical disjunction (OR)predicate=(user.scent=='smelly')|(user.income<=20000)# logical conjunction and disjunction combinedpredicate=(((user.scent=='smelly')|(user.age<=20))&(user.name=='Bob'))Moreover, predicates can be built up gradually, like so:predicate=(user.age<=20)ifsome_condition:predicate&=(user.income>100000)# |= also worksOnce you have your predicate, you can pass it into a query'swheremethod:query=store.select().where((user.age<=20)|(user.is_member==True))Order ByQuery results can be sorted by one or more values using theorder_byquery
method. For example:# sort results by age (in ascending order) first# created_at date (in descending order) second.query=store.select().order_by(user.age.asc,user.created_at.desc)Ordering By Non-scalar ValuesUnlike SQL, the store can sort non-scalar datatypes, like dicts, lists, and sets
-- in addition to plain ints and strings. This means that you can do things like
-- this:store.create_many([{'owner':'Mohammed','dog':{'age':10}},{'owner':'Kang Bo','dog':{'age':6}},])get_users=store.select().order_by(store.row.dog.asc)users=get_users(dtype=list)foru1,u2inzip(users,users[1:]):assertu1.dog['age']<=u2.dog['age']Note that, when sorting a dict, the dict's items are sorted and compared in the
resulting order.Limit & OffsetQueries support pagination via limit and offset parameters. Thelimitparameter is anintthat determines the maximum number of records returned by
the query while theoffsetparameter determines the starting index of the
returned slice. When using limit and offset, it is important to specify an order, usingorder_by.query=store.select(user.email).order_by(user.age.desc).offset(20).limit(10)TransactionsStores support transactions as well. If, for some reason you don't already know,
a database transaction is a mechanism that allows you to perform multiple
operations as if they were all performed int a single step. This way, if one
operation fails, then they all fail, and the state of the store remains intact.
The syntax for creating transactions is straight forward:withuser_store.transaction()asuser_trans:# update the name of one user and delete anotherusers=user_trans.get_many([1,2])users[1]['name']='Updated Name'users[2].delete()At the end of thewithblock, the transaction commits; otherwise, if an
exception is raised, the transaction rolls back, clearing its internal state.Alternate to using thewithstatement,commitandrollbackmethods can be
called explicitly.user_trans=user_store.transaction()try:users=user_trans.get_many([1,2])users[1]['name']='Updated Name'users[2].delete()user_trans.commit()exceptException:user_trans.rollback()