Module Expr Str ImportFrom alias Import alias alias alias alias alias alias alias alias alias Import alias Assign NameStore Call AttributeLoad NameLoad attr Str Assign NameStore Str Assign NameStore Str Assign NameStore Str TryExcept body Import alias identifier handlers ExceptHandler type NameLoad body Import alias ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults Str Num Num body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr BinOpAdd AttributeLoad NameLoad attr Str Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr BinOpAdd AttributeLoad NameLoad attr Str Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr BinOpAdd AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr AttributeStore NameLoad attr AttributeStore NameLoad attr AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad AttributeLoad NameLoad attr attr TryExcept body Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr handlers ExceptHandler body Pass TryExcept body Assign NameStore Call NameLoad AttributeLoad NameLoad attr Str handlers ExceptHandler body Assign NameStore Call NameLoad AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr TryExcept body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Call AttributeLoad Call NameLoad AttributeLoad NameLoad attr Str attr handlers ExceptHandler body Expr Call AttributeLoad NameLoad attr Str Assign AttributeStore NameLoad attr Dict Assign AttributeStore NameLoad attr AttributeStore NameLoad attr AttributeStore NameLoad attr AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr decorator_list FunctionDef arguments args NameParam defaults body If UnaryOpNot AttributeLoad NameLoad attr body Return Expr Call AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad Call NameLoad AttributeLoad NameLoad attr Str attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr BinOpMod Str TupleLoad AttributeLoad NameLoad attr Call NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad AttributeLoad NameLoad attr attr TryExcept body Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr handlers ExceptHandler type NameLoad body Pass TryExcept body Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr handlers ExceptHandler type NameLoad body Pass Expr Call AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr If CompareGt NameLoad BinOpAdd AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr If CompareGt NameLoad BinOpAdd AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Raise Call NameLoad Str decorator_list FunctionDef arguments args NameParam NameParam defaults body Raise Call NameLoad Str decorator_list FunctionDef arguments args NameParam NameParam defaults body Raise Call NameLoad Str decorator_list FunctionDef arguments args NameParam NameParam defaults body Raise Call NameLoad Str decorator_list FunctionDef arguments args NameParam NameParam defaults body Raise Call NameLoad Str decorator_list FunctionDef arguments args NameParam NameParam defaults body Raise Call NameLoad Str decorator_list FunctionDef arguments args NameParam defaults body Return BinOpMod Str AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If UnaryOpNot AttributeLoad NameLoad attr body Return Expr Call AttributeLoad AttributeLoad NameLoad attr attr For NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body If CompareLt SubscriptLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Index Num NameLoad body Delete SubscriptDel AttributeLoad NameLoad attr Index NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr decorator_list FunctionDef arguments args NameParam NameParam defaults body If UnaryOpNot AttributeLoad NameLoad attr body Raise NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr TryExcept body Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr Return NameLoad handlers ExceptHandler type NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Raise NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str If UnaryOpNot AttributeLoad NameLoad attr body Return Expr Call AttributeLoad AttributeLoad NameLoad attr attr TryExcept body Assign NameStore Call NameLoad AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Num Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameStore Call AttributeLoad NameLoad attr Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad ListLoad NameLoad Call NameLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr handlers ExceptHandler body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Raise NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If UnaryOpNot AttributeLoad NameLoad attr body Return Expr Call AttributeLoad AttributeLoad NameLoad attr attr TryExcept body Delete SubscriptDel AttributeLoad NameLoad attr Index NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr handlers ExceptHandler body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Raise NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If UnaryOpNot AttributeLoad NameLoad attr body Raise NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr TryExcept body Assign NameStore Call NameLoad AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr SubscriptLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Index Num Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Index Num Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr handlers ExceptHandler body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Raise NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If UnaryOpNot AttributeLoad NameLoad attr body Raise NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr TryExcept body Assign NameStore SubscriptLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Index Num Expr Call AttributeLoad AttributeLoad NameLoad attr attr Return NameLoad handlers ExceptHandler body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Raise NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore Call NameLoad AttributeLoad NameLoad attr Str Assign NameStore Call NameLoad AttributeLoad NameLoad attr Str Assign NameStore Dict Assign NameStore Num For NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body Expr Call AttributeLoad NameLoad attr SubscriptLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Index Num Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr SubscriptLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Index Num Assign SubscriptStore NameLoad Index NameLoad ListLoad Call AttributeLoad NameLoad attr SubscriptLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Index Num NameLoad Assign NameStore BinOpAdd NameLoad Num Expr Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr BinOpMod Str TupleLoad AttributeLoad NameLoad attr NameLoad NameLoad decorator_list decorator_list If CompareEq NameLoad Str body Assign NameStore Call NameLoad Str keyword Num keyword Num Assign SubscriptStore NameLoad Index Str Str For NameStore Call NameLoad Num Num body Expr Call AttributeLoad NameLoad attr Num Assign SubscriptStore NameLoad Index Str Dict Str Num Assign SubscriptStore NameLoad Index Str Str Assign SubscriptStore NameLoad Index Str Str Expr Call AttributeLoad NameLoad attr BinOpSub Call AttributeLoad NameLoad attr Num Delete SubscriptDel NameLoad Index Str Assert CompareEq SubscriptLoad NameLoad Index Str Str
N/A N/A 
Haystack.py

An on-disk cache with a dict-like API,inspired by Facebook's Haystack store

Created by Rui Carmo on 2010-04-05
Published under the MIT license.
 __future__ generators N/A os sys stat string urllib codecs time thread traceback N/A logging N/A log N/A N/A logging getLogger Snakelets.logger N/A __author__ Rui Carmo http://the.taoofmac.com N/A __revision__ $Id$ N/A __version__ 0.8 N/A N/A N/A cPickle pickle N/A N/A N/A ImportError N/A N/A pickle Haystack N/A dict N/A __init__ N/A N/A self path basename commit compact N/A haystack 300 3600 N/A N/A N/A self enabled True N/A N/A self mutex N/A N/A thread allocate_lock N/A N/A self commitinterval commit N/A N/A self compactinterval compact N/A N/A self path N/A N/A N/A os path normpath path N/A N/A self basename basename N/A N/A self cache N/A N/A N/A os path join N/A self path N/A N/A self basename .bin N/A N/A self index N/A N/A N/A os path join N/A self path N/A N/A self basename .idx N/A N/A self temp N/A N/A N/A os path join N/A self path N/A N/A self basename .tmp N/A N/A N/A self _rebuild N/A N/A self created N/A self modified N/A self compacted N/A self committed N/A N/A time time N/A N/A N/A dict __init__ self N/A _rebuild N/A N/A self N/A N/A N/A N/A N/A N/A self mutex acquire N/A N/A N/A N/A N/A os makedirs N/A self path N/A N/A N/A N/A N/A N/A N/A cache N/A open N/A self cache rb N/A N/A N/A N/A cache N/A open N/A self cache ab N/A N/A N/A cache close N/A N/A N/A N/A self _index N/A N/A pickle loads N/A N/A N/A open N/A self index rb read N/A N/A N/A N/A N/A N/A log debug Index retrieval from disk failed. N/A N/A self _index N/A N/A N/A self created N/A self modified N/A self compacted N/A self committed N/A N/A time time N/A N/A N/A N/A self mutex release N/A commit N/A N/A self N/A N/A N/A N/A N/A self enabled N/A N/A N/A N/A N/A N/A self mutex acquire N/A N/A N/A N/A open N/A self index wb write N/A N/A pickle dumps N/A self _index N/A N/A self committed N/A N/A time time N/A N/A N/A log info N/A Index %s commited, %d items. N/A N/A self index N/A len N/A N/A N/A self _index keys N/A N/A N/A N/A self mutex release N/A purge N/A N/A self N/A N/A N/A N/A N/A N/A self mutex acquire N/A N/A N/A N/A N/A os unlink N/A self index N/A N/A N/A OSError N/A N/A N/A N/A N/A N/A N/A os unlink N/A self cache N/A N/A N/A OSError N/A N/A N/A N/A N/A N/A self mutex release N/A N/A N/A self _rebuild N/A _cleanup N/A N/A self N/A N/A N/A N/A N/A log debug Cleanup check. N/A now N/A N/A time time N/A N/A now N/A N/A self committed N/A self commitinterval N/A N/A N/A N/A self commit N/A N/A now N/A N/A self compacted N/A self compactinterval N/A N/A N/A N/A self _compact N/A __eq__ N/A N/A self other N/A N/A N/A N/A TypeError Equality undefined for this kind of dictionary N/A __ne__ N/A N/A self other N/A N/A N/A N/A TypeError Equality undefined for this kind of dictionary N/A __lt__ N/A N/A self other N/A N/A N/A N/A TypeError Comparison undefined for this kind of dictionary N/A __le__ N/A N/A self other N/A N/A N/A N/A TypeError Comparison undefined for this kind of dictionary N/A __gt__ N/A N/A self other N/A N/A N/A N/A TypeError Comparison undefined for this kind of dictionary N/A __ge__ N/A N/A self other N/A N/A N/A N/A TypeError Comparison undefined for this kind of dictionary N/A __repr__ N/A N/A self N/A N/A N/A N/A <yaki '%s'> N/A self index N/A expire N/A N/A self when N/A N/A N/A 
    Remove from cache any items older than a specified time
     N/A N/A N/A self enabled N/A N/A N/A N/A N/A N/A self mutex acquire N/A k N/A N/A N/A self _index keys N/A N/A N/A N/A N/A N/A self _index N/A k N/A 0 when N/A N/A N/A N/A self _index N/A k N/A N/A N/A N/A self mutex release N/A N/A N/A self _cleanup N/A keys N/A N/A self N/A N/A N/A N/A N/A N/A self _index keys N/A stats N/A N/A self key N/A N/A N/A N/A N/A self enabled N/A N/A KeyError N/A N/A N/A N/A self mutex acquire N/A N/A N/A stats N/A N/A self _index N/A key N/A N/A N/A N/A self mutex release N/A stats N/A N/A N/A KeyError N/A N/A N/A N/A N/A self mutex release N/A KeyError N/A __setitem__ N/A N/A self key val N/A N/A N/A 
    Store an item in the cache
    Errors will cause the entire cache to be rebuilt
     N/A N/A N/A self enabled N/A N/A N/A N/A N/A N/A self mutex acquire N/A N/A N/A cache N/A open N/A self cache ab N/A buffer N/A N/A pickle dumps val 2 N/A offset N/A N/A cache tell N/A N/A N/A cache write buffer N/A N/A self modified mtime N/A N/A time time N/A N/A N/A self _index N/A key N/A mtime N/A len buffer offset N/A N/A N/A cache flush N/A N/A N/A cache close N/A N/A N/A N/A self mutex release N/A N/A N/A N/A N/A N/A N/A self mutex release N/A IOError N/A __delitem__ N/A N/A self key N/A N/A N/A 
    Remove item from cache
    In this case,we only remove it from the index
     N/A N/A N/A self enabled N/A N/A N/A N/A N/A N/A self mutex acquire N/A N/A N/A N/A N/A self _index N/A key N/A N/A N/A N/A self mutex release N/A N/A N/A N/A N/A N/A N/A self mutex release N/A KeyError N/A __getitem__ N/A N/A self key N/A N/A N/A 
    Retrieve item from the cache
     N/A N/A N/A self enabled N/A N/A KeyError N/A N/A N/A N/A self mutex acquire N/A N/A N/A cache N/A open N/A self cache rb N/A N/A N/A cache seek N/A N/A N/A self _index N/A key N/A 2 N/A buffer N/A N/A cache read N/A N/A N/A self _index N/A key N/A 1 N/A item N/A N/A pickle loads buffer N/A N/A N/A N/A self mutex release N/A N/A N/A N/A N/A N/A N/A self mutex release N/A KeyError N/A item N/A mtime N/A N/A self key N/A N/A N/A 
    Return the creation/modification time of a cache item
     N/A N/A N/A self enabled N/A N/A KeyError N/A N/A N/A N/A self mutex acquire N/A N/A N/A item N/A N/A N/A self _index N/A key N/A 0 N/A N/A N/A N/A self mutex release N/A item N/A N/A N/A N/A N/A N/A N/A self mutex release N/A KeyError N/A _compact N/A N/A self N/A N/A N/A N/A N/A N/A self mutex acquire N/A cache N/A open N/A self cache rb N/A compacted N/A open N/A self temp ab N/A newindex N/A N/A i 0 N/A key N/A N/A N/A self _index keys N/A N/A N/A N/A cache seek N/A N/A N/A self _index N/A key N/A 2 N/A offset N/A N/A compacted tell N/A N/A N/A compacted write N/A N/A cache read N/A N/A N/A self _index N/A key N/A 1 N/A N/A newindex N/A key N/A N/A N/A time time N/A N/A N/A self _index N/A key N/A 1 offset N/A i N/A i 1 N/A N/A N/A cache close N/A size N/A N/A compacted tell N/A N/A N/A compacted flush N/A N/A N/A os fsync N/A N/A compacted fileno N/A N/A N/A compacted close N/A N/A N/A os rename N/A self temp N/A self cache N/A N/A self compacted N/A N/A time time N/A N/A self _index newindex N/A N/A N/A N/A self mutex release N/A N/A N/A self commit N/A N/A N/A log debug N/A Compacted %s: %d items into %d bytes N/A N/A self cache i size N/A N/A N/A N/A __name__ __main__ N/A N/A c N/A Haystack . commit 3 compact 4 N/A N/A c N/A tired to expire in 2 seconds N/A i N/A range 1 10 N/A N/A N/A N/A time sleep 1 N/A N/A c N/A foo N/A a 1 N/A N/A c N/A zbr 42 N/A N/A c N/A test/path/name test N/A N/A N/A c expire N/A N/A N/A time time 2 N/A N/A c N/A foo N/A N/A N/A c N/A zbr 42