Package: pottery-prod
Pottery: Redis for Humans 🌎🌍🌏Redisis awesome, butRedis
commandsare not always intuitive.  Pottery is a
Pythonic way to access Redis.  If you know how to use Python dicts, then you
already know how to use Pottery.  Pottery is useful for accessing Redis more
easily, and also for implementing microservice resilience patterns; and it has
been battle tested in production at scale.Table of ContentsDicts 📖Sets 🛍️Lists ⛓Counters 🧮Deques 🖇️Queues 🚶‍♂️🚶‍♀️🚶‍♂️Redlock 🔒synchronize() 👯‍♀️AIORedlock 🔒NextID 🔢redis_cache()CachedOrderedDictBloom filters 🌸HyperLogLogs 🪵ContextTimer ⏱️Installation$pip3installpotteryUsageFirst, set up your Redis client:>>>fromredisimportRedis>>>redis=Redis.from_url('redis://localhost:6379/1')>>>Dicts 📖RedisDictis a Redis-backed container compatible with Python’sdict.Here is a small example using aRedisDict:>>>frompotteryimportRedisDict>>>tel=RedisDict({'jack':4098,'sape':4139},redis=redis,key='tel')>>>tel['guido']=4127>>>telRedisDict{'jack':4098,'sape':4139,'guido':4127}>>>tel['jack']4098>>>deltel['sape']>>>tel['irv']=4127>>>telRedisDict{'jack':4098,'guido':4127,'irv':4127}>>>list(tel)['jack','guido','irv']>>>sorted(tel)['guido','irv','jack']>>>'guido'intelTrue>>>'jack'notintelFalse>>>Notice the first two keyword arguments toRedisDict():  The first is your
Redis client.  The second is the Redis key name for your dict.  Other than
that, you can use yourRedisDictthe same way that you use any other Pythondict.Limitations:Keys and values must be JSON serializable.Sets 🛍️RedisSetis a Redis-backed container compatible with Python’sset.Here is a brief demonstration:>>>frompotteryimportRedisSet>>>basket=RedisSet({'apple','orange','apple','pear','orange','banana'},redis=redis,key='basket')>>>sorted(basket)['apple','banana','orange','pear']>>>'orange'inbasketTrue>>>'crabgrass'inbasketFalse>>>a=RedisSet('abracadabra',redis=redis,key='magic')>>>b=set('alacazam')>>>sorted(a)['a','b','c','d','r']>>>sorted(a-b)['b','d','r']>>>sorted(a|b)['a','b','c','d','l','m','r','z']>>>sorted(a&b)['a','c']>>>sorted(a^b)['b','d','l','m','r','z']>>>Notice the two keyword arguments toRedisSet():  The first is your Redis
client.  The second is the Redis key name for your set.  Other than that, you
can use yourRedisSetthe same way that you use any other Pythonset.Do more efficient membership testing for multiple elements using.contains_many():>>>nirvana=RedisSet({'kurt','krist','dave'},redis=redis,key='nirvana')>>>tuple(nirvana.contains_many('kurt','krist','chat','dave'))(True,True,False,True)>>>Limitations:Elements must be JSON serializable.Lists ⛓RedisListis a Redis-backed container compatible with Python’slist.>>>frompotteryimportRedisList>>>squares=RedisList([1,4,9,16,25],redis=redis,key='squares')>>>squaresRedisList[1,4,9,16,25]>>>squares[0]1>>>squares[-1]25>>>squares[-3:][9,16,25]>>>squares[:][1,4,9,16,25]>>>squares+[36,49,64,81,100]RedisList[1,4,9,16,25,36,49,64,81,100]>>>Notice the two keyword arguments toRedisList():  The first is your Redis
client.  The second is the Redis key name for your list.  Other than that, you
can use yourRedisListthe same way that you use any other Pythonlist.Limitations:Elements must be JSON serializable.Under the hood, Python implementslistusing an array.  Redis implements
list using adoubly linked list.
As such, inserting elements at the head or tail of aRedisListis fast,
O(1).  However, accessingRedisListelements by index is slow, O(n).  So
in terms of performance and ideal use cases,RedisListis more similar to
Python’sdequethan Python’slist.  Instead ofRedisList,
consider usingRedisDeque.Counters 🧮RedisCounteris a Redis-backed container compatible with Python’scollections.Counter.>>>frompotteryimportRedisCounter>>>c=RedisCounter(redis=redis,key='my-counter')>>>c=RedisCounter('gallahad',redis=redis,key='my-counter')>>>c.clear()>>>c=RedisCounter({'red':4,'blue':2},redis=redis,key='my-counter')>>>c.clear()>>>c=RedisCounter(redis=redis,key='my-counter',cats=4,dogs=8)>>>c.clear()>>>c=RedisCounter(['eggs','ham'],redis=redis,key='my-counter')>>>c['bacon']0>>>c['sausage']=0>>>delc['sausage']>>>c.clear()>>>c=RedisCounter(redis=redis,key='my-counter',a=4,b=2,c=0,d=-2)>>>sorted(c.elements())['a','a','a','a','b','b']>>>c.clear()>>>RedisCounter('abracadabra',redis=redis,key='my-counter').most_common(3)[('a',5),('b',2),('r',2)]>>>c.clear()>>>c=RedisCounter(redis=redis,key='my-counter',a=4,b=2,c=0,d=-2)>>>fromcollectionsimportCounter>>>d=Counter(a=1,b=2,c=3,d=4)>>>c.subtract(d)>>>cRedisCounter{'a':3,'b':0,'c':-3,'d':-6}>>>Notice the first two keyword arguments toRedisCounter():  The first is your
Redis client.  The second is the Redis key name for your counter.  Other than
that, you can use yourRedisCounterthe same way that you use any other
PythonCounter.Limitations:Keys must be JSON serializable.Deques 🖇️RedisDequeis a Redis-backed container compatible with Python’scollections.deque.Example:>>>frompotteryimportRedisDeque>>>d=RedisDeque('ghi',redis=redis,key='letters')>>>forelemind:...print(elem.upper())GHI>>>d.append('j')>>>d.appendleft('f')>>>dRedisDeque(['f','g','h','i','j'])>>>d.pop()'j'>>>d.popleft()'f'>>>list(d)['g','h','i']>>>d[0]'g'>>>d[-1]'i'>>>list(reversed(d))['i','h','g']>>>'h'indTrue>>>d.extend('jkl')>>>dRedisDeque(['g','h','i','j','k','l'])>>>d.rotate(1)>>>dRedisDeque(['l','g','h','i','j','k'])>>>d.rotate(-1)>>>dRedisDeque(['g','h','i','j','k','l'])>>>RedisDeque(reversed(d),redis=redis)RedisDeque(['l','k','j','i','h','g'])>>>d.clear()>>>d.extendleft('abc')>>>dRedisDeque(['c','b','a'])>>>Notice the two keyword arguments toRedisDeque():  The first is your Redis
client.  The second is the Redis key name for your deque.  Other than that, you
can use yourRedisDequethe same way that you use any other Pythondeque.Limitations:Elements must be JSON serializable.Queues 🚶‍♂️🚶‍♀️🚶‍♂️RedisSimpleQueueis a Redis-backed multi-producer, multi-consumer FIFO queue
compatible with Python’squeue.SimpleQueue.
In general, use a Pythonqueue.Queueif you’re using it in one or more
threads, usemultiprocessing.Queueif you’re using it between processes,
and useRedisSimpleQueueif you’re sharing it across machines or if you
need for your queue to persist across application crashes or restarts.Instantiate aRedisSimpleQueue:>>>frompotteryimportRedisSimpleQueue>>>cars=RedisSimpleQueue(redis=redis,key='cars')>>>Notice the two keyword arguments toRedisSimpleQueue():  The first is your
Redis client.  The second is the Redis key name for your queue.  Other than
that, you can use yourRedisSimpleQueuethe same way that you use any other
Pythonqueue.SimpleQueue.Check the queue state, put some items in the queue, and get those items back
out:>>>cars.empty()True>>>cars.qsize()0>>>cars.put('Jeep')>>>cars.put('Honda')>>>cars.put('Audi')>>>cars.empty()False>>>cars.qsize()3>>>cars.get()'Jeep'>>>cars.get()'Honda'>>>cars.get()'Audi'>>>cars.empty()True>>>cars.qsize()0>>>Limitations:Items must be JSON serializable.Redlock 🔒Redlockis a safe and reliable lock to coordinate access to a resource shared
across threads, processes, and even machines, without a single point of
failure.Rationale and algorithm
description.Redlockimplements Python’s excellentthreading.LockAPI as closely as is feasible.  In other words, you can useRedlockthe same
way that you usethreading.Lock.  The main reason to useRedlockoverthreading.Lockis thatRedlockcan coordinate access to a resource shared
across different machines;threading.Lockcan’t.Instantiate aRedlock:>>>frompotteryimportRedlock>>>printer_lock=Redlock(key='printer',masters={redis},auto_release_time=.2)>>>Thekeyargument represents the resource, and themastersargument
specifies your Redis masters across which to distribute the lock.  In
production, you should have 5 Redis masters.  This is to eliminate a single
point of failure — you can lose up to 2 out of the 5 Redis masters and
yourRedlockwill remain available and performant.  Now you can protect
access to your resource:>>>ifprinter_lock.acquire():...# Critical section - print stuff here....print('printer_lock is locked')...printer_lock.release()printer_lockislocked>>>bool(printer_lock.locked())False>>>Or you can protect access to your resource inside a context manager:>>>withprinter_lock:...# Critical section - print stuff here....print('printer_lock is locked')printer_lockislocked>>>bool(printer_lock.locked())False>>>It’s safest to instantiate a newRedlockobject every time you need to
protect your resource and to not shareRedlockinstances across different
parts of code.  In other words, think of thekeyas identifying the resource;
don’t think of any particularRedlockas identifying the resource.
Instantiating a newRedlockevery time you need a lock sidesteps bugs by
decoupling how you useRedlockfrom the forking/threading model of your
application/service.Redlocks are automatically released (by default, after 10 seconds).  You
should take care to ensure that your critical section completes well within
that timeout.  The reasons thatRedlocks are automatically released are to
preserve“liveness”and to avoid deadlocks (in the event that a process dies inside a critical
section before it releases its lock).>>>importtime>>>ifprinter_lock.acquire():...# Critical section - print stuff here....time.sleep(printer_lock.auto_release_time)>>>bool(printer_lock.locked())False>>>If 10 seconds isn’t enough to complete executing your critical section,
then you can specify your own auto release time (in seconds):>>>printer_lock=Redlock(key='printer',masters={redis},auto_release_time=.2)>>>ifprinter_lock.acquire():...# Critical section - print stuff here....time.sleep(printer_lock.auto_release_time/2)>>>bool(printer_lock.locked())True>>>time.sleep(printer_lock.auto_release_time/2)>>>bool(printer_lock.locked())False>>>By default,.acquire()blocks indefinitely until the lock is acquired.  You
can make.acquire()return immediately with theblockingargument..acquire()returnsTrueif the lock was acquired;Falseif not.>>>printer_lock_1=Redlock(key='printer',masters={redis},auto_release_time=.2)>>>printer_lock_2=Redlock(key='printer',masters={redis},auto_release_time=.2)>>>printer_lock_1.acquire(blocking=False)True>>>printer_lock_2.acquire(blocking=False)# Returns immediately.False>>>printer_lock_1.release()>>>You can make.acquire()block but not indefinitely by specifying thetimeoutargument (in seconds):>>>printer_lock_1.acquire()True>>>printer_lock_2.acquire(timeout=printer_lock_1.auto_release_time/2)# Waits 100 milliseconds.False>>>importcontextlib>>>frompotteryimportReleaseUnlockedLock>>>withcontextlib.suppress(ReleaseUnlockedLock):...printer_lock_1.release()>>>You can similarly configure the Redlock context manager’s
blocking/timeout behavior during Redlock initialization.  If the context
manager fails to acquire the lock, it raises theQuorumNotAchievedexception.>>>importcontextlib>>>frompotteryimportQuorumNotAchieved>>>printer_lock_1=Redlock(key='printer',masters={redis},context_manager_blocking=True,context_manager_timeout=0.2)>>>printer_lock_2=Redlock(key='printer',masters={redis},context_manager_blocking=True,context_manager_timeout=0.2)>>>withprinter_lock_1:...withcontextlib.suppress(QuorumNotAchieved):...withprinter_lock_2:# Waits 200 milliseconds; raises QuorumNotAchieved....pass...print(f"printer_lock_1 is{'locked'ifprinter_lock_1.locked()else'unlocked'}")...print(f"printer_lock_2 is{'locked'ifprinter_lock_2.locked()else'unlocked'}")printer_lock_1islockedprinter_lock_2isunlocked>>>synchronize() 👯‍♀️synchronize()is a decorator that allows only one thread to execute a
function at a time.  Under the hood,synchronize()uses a Redlock, so refer
to theRedlock documentationfor more details.Here’s how to usesynchronize():>>>frompotteryimportsynchronize>>>@synchronize(key='synchronized-func',masters={redis},auto_release_time=1.5,blocking=True,timeout=-1)...deffunc():...# Only one thread can execute this function at a time....returnTrue...>>>func()True>>>AIORedlock 🔒AIORedlockis the asyncio implementation of Redlock, compatible with
Python’sasyncio.Lock.Instantiate anAIORedlockand protect a resource:>>>importasyncio>>>fromredis.asyncioimportRedisasAIORedis>>>frompotteryimportAIORedlock>>>asyncdefmain():...aioredis=AIORedis.from_url('redis://localhost:6379/1')...shower=AIORedlock(key='shower',masters={aioredis})...ifawaitshower.acquire():...# Critical section - no other coroutine can enter while we hold the lock....print(f"shower is{'occupied'ifawaitshower.locked()else'available'}")...awaitshower.release()...print(f"shower is{'occupied'ifawaitshower.locked()else'available'}")...>>>asyncio.run(main(),debug=True)showerisoccupiedshowerisavailable>>>Or you can protect access to your resource inside a context manager:>>>asyncio.set_event_loop(asyncio.new_event_loop())>>>asyncdefmain():...aioredis=AIORedis.from_url('redis://localhost:6379/1')...shower=AIORedlock(key='shower',masters={aioredis})...asyncwithshower:...# Critical section - no other coroutine can enter while we hold the lock....print(f"shower is{'occupied'ifawaitshower.locked()else'available'}")...print(f"shower is{'occupied'ifawaitshower.locked()else'available'}")...>>>asyncio.run(main(),debug=True)showerisoccupiedshowerisavailable>>>NextID 🔢NextIDsafely and reliably produces increasing IDs across threads, processes,
and even machines, without a single point of failure.Rationale and algorithm
description.Instantiate an ID generator:>>>frompotteryimportNextID>>>tweet_ids=NextID(key='tweet-ids',masters={redis})>>>Thekeyargument represents the sequence (so that you can have different
sequences for user IDs, comment IDs, etc.), and themastersargument
specifies your Redis masters across which to distribute ID generation (in
production, you should have 5 Redis masters).  Now, whenever you need a user
ID, callnext()on the ID generator:>>>next(tweet_ids)1>>>next(tweet_ids)2>>>next(tweet_ids)3>>>Two caveats:If many clients are generating IDs concurrently, then there may be
“holes” in the sequence of IDs (e.g.: 1, 2, 6, 10, 11, 21,
…).This algorithm scales to about 5,000 IDs per second (with 5 Redis masters).
If you need IDs faster than that, then you may want to consider other
techniques.redis_cache()redis_cache()is a simple lightweight unbounded function return value cache,
sometimes called“memoize”.redis_cache()implements Python’s excellentfunctools.cache()API as closely as is feasible.  In other words, you can useredis_cache()the
same way that you usefunctools.cache().Limitations:Arguments to the function must be hashable.Return values from the function must be JSON serializable.Just likefunctools.cache(),redis_cache()does not allow for a maximum
size, and does not evict old values, and grows unbounded.  Only useredis_cache()in one of these cases:Your function’s argument space has a known small cardinality.You specify atimeoutwhen callingredis_cache()to decorate your
function, to dump yourentirereturn value cachetimeoutseconds
after the last cache access (hit or miss).You periodically call.cache_clear()to dump yourentirereturn
value cache.You’re ok with your return value cache growing unbounded, and youunderstand the implicationsof this for your underlying Redis instance.In general, you should only useredis_cache()when you want to reuse
previously computed values.  Accordingly, it doesn’t make sense to cache
functions with side-effects or impure functions such astime()orrandom().Decorate a function:>>>importtime>>>frompotteryimportredis_cache>>>@redis_cache(redis=redis,key='expensive-function-cache')...defexpensive_function(n):...time.sleep(.1)# Simulate an expensive computation or database lookup....returnn...>>>Notice the two keyword arguments toredis_cache(): The first is your Redis
client.  The second is the Redis key name for your function’s return
value cache.Call your function and observe the cache hit/miss rates:>>>expensive_function(5)5>>>expensive_function.cache_info()CacheInfo(hits=0,misses=1,maxsize=None,currsize=1)>>>expensive_function(5)5>>>expensive_function.cache_info()CacheInfo(hits=1,misses=1,maxsize=None,currsize=1)>>>expensive_function(6)6>>>expensive_function.cache_info()CacheInfo(hits=1,misses=2,maxsize=None,currsize=2)>>>Notice that the first call toexpensive_function()takes 1 second and results
in a cache miss; but the second call returns almost immediately and results in
a cache hit.  This is because after the first call,redis_cache()cached the
return value for the call whenn == 5.You can access your original undecorated underlyingexpensive_function()asexpensive_function.__wrapped__.  This is useful for introspection, for
bypassing the cache, or for rewrapping the original function with a different
cache.You can force a cache reset for a particular combination ofargs/kwargswithexpensive_function.__bypass__.  A call toexpensive_function.__bypass__(*args, **kwargs)bypasses the cache lookup,
calls the original underlying function, then caches the results for future
calls toexpensive_function(*args, **kwargs).  Note that a call toexpensive_function.__bypass__(*args, **kwargs)results in neither a cache hit
nor a cache miss.Finally, clear/invalidate your function’s entire return value cache withexpensive_function.cache_clear():>>>expensive_function.cache_info()CacheInfo(hits=1,misses=2,maxsize=None,currsize=2)>>>expensive_function.cache_clear()>>>expensive_function.cache_info()CacheInfo(hits=0,misses=0,maxsize=None,currsize=0)>>>CachedOrderedDictThe best way that I can explainCachedOrderedDictis through an example
use-case.  Imagine that your search engine returns document IDs, which then you
have to hydrate into full documents via the database to return to the client.
The data structure used to represent such search results must have the
following properties:It must preserve the order of the document IDs returned by the search engine.It must map document IDs to hydrated documents.It must cache previously hydrated documents.Properties 1 and 2 are satisfied by Python’scollections.OrderedDict.
However,CachedOrderedDictextends Python’sOrderedDictto also
satisfy property 3.The most common usage pattern forCachedOrderedDictis as follows:InstantiateCachedOrderedDictwith the IDs that you must look up or
compute passed in as thedict_keysargument to the initializer.Compute and store the cache misses for future lookups.Return some representation of yourCachedOrderedDictto the client.Instantiate aCachedOrderedDict:>>>frompotteryimportCachedOrderedDict>>>search_results_1=CachedOrderedDict(...redis_client=redis,...redis_key='search-results',...dict_keys=(1,2,3,4,5),...)>>>Theredis_clientargument to the initializer is your Redis client, and theredis_keyargument is the Redis key for the Redis Hash backing your cache.
Thedict_keysargument represents an ordered iterable of keys to be looked up
and automatically populated in yourCachedOrderedDict(on cache hits), or
that you’ll have to compute and populate for future lookups (on cache
misses).  Regardless of whether keys are cache hits or misses,CachedOrderedDictpreserves the order ofdict_keys(like a list), maps
those keys to values (like a dict), and maintains an underlying cache for
future key lookups.In the beginning, the cache is empty, so let’s populate it:>>>sorted(search_results_1.misses())[1,2,3,4,5]>>>search_results_1[1]='one'>>>search_results_1[2]='two'>>>search_results_1[3]='three'>>>search_results_1[4]='four'>>>search_results_1[5]='five'>>>sorted(search_results_1.misses())[]>>>Note thatCachedOrderedDictpreserves the order ofdict_keys:>>>forkey,valueinsearch_results_1.items():...print(f'{key}:{value}')1:one2:two3:three4:four5:five>>>Now, let’s look at a combination of cache hits and misses:>>>search_results_2=CachedOrderedDict(...redis_client=redis,...redis_key='search-results',...dict_keys=(2,4,6,8,10),...)>>>sorted(search_results_2.misses())[6,8,10]>>>search_results_2[2]'two'>>>search_results_2[6]='six'>>>search_results_2[8]='eight'>>>search_results_2[10]='ten'>>>sorted(search_results_2.misses())[]>>>forkey,valueinsearch_results_2.items():...print(f'{key}:{value}')2:two4:four6:six8:eight10:ten>>>Limitations:Keys and values must be JSON serializable.Bloom filters 🌸Bloom filters are a powerful data structure that help you to answer the
questions,“Have I seen this element before?”and“How
many distinct elements have I seen?”; but not the question,“What
are all of the elements that I’ve seen before?”So think of Bloom
filters as Python sets that you can add elements to, use to test element
membership, and get the length of; but that you can’t iterate through or
get elements back out of.Bloom filters are probabilistic, which means that they can sometimes generate
false positives (as in, they may report that you’ve seen a particular
element before even though you haven’t).  But they will never generate
false negatives (so every time that they report that you haven’t seen a
particular element before, you really must never have seen it).  You can tune
your acceptable false positive probability, though at the expense of the
storage size and the element insertion/lookup time of your Bloom filter.Create aBloomFilter:>>>frompotteryimportBloomFilter>>>dilberts=BloomFilter(...num_elements=100,...false_positives=0.01,...redis=redis,...key='dilberts',...)>>>Here,num_elementsrepresents the number of elements that you expect to
insert into yourBloomFilter, andfalse_positivesrepresents your
acceptable false positive probability.  Using these two parameters,BloomFilterautomatically computes its own storage size and number of times
to run its hash functions on element insertion/lookup such that it can
guarantee a false positive rate at or below what you can tolerate, given that
you’re going to insert your specified number of elements.Insert an element into theBloomFilter:>>>dilberts.add('rajiv')>>>Test for membership in theBloomFilter:>>>'rajiv'indilbertsTrue>>>'raj'indilbertsFalse>>>'dan'indilbertsFalse>>>See how many elements we’ve inserted into theBloomFilter:>>>len(dilberts)1>>>Note thatBloomFilter.__len__()is an approximation, not an exact value,
though it’s quite accurate.Insert multiple elements into theBloomFilter:>>>dilberts.update({'raj','dan'})>>>Do more efficient membership testing for multiple elements using.contains_many():>>>tuple(dilberts.contains_many('rajiv','raj','dan','luis'))(True,True,True,False)>>>Remove all of the elements from theBloomFilter:>>>dilberts.clear()>>>len(dilberts)0>>>Limitations:Elements must be JSON serializable.len(bf)is probabilistic in that it’s an accurate approximation.  You
can tune how accurate you want it to be with thenum_elementsandfalse_positivesarguments to.__init__(), at the expense of storage space
and insertion/lookup time.Membership testing against a Bloom filter is probabilistic in that itmayreturn false positives, butneverreturns false negatives.  This means that
ifelement in bfevaluates toTrue, then youmayhave inserted the
element into the Bloom filter.  But ifelement in bfevaluates toFalse,
then youmust nothave inserted it.  Again, you can tune accuracy with thenum_elementsandfalse_positivesarguments to.__init__(), at the
expense of storage space and insertion/lookup time.HyperLogLogs 🪵HyperLogLogs are an interesting data structure designed to answer the question,“How many distinct elements have I seen?”; but not the questions,“Have I seen this element before?”or“What are all of the
elements that I’ve seen before?”So think of HyperLogLogs as
Python sets that you can add elements to and get the length of; but that you
can’t use to test element membership, iterate through, or get elements
out of.HyperLogLogs are probabilistic, which means that they’re accurate within
a margin of error up to 2%.  However, they can reasonably accurately estimate
the cardinality (size) of vast datasets (like the number of unique Google
searches issued in a day) with a tiny amount of storage (1.5 KB).Create aHyperLogLog:>>>frompotteryimportHyperLogLog>>>google_searches=HyperLogLog(redis=redis,key='google-searches')>>>Insert an element into theHyperLogLog:>>>google_searches.add('sonic the hedgehog video game')>>>See how many elements we’ve inserted into theHyperLogLog:>>>len(google_searches)1>>>Insert multiple elements into theHyperLogLog:>>>google_searches.update({...'google in 1998',...'minesweeper',...'joey tribbiani',...'wizard of oz',...'rgb to hex',...'pac-man',...'breathing exercise',...'do a barrel roll',...'snake',...})>>>len(google_searches)10>>>Through a clever hack, we can do membership testing against aHyperLogLog,
even though it was never designed for this purpose.  The way that the hack works
is that it creates a temporary copy of theHyperLogLog, then inserts the
element that you’re running the membership test for into the temporary
copy.  If the insertion changes the temporaryHyperLogLog’s cardinality,
then the element must not have been inserted into the originalHyperLogLog.>>>'joey tribbiani'ingoogle_searchesTrue>>>'jennifer aniston'ingoogle_searchesFalse>>>Do more efficient membership testing for multiple elements using.contains_many():>>>tuple(google_searches.contains_many('joey tribbiani','jennifer aniston'))(True,False)>>>Remove all of the elements from theHyperLogLog:>>>google_searches.clear()>>>len(google_searches)0>>>Limitations:Elements must be JSON serializable.len(hll)is probabilistic in that it’s an accurate approximation.Membership testing against a HyperLogLog is probabilistic in that itmayreturn false positives, butneverreturns false negatives.  This means that
ifelement in hllevaluates toTrue, then youmayhave inserted the
element into the HyperLogLog.  But ifelement in hllevaluates toFalse,
then youmust nothave inserted it.ContextTimer ⏱️ContextTimerhelps you easily and accurately measure elapsed time.  Note thatContextTimermeasures wall (real-world) time, not CPU time; and thatelapsed()returns time in milliseconds.You can useContextTimerstand-alone…>>>importtime>>>frompotteryimportContextTimer>>>timer=ContextTimer()>>>timer.start()>>>time.sleep(0.1)>>>100<=timer.elapsed()<200True>>>timer.stop()>>>time.sleep(0.1)>>>100<=timer.elapsed()<200True>>>…or as a context manager:>>>tests=[]>>>withContextTimer()astimer:...time.sleep(0.1)...tests.append(100<=timer.elapsed()<200)>>>time.sleep(0.1)>>>tests.append(100<=timer.elapsed()<200)>>>tests[True,True]>>>ContributingObtain source codeClone the git repo:$ git clone git@github.com:brainix/pottery.git$ cd pottery/Install project-level dependencies:$ make installRun testsIn one Terminal session:$ cd pottery/$ redis-serverIn a second Terminal session:$ cd pottery/$ make test$ make test-readmemake testruns all of the unit tests as well as the coverage test.  However,
sometimes, when debugging, it can be useful to run an individual test module,
class, or method:In one Terminal session:$ cd pottery/$ redis-serverIn a second Terminal session:Run a test module with$ make test tests=tests.test_dictRun a test class with:$ make test tests=tests.test_dict.DictTestsRun a test method with:$ make test tests=tests.test_dict.DictTests.test_keyexistserrormake test-readmedoctests the Python code examples in this README to ensure
that they’re correct.