Package: techies
Opinionated Python toolboxMaster branch:PrerequisitesRedis server for Counters and PythonQueueimplementations.Installation$ pip install techies
$ pip install techies --upgradeIn The BoxCounters (backed by Redis)New in 0.2.0techies.MultiCounteris a stateless multi-event
counter, based on RedisHash.fromtechiesimportMultiCountercounter=MultiCounter(key='demo_counter')counter.incr('event_1')counter.incr('event_1')counter.incr('event_2')counter.incr('event_3')counter.incr('event_2')print(counter.get_count('event_1'))# 2print(counter.get_count('event_2'))# 2print(counter.get_count('event_3'))# 1print(counter.get_count('event_null'))# 0print(counter.json())# {u'event_2': u'4', u'event_3': u'2', u'event_1': u'4'}print(unicode(counter))# {"event_2": "2", "event_3": "1", "event_1": "2"}print(str(counter))# same as above# clears the countscounter.clear()New in 0.2.0techies.TsCounteris a stateless multi-key,
single-event timestamp counter, based on RedisHash.fromtechiesimportTsCounterimporttime# initialize with chunk_size and ttl definedcounter=TsCounter(key='demo_event',chunk_size=24*60*60,ttl=48*60*60)# or call initialize() method later to customize chunk_size and/or ttlcounter.initialize(chunk_size=24*60*60,ttl=48*60*60)t=time.time()counter.incr(timestamp=t-86400)counter.incr(timestamp=t-86400)counter.incr(timestamp=t)counter.incr(timestamp=t-86400)counter.incr(timestamp=t+86400)counter.incr(timestamp=t+86400)print(counter.get_count(timestamp=t-86400))# 3print(counter.get_count(timestamp=t))# 1print(counter.get_count(timestamp=t+86400))# 2print(counter.json())# {u'demo_event:1429142400': {u'1429162301': u'3'}, u'demo_event:1429228800': {u'1429248701': u'1'}, u'demo_event:1429315200': {u'1429335101': u'2'}}print(unicode(counter))# {"demo_event:1429142400": {"1429162301": "3"}, "demo_event:1429228800": {"1429248701": "1"}, "demo_event:1429315200": {"1429335101": "2"}}print(str(counter))# same as above# clears the countscounter.clear()techies.StateCounteris a single event state counter, based on RedisHash. Project`tidehunter<https://github.com/woozyking/tidehunter>`__ is built
around the concept and APIs of this counter, you can find some extended
usage example on itsproject
page.Breaking API
Changes from 0.1.4 to 0.2.0:StateCounternow has a new behavior
when its objects are casted bystrandunicode.get_all()is
nowjson(), andstartedandstoppedare now properties
instead of methods.PythonQueueImplementations (backed by Redis)techies.Queue, based on RedisList. Interfaces are almost
standard queue compatible.fromtechiesimportQueueq=Queue(key='demo_q',host='localhost',port=6379,db=0)# put, or enqueueq.put('lol')q.put('dota')q.put('skyrim')q.put('dota')# Check size of the queue, two waysprint(q.qsize())# 4print(len(q))# 4# get, or dequeueprint(q.get())# 'lol'print(q.get())# 'dota'print(q.get())# 'skyrim'print(q.get())# 'dota'print(q.get())# ''# clear the queueq.clear()techies.UniQueue, based on RedisSorted Set. Inheritstechies.Queuebut ignores repetitive items, keeps items unique.
Score of the sorted set member is epoch timestamp fromtime.time().fromtechiesimportUniQueueq=UniQueue(key='demo_q',host='localhost',port=6379,db=0)# put, or enqueueq.put('lol')q.put('dota')q.put('skyrim')q.put('dota')# this one is ignored# Check size of the unique queue, two waysprint(q.qsize())# 3print(len(q))# 3# get, or dequeueprint(q.get())# 'lol'print(q.get())# 'dota'print(q.get())# 'skyrim'print(q.get())# ''  # only 3 unique items# clear the queueq.clear()techies.CountQueue, based on RedisSorted Set. Inheritstechies.UniQueuebut score is used as a count of item appearance,
that the item has the highest count gets placed in front to begetfirst.fromtechiesimportCountQueueq=CountQueue(key='demo_q',host='localhost',port=6379,db=0)# put, or enqueueq.put('lol')q.put('dota')q.put('skyrim')q.put('dota')# increment the count of the existing 'dota'# Check size of the unique queue, two waysprint(q.qsize())# 3print(len(q))# 3# get, or dequeueprint(q.get())# ('dota', 2)  # the one with the most count is returned firstprint(q.get())# ('lol', 1)print(q.get())# ('skyrim', 1)print(q.get())# ()  # only 3 unique items still# clear the queueq.clear()Pythonlogging.HandlerImplementationtechies.QueueHandler, inherits standardlogging.Handlerthatemitto any standardQueuecompatible implementations, including
all theQueueimplementations in this library.importloggingfromtechiesimport(Queue,UniQueue,CountQueue,QueueHandler,REF_LOG_FORMAT)fromtechies.compatimportxrangeif__name__=='__main__':key='test_q'q=Queue(key=key,host='localhost',port=6379,db=0)uq=UniQueue(key=key,host='localhost',port=6379,db=1)cq=CountQueue(key=key,host='localhost',port=6379,db=2)logger=logging.getLogger(__name__)foriin[q,uq,cq]:handler=QueueHandler(i)handler.setFormatter(logging.Formatter(REF_LOG_FORMAT))logger.addHandler(handler)# Enqueue multiple times of the same errorforiinxrange(3):try:1/0exceptZeroDivisionErrorase:logger.error(e)# simple queue, should print error log 3 timeswhilelen(q):print(q.get())# unique queue, should just have 1 item in this caseprint(len(uq)==1)print(uq.get())# count queue, should just have 1 item as unique queueprint(len(cq)==1)print(cq.get()[1])# 3, the count of the same itemforiin[q,uq,cq]:i.clear()Test (Unit Tests)To run unit tests locally, make sure that you have Redis server
installed and running locally, where DB 0 is not occupied by any data
that you cannot afford to lose.$ pip install -r requirements.txt
$ pip install -r test_requirements.txt
$ nosetests --with-coverage --cover-package=techiesLicenseThe MIT License (MIT). See the fullLICENSE.ContributorsRunzhou Li (Leo)Frank Yin- InitialQueueandUniQueueimplementationsChangelog0.2.0 (2015-04-17)Removedhiredisfrom requirements.txt since it is not a hard
requirement. Users who wish to take advantage ofhirediscan
always install it themselves, following the concept ofredis-py.AddedMultiCounter, a stateless multi-event counter, based on
RedisHash.AddedTsCounter, a stateless timestamp counter based on RedisHash, and making use of multiple keys to group timestamps, with
auto expiration based on Redis key TTL mechanism.Breaking Change:StateCounternow has a new behavior when its
objects are casted bystrandunicode.get_all()is nowjson(). This change directly reflects the underlying change
described in the next point. Also,startedandstoppedare
now properties instead of methods.Simplifiedtechies.landmines.RedisBaseby factoring out its new
child classtechies.landmines.RedisHashBase. All Redis Hash based
implementations now extends from this class to get unicodedictby callingjson()method (previouslyget_all()), and unicode
string serialization of thedictby casting the objects usingstrorunicode(both behave exactly the same).0.1.4 (2014-01-22)AddedStateCounter, a state counter based on RedisHash. To
see an example of its usage, seetidehunter.Includedhiredisin requirements.txt for added performance gain.0.1.3 (2014-01-20)AddedQueueHandler, inherits standardlogging.Handlerthatemitto any standardQueuecompatible implementations,
including all theQueueimplementations in this libraryExposed direct accessibility of all intended APIs, for example,from techies import Queue; or if you will,from techies import *Behavior ofCountQueue.gethas changed from just returning the
item value to atupleof item value and its number of appearances
in the queue, eg:('dota', 2); an emptytupleis returned
when theCountQueueis empty0.1.2 (2014-01-17)AddedCountQueue, inheritsUniQueuebut the score is used as
a count of item appearance, that the item has the highest count gets
placed in front to begetfirstBehavior of allQueueand its subclasses’putandput_nowaitmethods changed from return certain value to not
return anything (more Python standard Queue like)0.1.1 (2014-01-16)Python 3.2 and 3.3 supported nowBehavior of bothQueueandUniQueue’sgetmethod changed
from returningNoneto empty “native string” (unicode for Python
2.x, str for Python 3.x) when attempting to dequeue from an empty
queue0.1.0 (2014-01-16)Initial release with Redis backedQueueandUniQueue