Package: py-dspinlock
Distributed SpinlockIntroductionThis is a simple, yet flexible implementation of a distributed spinlock mutex. A mutex is a way to lock a resource
under contention, for more information about mutexes you can readhere. In particular, aSpinlockis a type
of mutex that blocks the waiting thread and continuously polls the target mutex to see when it can be acquired. This
can be useful when you have many stateless distributed services that are in contention for the same resource. For
example, this can be reflected when you want to avoid duplicate computations while a query is already being executed.Note: the documentation inreadthedocsis always generated against the latest published version, thus there might
be differences if you are using the checkout frommainavenue to install.PreliminariesThe implementation is based onredisand practically is the only required dependency to successfully install the
library. For more configuration options and usage directions, please read on.InstallationTo use the library, please install it as follows,# assuming you are in a virtual environmentpipinstallpy-dspinlockAny requirements will be installed automatically, but the minimum required version ofredisclient is set to
be4.0.0.UsageOut of the box, the library provides a distributed spinlock implementation that can be used withanyobject that
implementsHashable. Practically, this means that the object instance can behashed. For example,# a string is hashable,s="example"# its hash value can be extracted as such,s_hash=hash(s)Thus, a practical example that you can use out of the box is the following,"""A basic example of how to use the HashDSpinlock."""fromdspinlockimportHashDSpinlockif__name__=="__main__":withHashDSpinlock("example"):print("executed task_id: 1")If you do not supply any arguments, then the default parameters are used. Meaning that the following hold,redisconnection gets created with the predefined parameters (host:localhost, db:1, port:6379)fail_if_key_exists: is set toFalsecached_if_computed: is set toFalse.ParameterisingredisconnectionIdeally, theredisconnection should not be created for every call we make to the spinlock mutex. Thus, ideally,
we should be caching that and passing it as an argument as such,"""An example of how to use the HashDSpinlock with an existing `redis` connection."""fromdspinlockimportHashDSpinlockfromdspinlock.utilsimportcreate_redis_conn# using default parameterssess=create_redis_conn()if__name__=="__main__":withHashDSpinlock("example",sess=sess):print("executed task_id: 1")If no arguments are supplied tocreate_redis_connmethod, then a connection with the default parameters is created.
To parameterise it, we can use the following,"""An example of how to use the HashDSpinlock with an existing parameterised `redis` connection."""fromdspinlockimportHashDSpinlockfromdspinlock.utilsimportcreate_redis_conn,RedisParametersparams=RedisParameters(redis_host="localhost",rdb=0,port=6379,ssl=False,socket_connect_timeout=2,decode_responses=True)# using default parameterssess=create_redis_conn(params)if__name__=="__main__":withHashDSpinlock("example",sess=sess):print("executed task_id: 1")For more information, please check the source code at thelink.Extending the base class to fit your needsWhile a default Spinlock is supplied, it is very common that you might want to subclass it in order to customise
its functionality. There are two avenues to do that, you can subclassHashDSpinlockdirectly, or you can
subclass its base abstract classDSpinlockBase. Most of the functionality is contained in the base abstract class,
hence you need to only implement the methods shown inHashDSpinlockand tweak any of the base class values such as,max_spinlock_tries: The spinlock max retries, by default 10 tries.spinlock_sleep_thresh: The spinlock sleep threshold, by default 0.5 seconds.expire_at_timedelta: Sets timedelta from creation that the mutex expires, by default 1 hour.max_block_time: The max block time allowed for a query mutex to be held, if not released it's forcefully unblocked.ExtendingHashDSpinLockto tweak above values"""An example of how to subclass `HashDSpinlock` while tweaking its base attributes."""fromdspinlockimportHashDSpinlockclassCustomSpinlock(HashDSpinlock):"""Custom class that overrides the basic variables"""spinlock_sleep_thresh:float=0.1"""Tweak the threshold for spinlock sleep."""if__name__=="__main__":withCustomSpinlock("example"):print("executed task_id: 1")Advanced customisationThe library provides the primitives so that you can create your own subclasses based on eitherHashDSpinlockor the
baseclass itself. To so do, you can override these functions,_get_uid: returns the unique identifier for the instance, by default it is thehashof the object,_get_tag: the tag of the instance which is used to "tag" the actual mutex value,_unpack_value: defines the way we unpack the value retrieved fromredis,get_key: returns the key that should be uniquely identifying each entry.You do not have to override all of them, but for more details you can see how theHashDSpinlockitself is
implemented.Note: Python 3 does not support stable hashing forstr,bytes, anddatetime. In these instances, we use a stable
hashing algorithm fromhashlib, namelysha256based digests.Enable loggingThe library uses a custom logger in order to provide diagnostic information. To enable the logger, to do two things.
Firstly you need to set the environment variable - using any value - with the key:SL_LOG_ENABLED. After, you have
to set your basic logger to accept at leastDEBUGlevel messages, as all diagnostic ones are set to that level.The code snippet that accomplishes that is shown below,importloggingfromdspinlockimportHashDSpinlockfromdspinlock.constsimportSL_LOG_LEVEL# ensure your basic logger existslogging.basicConfig(level=SL_LOG_LEVEL)if__name__=="__main__":# now you see some debug messages in the default format of the library.withHashDSpinlock("example"):print("executed task_id: 1")