Package: python-redis-lock
Lock context manager implemented via redis SETNX/BLPOP.Free software: BSD 2-Clause LicenseInterface targeted to be exactly likethreading.Lock.UsageBecause we don’t want to require users to share the lock instance across processes you will have to give them names.fromredisimportRedisconn=Redis()importredis_locklock=redis_lock.Lock(conn,"name-of-the-lock")iflock.acquire(blocking=False):print("Got the lock.")lock.release()else:print("Someone else has the lock.")Locks as Context Managersconn=StrictRedis()withredis_lock.Lock(conn,"name-of-the-lock"):print("Got the lock. Doing some work ...")time.sleep(5)You can also associate an identifier along with the lock so that it can be retrieved later by the same process, or by a
different one. This is useful in cases where the application needs to identify the lock owner (find out who currently
owns the lock).importsockethost_id="owned-by-%s"%socket.gethostname()lock=redis_lock.Lock(conn,"name-of-the-lock",id=host_id)iflock.acquire(blocking=False):assertlock.locked()isTrueprint("Got the lock.")lock.release()else:iflock.get_owner_id()==host_id:print("I already acquired this in another process.")else:print("The lock is held on another machine.")Avoid dogpile effect in djangoThe dogpile is also known as the thundering herd effect or cache stampede. Here’s a pattern to avoid the problem
without serving stale data. The work will be performed a single time and every client will wait for the fresh data.To use this you will needdjango-redis, however,python-redis-lockprovides you a cache backend that has a cache method for your convenience. Just installpython-redis-locklike
this:pipinstall"python-redis-lock[django]"Now put something like this in your settings:CACHES={'default':{'BACKEND':'redis_lock.django_cache.RedisCache','LOCATION':'redis://127.0.0.1:6379/1','OPTIONS':{'CLIENT_CLASS':'django_redis.client.DefaultClient'}}}NoteIf using adjango-redis<3.8.x, you’ll probably needredis_cachewhich has been deprecated in favor todjango_redis. Theredis_cachemodule is removed indjango-redisversions >3.9.x. Seedjango-redis notes.This backend just adds a convenient.lock(name, expire=None)function to django-redis’s cache backend.You would write your functions like this:fromdjango.core.cacheimportcachedeffunction():val=cache.get(key)ifnotval:withcache.lock(key):val=cache.get(key)ifnotval:# DO EXPENSIVE WORKval=...cache.set(key,value)returnvalTroubleshootingIn some cases, the lock remains in redis forever (like a server blackout / redis or application crash / an unhandled
exception). In such cases, the lock is not removed by restarting the application. One solution is to turn on theauto_renewalparameter in combination withexpireto set a time-out on the lock, but letLock()automatically
keep resetting the expire time while your application code is executing:# Get a lock with a 60-second lifetime but keep renewing it automatically# to ensure the lock is held for as long as the Python process is running.withredis_lock.Lock(conn,name='my-lock',expire=60,auto_renewal=True):# Do work....Another solution is to use thereset_all()function when the application starts:# On application start/restartimportredis_lockredis_lock.reset_all()Alternatively, you can reset individual locks via theresetmethod.Use these carefully, if you understand what you do.Featuresbased on the standard SETNX recipeoptional expiryoptional timeoutoptional lock renewal (use a low expire but keep the lock active)no spinloops at acquireImplementationredis_lockwill use 2 keys for each lock named<name>:lock:<name>- a string value for the actual locklock-signal:<name>- a list value for signaling the waiters when the lock is releasedThis is how it works:Documentationhttps://python-redis-lock.readthedocs.io/en/latest/DevelopmentTo run the all tests run:toxRequirementsOS:AnyRuntime:Python 2.7, 3.3 or later, or PyPyServices:Redis 2.6.12 or later.Similar projectsbbangert/retools- acquire does spinloopdistributing-locking-python-and-redis- acquire does pollingcezarsa/redis_lock- acquire does not blockandymccurdy/redis-py- acquire does spinloopmpessas/python-redis-lock- blocks fine but no expirationbrainix/pottery- acquire does spinloopChangelog4.0.0 (2022-10-17)Dropped support for Python 2.7 and 3.6.Switched from Travis to GitHub Actions.Made logging messages more consistent.Replaced theredis_lock.refresh.thread.*loggers with a singleredis_lock.refresh.threadlogger.Various testing cleanup (mainly removal of hardcoded tmp paths).3.7.0 (2020-11-20)Made logger names more specific. Now can have granular filtering on these new logger names:redis_lock.acquire(emitsDEBUGmessages)redis_lock.acquire(emitsWARNmessages)redis_lock.acquire(emitsINFOmessages)redis_lock.refresh.thread.start(emitsDEBUGmessages)redis_lock.refresh.thread.exit(emitsDEBUGmessages)redis_lock.refresh.start(emitsDEBUGmessages)redis_lock.refresh.shutdown(emitsDEBUGmessages)redis_lock.refresh.exit(emitsDEBUGmessages)redis_lock.release(emitsDEBUGmessages)Contributed by Salomon Smeke Cohen in80.Fixed few CI issues regarding doc checks.
Contributed by Salomon Smeke Cohen in81.3.6.0 (2020-07-23)Improvedtimeout/expirevalidation so that:timeoutandexpire are converted to ``Noneif they are falsy. Previously onlyNonedisabled these options, other falsy
values created buggy situations.Usingtimeoutgreater thanexpireis now allowed, ifauto_renewalis set toTrue. Previously aTimeoutTooLargeerror
was raised.
See74.Negativetimeoutorexpireare disallowed. Previously such values were allowed, and created buggy situations.
See73.Updated benchmark and examples.Removed the custom script caching code. Now theregister_scriptmethod from the redis client is used.
This will fix possible issue with redis clusters in theory, as the redis client has some specific handling for that.3.5.0 (2020-01-13)Added alockedmethod. Contributed by Artem Slobodkin in72.3.4.0 (2019-12-06)Fixed regression that can cause deadlocks or slowdowns in certain configurations.
See:71.3.3.1 (2019-01-19)Fixed failures when running python-redis-lock 3.3 alongside 3.2.
See:64.3.3.0 (2019-01-17)Fixed deprecated use ofwarningsAPI. Contributed by Julie MacDonell in54.Addedauto_renewaloption inRedisCache.lock(the Django cache backend wrapper). Contributed by c
in55.Changed log level for “%(script)s not cached” from WARNING to INFO.Added support for usingdecode_responses=True. Lock keys are pure ascii now.3.2.0 (2016-10-29)Changed the signal key cleanup operation do be done without any expires. This prevents lingering keys around for some time.
Contributed by Andrew Pashkin in38.Allow locks with givenidto acquire. Previously it assumed that if you specify theidthen the lock was already
acquired. See44and39.Allow using other redis clients with astrict=False. Normally you’re expected to pass in an instance
ofredis.StrictRedis.Added convenience methodlocked_get_or_setto Django cache backend.3.1.0 (2016-04-16)Changed the auto renewal to automatically stop the renewal thread if lock gets garbage collected. Contributed by
Andrew Pashkin in33.3.0.0 (2016-01-16)Changedreleaseso that it expires signal-keys immediately. Contributed by Andrew Pashkin in28.Resetting locks (resetorreset_all) will release the lock. If there’s someone waiting on the reset lock now it will
acquire it. Contributed by Andrew Pashkin in29.Added theextendmethod onLockobjects. Contributed by Andrew Pashkin in24.Documentation improvements onreleasemethod. Contributed by Andrew Pashkin in22.Fixedacquire(block=True)handling whenexpireoption was used (it wasn’t blocking indefinitely). Contributed by
Tero Vuotila in35.Changedreleaseto check if lock was acquired with he same id. If not,NotAcquiredwill be raised.
Previously there was just a check if it was acquired with the same instance (self._held).BACKWARDS INCOMPATIBLERemoved theforceoption fromrelease- it wasn’t really necessary and it only encourages sloppy programming. See25.BACKWARDS INCOMPATIBLEDropped tests for Python 2.6. It may work but it is unsupported.2.3.0 (2015-09-27)Added thetimeoutoption. Contributed by Victor Torres in20.2.2.0 (2015-08-19)Added theauto_renewaloption. Contributed by Nick Groenen in18.2.1.0 (2015-03-12)New specific exception classes:AlreadyAcquiredandNotAcquired.Slightly improved efficiency when non-waiting acquires are used.2.0.0 (2014-12-29)RenameLock.tokentoLock.id. Now only allowed to be set via constructor. Contributed by Jardel Weyrich in11.1.0.0 (2014-12-23)Fix Django integration. (reported by Jardel Weyrich)Reorganize tests to use py.test.Add test for Django integration.Addreset_allfunctionality. Contributed by Yokotoka in7.AddLock.resetfunctionality.Expose theLock.tokenattribute.0.1.2 (2013-11-05)?0.1.1 (2013-10-26)?0.1.0 (2013-10-26)?0.0.1 (2013-10-25)First release on PyPI.