python-redis-lock 4.0.0
Lock context manager implemented via redis SETNX/BLPOP.
License: BSD-2-Clause
Requires Python: >=3.7

========
Overview
========



Lock context manager implemented via redis SETNX/BLPOP.

* Free software: BSD 2-Clause License

Interface targeted to be exactly like `threading.Lock <https://docs.python.org/2/library/threading.html#threading.Lock>`_.

Usage
=====

Because we don't want to require users to share the lock instance across processes you will have to give them names.

.. code-block:: python

    from redis import Redis
    conn = Redis()

    import redis_lock
    lock = redis_lock.Lock(conn, "name-of-the-lock")
    if lock.acquire(blocking=False):
        print("Got the lock.")
        lock.release()
    else:
        print("Someone else has the lock.")

Locks as Context Managers
=========================

.. code-block:: python

    conn = StrictRedis()
    with redis_lock.Lock(conn, "name-of-the-lock"):
        print("Got the lock. Doing some work ...")
        time.sleep(5)


You can also associate an identifier along with the lock so that it can be retrieved later by the same process, or by a
different one. This is useful in cases where the application needs to identify the lock owner (find out who currently
owns the lock).

.. code-block:: python

    import socket
    host_id = "owned-by-%s" % socket.gethostname()
    lock = redis_lock.Lock(conn, "name-of-the-lock", id=host_id)
    if lock.acquire(blocking=False):
        assert lock.locked() is True
        print("Got the lock.")
        lock.release()
    else:
        if lock.get_owner_id() == host_id:
            print("I already acquired this in another process.")
        else:
            print("The lock is held on another machine.")


Avoid dogpile effect in django
------------------------------

The dogpile is also known as the thundering herd effect or cache stampede. Here's a pattern to avoid the problem
without serving stale data. The work will be performed a single time and every client will wait for the fresh data.

To use this you will need `django-redis <https://github.com/jazzband/django-redis>`_, however, ``python-redis-lock``
provides you a cache backend that has a cache method for your convenience. Just install ``python-redis-lock`` like
this:

.. code-block:: bash

    pip install "python-redis-lock[django]"

Now put something like this in your settings:

.. code-block:: python

    CACHES = {
        'default': {
            'BACKEND': 'redis_lock.django_cache.RedisCache',
            'LOCATION': 'redis://127.0.0.1:6379/1',
            'OPTIONS': {
                'CLIENT_CLASS': 'django_redis.client.DefaultClient'
            }
        }
    }

.. note::
    If using a `django-redis` < `3.8.x`, you'll probably need `redis_cache`
    which has been deprecated in favor to `django_redis`. The `redis_cache`
    module is removed in `django-redis` versions > `3.9.x`. See `django-redis notes <https://github.com/jazzband/django-redis#configure-as-cache-backend>`_.


This backend just adds a convenient ``.lock(name, expire=None)`` function to django-redis's cache backend.

You would write your functions like this:

.. code-block:: python

    from django.core.cache import cache

    def function():
        val = cache.get(key)
        if not val:
            with cache.lock(key):
                val = cache.get(key)
                if not val:
                    # DO EXPENSIVE WORK
                    val = ...
                    cache.set(key, value)
        return val

Troubleshooting
---------------

In some cases, the lock remains in redis forever (like a server blackout / redis or application crash / an unhandled
exception). In such cases, the lock is not removed by restarting the application. One solution is to turn on the
`auto_renewal` parameter in combination with `expire` to set a time-out on the lock, but let `Lock()` automatically
keep resetting the expire time while your application code is executing:

.. code-block:: python

    # Get a lock with a 60-second lifetime but keep renewing it automatically
    # to ensure the lock is held for as long as the Python process is running.
    with redis_lock.Lock(conn, name='my-lock', expire=60, auto_renewal=True):
        # Do work....

Another solution is to use the ``reset_all()`` function when the application starts:

.. code-block:: python

    # On application start/restart
    import redis_lock
    redis_lock.reset_all()

Alternatively, you can reset individual locks via the ``reset`` method.

Use these carefully, if you understand what you do.


Features
========

* based on the standard SETNX recipe
* optional expiry
* optional timeout
* optional lock renewal (use a low expire but keep the lock active)
* no spinloops at acquire

Implementation
==============

``redis_lock`` will use 2 keys for each lock named ``<name>``:

* ``lock:<name>`` - a string value for the actual lock
* ``lock-signal:<name>`` - a list value for signaling the waiters when the lock is released

This is how it works:

.. image:: https://raw.githubusercontent.com/ionelmc/python-redis-lock/master/docs/redis-lock%20diagram%20(v3.0).png
    :alt: python-redis-lock flow diagram

Documentation
=============

https://python-redis-lock.readthedocs.io/en/latest/

Development
===========

To run the all tests run::

    tox

Requirements
============

:OS: Any
:Runtime: Python 2.7, 3.3 or later, or PyPy
:Services: Redis 2.6.12 or later.

Similar projects
================

* `bbangert/retools <https://github.com/bbangert/retools/blob/0.4/retools/lock.py>`_ - acquire does spinloop
* `distributing-locking-python-and-redis <https://chris-lamb.co.uk/posts/distributing-locking-python-and-redis>`_ - acquire does polling
* `cezarsa/redis_lock <https://github.com/cezarsa/redis_lock/blob/0.2.0/redis_lock/__init__.py>`_ - acquire does not block
* `andymccurdy/redis-py <https://github.com/andymccurdy/redis-py/blob/3.5.3/redis/lock.py>`_ - acquire does spinloop
* `mpessas/python-redis-lock <https://github.com/mpessas/python-redis-lock/blob/b512eef0fc5e1e2e82a6a31f65cd88c2c37dfe4b/redislock/lock.py>`_ - blocks fine but no expiration
* `brainix/pottery <https://github.com/brainix/pottery/blob/v1.1.5/pottery/redlock.py>`_ - acquire does spinloop


Changelog
=========

4.0.0 (2022-10-17)
------------------

* Dropped support for Python 2.7 and 3.6.
* Switched from Travis to GitHub Actions.
* Made logging messages more consistent.
* Replaced the ``redis_lock.refresh.thread.*`` loggers with a single ``redis_lock.refresh.thread`` logger.
* Various testing cleanup (mainly removal of hardcoded tmp paths).

3.7.0 (2020-11-20)
------------------

* Made logger names more specific. Now can have granular filtering on these new logger names:

  * ``redis_lock.acquire`` (emits `DEBUG` messages)
  * ``redis_lock.acquire`` (emits `WARN` messages)
  * ``redis_lock.acquire`` (emits `INFO` messages)
  * ``redis_lock.refresh.thread.start`` (emits `DEBUG` messages)
  * ``redis_lock.refresh.thread.exit`` (emits `DEBUG` messages)
  * ``redis_lock.refresh.start`` (emits `DEBUG` messages)
  * ``redis_lock.refresh.shutdown`` (emits `DEBUG` messages)
  * ``redis_lock.refresh.exit`` (emits `DEBUG` messages)
  * ``redis_lock.release`` (emits `DEBUG` messages)

  Contributed by Salomon Smeke Cohen in ``80``.
* Fixed few CI issues regarding doc checks.
  Contributed by Salomon Smeke Cohen in ``81``.

3.6.0 (2020-07-23)
------------------

* Improved ``timeout``/``expire`` validation so that:

  - ``timeout`` and ``expire are converted to ``None`` if they are falsy. Previously only ``None`` disabled these options, other falsy
    values created buggy situations.
  - Using ``timeout`` greater than ``expire`` is now allowed, if ``auto_renewal`` is set to ``True``. Previously a ``TimeoutTooLarge`` error
    was raised.
    See ``74``.
  - Negative ``timeout`` or ``expire`` are disallowed. Previously such values were allowed, and created buggy situations.
    See ``73``.
* Updated benchmark and examples.
* Removed the custom script caching code. Now the ``register_script`` method from the redis client is used.
  This will fix possible issue with redis clusters in theory, as the redis client has some specific handling for that.

3.5.0 (2020-01-13)
------------------

* Added a ``locked`` method. Contributed by Artem Slobodkin in ``72``.

3.4.0 (2019-12-06)
------------------

* Fixed regression that can cause deadlocks or slowdowns in certain configurations.
  See: ``71``.

3.3.1 (2019-01-19)
------------------

* Fixed failures when running python-redis-lock 3.3 alongside 3.2.
  See: ``64``.

3.3.0 (2019-01-17)
------------------

* Fixed deprecated use of ``warnings`` API. Contributed by Julie MacDonell in
  ``54``.
* Added ``auto_renewal`` option in ``RedisCache.lock`` (the Django cache backend wrapper). Contributed by c
  in ``55``.
* Changed log level for "%(script)s not cached" from WARNING to INFO.
* Added support for using ``decode_responses=True``. Lock keys are pure ascii now.

3.2.0 (2016-10-29)
------------------

* Changed the signal key cleanup operation do be done without any expires. This prevents lingering keys around for some time.
  Contributed by Andrew Pashkin in ``38``.
* Allow locks with given `id` to acquire. Previously it assumed that if you specify the `id` then the lock was already
  acquired. See ``44`` and
  ``39``.
* Allow using other redis clients with a ``strict=False``. Normally you're expected to pass in an instance
  of ``redis.StrictRedis``.
* Added convenience method `locked_get_or_set` to Django cache backend.

3.1.0 (2016-04-16)
------------------

* Changed the auto renewal to automatically stop the renewal thread if lock gets garbage collected. Contributed by
  Andrew Pashkin in ``33``.

3.0.0 (2016-01-16)
------------------

* Changed ``release`` so that it expires signal-keys immediately. Contributed by Andrew Pashkin in ``28``.
* Resetting locks (``reset`` or ``reset_all``) will release the lock. If there's someone waiting on the reset lock now it will
  acquire it. Contributed by Andrew Pashkin in ``29``.
* Added the ``extend`` method on ``Lock`` objects. Contributed by Andrew Pashkin in ``24``.
* Documentation improvements on ``release`` method. Contributed by Andrew Pashkin in ``22``.
* Fixed ``acquire(block=True)`` handling when ``expire`` option was used (it wasn't blocking indefinitely). Contributed by
  Tero Vuotila in ``35``.
* Changed ``release`` to check if lock was acquired with he same id. If not, ``NotAcquired`` will be raised.
  Previously there was just a check if it was acquired with the same instance (self._held).
  **BACKWARDS INCOMPATIBLE**
* Removed the ``force`` option from ``release`` - it wasn't really necessary and it only encourages sloppy programming. See
  ``25``.
  **BACKWARDS INCOMPATIBLE**
* Dropped tests for Python 2.6. It may work but it is unsupported.

2.3.0 (2015-09-27)
------------------

* Added the ``timeout`` option. Contributed by Victor Torres in ``20``.

2.2.0 (2015-08-19)
------------------

* Added the ``auto_renewal`` option. Contributed by Nick Groenen in ``18``.

2.1.0 (2015-03-12)
------------------

* New specific exception classes: ``AlreadyAcquired`` and ``NotAcquired``.
* Slightly improved efficiency when non-waiting acquires are used.

2.0.0 (2014-12-29)
------------------

* Rename ``Lock.token`` to ``Lock.id``. Now only allowed to be set via constructor. Contributed by Jardel Weyrich in ``11``.

1.0.0 (2014-12-23)
------------------

* Fix Django integration. (reported by Jardel Weyrich)
* Reorganize tests to use py.test.
* Add test for Django integration.
* Add ``reset_all`` functionality. Contributed by Yokotoka in ``7``.
* Add ``Lock.reset`` functionality.
* Expose the ``Lock.token`` attribute.

0.1.2 (2013-11-05)
------------------

* `?`

0.1.1 (2013-10-26)
------------------

* `?`

0.1.0 (2013-10-26)
------------------

* `?`

0.0.1 (2013-10-25)
------------------

* First release on PyPI.