lox 1.0.0
Threading and Multiprocessing for every project.
License: MIT license
Keywords: lox

.. image:: https://raw.githubusercontent.com/BrianPugh/lox/main/assets/lox_200w.png


.. image:: https://img.shields.io/pypi/v/lox.svg
        :target: https://pypi.python.org/pypi/lox

.. image:: https://circleci.com/gh/BrianPugh/lox.svg?style=svg
        :target: https://circleci.com/gh/BrianPugh/lox

.. image:: https://readthedocs.org/projects/lox/badge/?version=latest
        :target: https://lox.readthedocs.io/en/latest/?badge=latest
        :alt: Documentation Status


Threading and multiprocessing made easy.


* Free software: Apache-2.0 license
* Documentation: https://lox.readthedocs.io.
* Python >=3.6


**Lox** provides decorators and synchronization primitives to quickly add
concurrency to your projects.

Installation
------------

    pip3 install --user lox

Features
--------

* **Multithreading**: Powerful, intuitive multithreading in just 2 additional lines of code.

* **Multiprocessing**: Truly parallel function execution with the same interface as **multithreading**.

* **Synchronization**: Advanced thread synchronization, communication, and resource management tools.

Todos
-----

* All objects except ``lox.process`` are for threads. These will eventually be multiprocess friendly.

Usage
-----

Easy Multithreading
^^^^^^^^^^^^^^^^^^^

    >>> import lox
    >>>
    >>> @lox.thread(4) # Will operate with a maximum of 4 threads
    ... def foo(x,y):
    ...     return x*y
    >>> foo(3,4) # normal function calls still work
    12
    >>> for i in range(5):
    ...     foo.scatter(i, i+1)
    -ignore-
    >>> # foo is currently being executed in 4 threads
    >>> results = foo.gather() # block until results are ready
    >>> print(results) # Results are in the same order as scatter() calls
    [0, 2, 6, 12, 20]

Or, for example, if you aren't allowed to directly decorate the function you
would like multithreaded/multiprocessed, you can just directly invoke the
decorator:

.. code-block:: pycon

    >>> # Lets say we don't have direct access to this function
    ... def foo(x, y):
    ...     return x * y
    ...
    >>>
    >>> def my_func():
    ...     foo_threaded = lox.thread(foo)
    ...     for i in range(5):
    ...         foo_threaded.scatter(i, i + 1)
    ...     results = foo_threaded.gather()
    ...     # foo is currently being executed in default 50 thread executor pool
    ...     return results
    ...


This also makes it easier to dynamically control the number of
thread/processes in the executor pool. The syntax is a little weird, but
this is just explicitly invoking a decorator that has optional arguments:

.. code-block:: pycon

    >>> # Set the number of executer threads to 10
    >>> foo_threaded = lox.thread(10)(foo)


Easy Multiprocessing
^^^^^^^^^^^^^^^^^^^^

.. code-block:: pycon

    >>> import lox
    >>>
    >>> @lox.process(4)  # Will operate with a pool of 4 processes
    ... def foo(x, y):
    ...     return x * y
    ...
    >>> foo(3, 4)  # normal function calls still work
    12
    >>> for i in range(5):
    ...     foo.scatter(i, i + 1)
    ...
    -ignore-
    >>> # foo is currently being executed in 4 processes
    >>> results = foo.gather()  # block until results are ready
    >>> print(results)  # Results are in the same order as scatter() calls
    [0, 2, 6, 12, 20]


Progress Bar Support (tqdm)
^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: pycon

    >>> import lox
    >>> from random import random
    >>> from time import sleep
    >>>
    >>> @lox.thread(2)
    ... def foo(multiplier):
    ...     sleep(multiplier * random())
    ...
    >>> for i in range(10):
    >>>     foo.scatter(i)
    >>> results = foo.gather(tqdm=True)
    90%|████████████████████████████████▌        | 9/10 [00:03<00:00,  1.32it/s]
    100%|███████████████████████████████████████| 10/10 [00:06<00:00,  1.46s/it]


=======
History
=======

0.11.0 (2022-04-07)
-------------------
* Set number of workers to 0 (in thread execution) if the environment variable ``LOX_DEBUG`` is set to a true-like value (``true``, ``1``, etc.). Makes it easier to set breakpoints in multithreaded code without having to manually edit the decorator.

0.10.0 (2021-12-18)
-------------------
* Remove dependency pinning.
* Allow ``@lox.thread(0)``. This will execute ``scatter`` calls in parent thread.
  Useful for debugging breakpoints in parallelized code.

0.9.0 (2020-11-25)
------------------
* ``tqdm`` support on ``lox.process.gather``. See v0.8.0 release notes for usage.

0.8.0 (2020-11-25)
------------------
* ``tqdm`` support on ``lox.thread.gather``
  * Can be a bool::

        >>> my_func.gather(tqdm=True)

  * Can be a ``tqdm`` object::

        >>> from tqdm import tqdm
        >>> pbar = tqdm(total=100)
        >>> for _ in range(100):
        >>>     my_func.scatter()
        >>> my_func.gather(tqdm=pbar)

0.7.0 (2020-07-20)
------------------
* Complete rework of workers
  + Fix memory leaks
* Drop support for python3.5
* Drop support for chaining in favor of simpler codebase

0.6.3 (2019-07-30)
------------------
* Alternative fix for 0.6.2.

0.6.2 (2019-07-21)
------------------
* Update dependencies
* Fix garbage-collecting exclusiviity

0.6.1 (2019-07-21)
------------------
* Fix memory leak in ``lox.process``.

0.6.0 (2019-07-21)
------------------

* ``lox.Announcement`` ``subscribe()`` calls now return another ``Announcement``
  object that behaves like a queue instead of an actual queue. Allows for
  many-queue-to-many-queue communications.

* New Object: ``lox.Funnel``. allows for waiting on many queues for a complete
  set of inputs indicated by a job ID.

0.5.0 (2019-07-01)
------------------

* New Object: ``lox.Announcement``. Allows a one-to-many thread queue with
  backlog support so that late subscribers can still get all (or most recent)
  announcements before they subscribed.

* New Feature: ``lox.thread``  ``scatter`` calls can now be chained together.
  ``scatter`` now returns an ``int`` subclass that contains metadata to allow
  chaining. Each scatter call can have a maximum of 1 previous ``scatter`` result.

* Documentation updates, theming, and logos

0.4.3 (2019-06-24)
------------------
* Garbage collect cached decorated object methods

0.4.2 (2019-06-23)
------------------
* Fixed multiple instances and successive scatter and gather calls to wrapped methods

0.4.1 (2019-06-23)
------------------
* Fixed broken workers and unit tests for workers

0.4.0 (2019-06-22)
------------------
* Semi-breaking change: **lox.thread** and **lox.process** now automatically pass
  the object instance when decorating a method.

0.3.4 (2019-06-20)
------------------
* Print traceback in red when a thread crashes

0.3.3 (2019-06-19)
------------------
* Fix bug where thread in scatter of lox.thread double releases on empty queue

0.3.2 (2019-06-17)
------------------

* Fix manifest for installation from wheel

0.3.1 (2019-06-17)
------------------

* Fix package on pypi

0.3.0 (2019-06-01)
------------------

* Multiprocessing decorator. **lox.pool** renamed to **lox.thread**

* Substantial pytest bug fixes

* Documentation examples

* timeout for RWLock

0.2.1 (2019-05-25)
------------------

* Fix IndexSemaphore context manager

0.2.0 (2019-05-24)
------------------

* Added QLock

* Documentation syntax fixes

0.1.1 (2019-05-24)
------------------

* CICD test

0.1.0 (2019-05-24)
------------------

* First release on PyPI.