psyrun 0.8.0
Easy parameter space evaluation and serial farming.
License: MIT

Psyrun
======

.. image:: https://travis-ci.org/jgosmann/psyrun.svg?branch=master
    :target: https://travis-ci.org/jgosmann/psyrun

.. image:: https://coveralls.io/repos/github/jgosmann/psyrun/badge.svg?branch=master
    :target: https://coveralls.io/github/jgosmann/psyrun?branch=master

Psyrun is a Python_ tool to define parameter spaces
and execute an evaluation function for each parameter assignment. In addition
Psyrun makes it easy to use serial farming, i.e. evaluating multiple parameter
assignments in parallel, on a multicore computers and high-performance clusters.

Documentation
-------------

`The documentation can be found here.
<http://psyrun.readthedocs.io/en/latest/>`_

Overview
--------

Define parameter spaces and evaluate them:

.. code-block:: python

    from psyrun import map_pspace, Param

    def objective(a, b, c):
        return a * b + c

    pspace = (Param(a=np.arange(1, 5))
              * Param(b=np.linspace(0, 1, 10))
              * Param(c=[1., 1.5, 10., 10.5]))
    results = map_pspace(objective, pspace) 

Or do it in parallel:

.. code-block:: python

    from psyrun import map_pspace_parallel
    results = map_pspace_parallel(objective, pspace)

Define tasks by placing ``task_<name>.py`` files in the `psy-tasks`` directory:

.. code-block:: python

    from psyrun import Param

    pspace = (Param(a=np.arange(1, 5))
              * Param(b=np.linspace(0, 1, 10))
              * Param(c=[1., 1.5, 10., 10.5]))

    def execute(a, b, c):
        return {'result': a * b + c}

and run them by typing ``psy run`` with support for serial farming on high
performance clusters.


Installation
------------

``pip install psyrun``

To be able to use the NPZ store::

    pip install numpy
    pip install 'psyrun[npz]'

To be able to use the HDF5 store::

    pip install numpy
    pip install 'psyrun[h5]'


Requirements
------------

* Python_ >=2.7, >=3.4
* `six <https://pypi.python.org/pypi/six>`_

Optional requirements
^^^^^^^^^^^^^^^^^^^^^

To have `faulthandler <http://faulthandler.readthedocs.io/>`_ activated for
jobs submitted with ``psy run`` in Python 2.7:

* `faulthandler <http://faulthandler.readthedocs.io/>`_

Python 3.4+ already includes the faulthandler module.

To use ``map_pspace_parallel``:

* `joblib <https://pythonhosted.org/joblib/>`_

To use NPZ files as store:

* `NumPy <http://www.numpy.org/>`_

To use HDF5 files as store:

* `NumPy <http://www.numpy.org/>`_
* `pytables <http://www.pytables.org/>`_

To run the unit tests:

* `joblib <https://pythonhosted.org/joblib/>`_
* `NumPy <http://www.numpy.org/>`_
* `pytables <http://www.pytables.org/>`_
* `pytest <http://doc.pytest.org/en/latest/>`_ >= 2.8


To build the documentation:

* `numpydoc <https://pypi.python.org/pypi/numpydoc>`_


.. _Python: https://www.python.org/


Changes
=======

0.8.0 (June 26, 2018)
---------------------

New features
^^^^^^^^^^^^

* Added the possibility to specify a ``setup`` function in task files that
  will be executed once at the start of each worker process and can be used
  to acquire resources and pass them to the processing function for individual
  parameter sets.



0.7.1 (April 19, 2018)
----------------------

Documentation improvements
^^^^^^^^^^^^^^^^^^^^^^^^^^

* Added documentation for the ``pool_size`` task attribute and included it in
  the task template.

Bug fixes
^^^^^^^^^

* Allow to continue tasks if results are missing even if the result file is
  newer than the task file.


0.7.0 (February 18, 2018)
-------------------------

New features
^^^^^^^^^^^^

* Added support for the Slurm Workload Manager.


0.6.0
-----

New features
^^^^^^^^^^^^

* Add ``psy new-task`` and ``psy kill`` commands.
* Added ``AutodetectStore`` that determines the appropriate store from the
  filename extension.
* Added possibility to let ``psy merge`` custom stores if provided as
  ``psyrun.stores`` entry point.
* Added capability to set scheduler arguments based on the job name.


0.5.4
-----

Bug fixes
^^^^^^^^^

* Fix the ``psy run`` continue functionality.


0.5.3
-----

Bug fixes
^^^^^^^^^

* Fix ``psy status`` and
  ``psyrun.backend.distribute.DistributeBackend.get_missing`` trying to read
  incompatible data files in the output directory.
* Fix ``psy status`` and
  ``psyrun.backend.distribute.DistributeBackend.get_missing`` easily hitting
  Python's recursion depth limit.
* Fix merging of npz files with missing integer values by converting them to
  float where ``np.nan`` can be used.


0.5.2
-----

Bug fixes
^^^^^^^^^

* Fix incorrect ``psy status``.
* Fix ``psy run <task1> <task2> ...`` not running all tasks and run them in
  order.


0.5.1
-----

Bug fixes
^^^^^^^^^

* Fix ``psy merge`` always assuming ``PickleStore``.

Documentation improvements
^^^^^^^^^^^^^^^^^^^^^^^^^^

* Add recipe for converting data to Pandas data frame to documentation.


0.5
---

* Initial release