Package: pyina
About PyinaThepyinapackage provides several basic tools to make MPI-based
parallel computing more accessable to the end user. The goal ofpyinais to allow the user to extend their own code to MPI-based parallel
computing with minimal refactoring.The central element ofpyinais the parallel map algorithm.pyinacurrently provides two strategies for executing the parallel-map,
where a strategy is the algorithm for distributing the work list of
jobs across the availble nodes.  These strategies can be used“in-the-raw”(i.e. directly) to provide the map algorithm to a user’s own mpi-aware code.
Further, inpyina.mpipyinaprovides pipe and map implementations
(known as“easy map”) that hide the MPI internals from the user. With the“easy map”, the user can launch their code in parallel batch mode – using
standard Python and without ever having to write a line of MPI code.There are several ways that a user would typically launch their code in
parallel – directly withmpirunormpiexec, or through the use of a
scheduler such astorqueorslurm.pyinaencapsulates several of these“launchers”, and provides a common interface to the different methods of
launching a MPI job.pyinais part ofpathos, a Python framework for heterogeneous computing.pyinais in active development, so any user feedback, bug reports, comments,
or suggestions are highly appreciated.  A list of issues is located athttps://github.com/uqfoundation/pyina/issues, with a legacy list maintained athttps://uqfoundation.github.io/project/pathos/query.Major Featurespyinaprovides a highly configurable parallel map interface
to running MPI jobs, with:a map interface that extends the Pythonmapstandardthe ability to submit batch jobs to a selection of schedulersthe ability to customize node and process launch configurationsthe ability to launch parallel MPI jobs with standard Pythonease in selecting different strategies for processing a work listCurrent ReleaseThe latest released version ofpyinais available at:https://pypi.org/project/pyinapyinais distributed under a 3-clause BSD license.Development VersionYou can get the latest development version with all the shiny new features at:https://github.com/uqfoundationIf you have a new contribution, please submit a pull request.Installationpyinacan be installed withpip:$ pip install pyinaA version of MPI must also be installed. Launchers inpyinathat
submit to a scheduler will throw errors if the underlying scheduler is
not available, however a scheduler is not required forpyinato execute.Requirementspyinarequires:python(orpypy),>=3.8setuptools,>=42cython,>=0.29.30numpy,>=1.0mpi4py,>=1.3dill,>=0.3.8pox,>=0.3.4pathos,>=0.3.2More InformationProbably the best way to get started is to look at the documentation athttp://pyina.rtfd.io. Also seehttps://github.com/uqfoundation/pyina/tree/master/examplesandpyina.testsfor a set of scripts that demonstrate the
configuration and launching of mpi-based parallel jobs using the“easy map”interface. You can run the tests withpython-mpyina.tests. A script is
included for querying, setting up, and tearing down an MPI environment, seepython-mpyinafor more information. The source code is generally well
documented, so further questions may be resolved by inspecting the code itself.
Please feel free to submit a ticket on github, or ask a question on
stackoverflow (@Mike McKerns). If you would like to share how you usepyinain your work, please send an email (tommckerns at uqfoundation dot
org).Important classes and functions are found here:pyina.mpi[the map API definition]pyina.schedulers[all available schedulers]pyina.launchers[all available launchers]Mapping strategies are found here:pyina.mpi_scatter[the scatter-gather strategy]pyina.mpi_pool[the worker pool strategy]pyinaalso provides a convience script that helps navigate the
MPI environment. This script can be run from anywhere with:$ mpi_worldIf may also be convienent to set a shell alias for the launch of ‘raw’
mpi-python jobs. Set something like the following (for bash):$ alias mpython1='mpiexec -np 1 `which python`'
$ alias mpython2='mpiexec -np 2 `which python`'
$ ...CitationIf you usepyinato do research that leads to publication, we ask that you
acknowledge use ofpyinaby citing the following in your publication:M.M. McKerns, L. Strand, T. Sullivan, A. Fang, M.A.G. Aivazis,
"Building a framework for predictive science", Proceedings of
the 10th Python in Science Conference, 2011;
http://arxiv.org/pdf/1202.1056

Michael McKerns and Michael Aivazis,
"pathos: a framework for heterogeneous computing", 2010- ;
https://uqfoundation.github.io/project/pathosPlease seehttps://uqfoundation.github.io/project/pathosorhttp://arxiv.org/pdf/1202.1056for further information.