Package: py-package-template
Python Package Template ProjectThe py-template-project package allows users to download the contents of thisGiHub repository,  containing a skeleton Python package project to be used as a template for kick-starting development ofanytype of Package; destined for upload to PyPI, or just for local install using Pip. The downloaded package includes the following components to aid rapid development without having to spend time cloning existing set-ups from other projects:a minimalsetup.pyfile;testing with PyTest;documentation (HTML and PDF) generated using Sphinx with auto-documentation setup;an entry-point that allows the package to execute functions directly from the command line - e.g. to start a server, interact with a user, download a GitHub repository, etc.; and,automated testing and deployment using Travis CI.A description of how to work with (and modify) each of these components, is provided in more detail in the sections that follow-on below, as well as in the documentation and within the example code bundled with the package.This is obviously a opinionated view of how a Python package project ought to be structured, based largely on my own experiences and requirements. Where I have needed guidance on this subject, I have leant heavily on the advice given by thePython Packaging Authority (PyPA)and used the excellentRequestsandFlaskprojects as references for 'best practices'.InstallingInstall and update usingpip:pip3installpy-template-projectDownloading a Python Package Template ProjectTo down load the latest version of the Python Package Template project located inthis GiHub repository, execute the following command from the command line:py-package-templateinstallThis will be downloaded to the current directory and will contain the following directory structure:py-package-tempate/|--docs/|--|--build_html/|--|--build_latex/|--|--source/|--py-pkg/|--|--__init__.py|--|--__version__.py|--|--curves.py|--|--entry_points.py|--tests/|--|--test_data/|--||--supply_demand_data.json|--|__init__.py|--|conftest.py|--|test_curves.py|--.env|--.gitignore|--Pipfile|--Pipfile.lock|--README.md|--setup.pyThe Python Package Template ProjectWe now describe the various components of the template project and the workflows associated with it. The template package project contains two modules to get things started:curves.pyentry_points.pyThecurves.pymodule contains sample code for modelling economic supply and demand curves and makes for a useful demonstration of how Python type annotation and interface definition via abstract base classes, can make code easier to read, document and reason about (I am a big fan). The test suite for this module is contained in thetestsfolder and demonstrates how to get up-and-running with PyTest.Theentry_points.pymodule is referenced in thesetup.pyfile via theentry_pointsdefinitions:entry_points={'console_scripts':['py-package-template=py_pkg.entry_points:main'],}It enables the declared entry point -py_pkg.entry_points.main-  to be invoked whenpy-package-templateis called from the command line. This is what enables the template project to be downloaded programmatically (check the code for the full details). This could easily be extended to start a server (e.g. using Flask), or run any other type of script.Project DependenciesWe usepipenvfor managing project dependencies and Python environments (i.e. virtual environments). These dependencies arenotto be confused with the package installation dependencies for the package under developement - i.e. those that need to be defined in theinstall_requiressection ofsetup.py. All of the direct packages dependencies required to run the project's code (e.g. NumPy for tensors), as well as all the packages used during development (e.g. flake8 for code linting and IPython for interactive console sessions), are described in thePipfile. Their precise downstream dependencies are crystallised inPipfile.lock, which is used to guarentee repeatable (i.e. deterministic) builds.Installing PipenvTo get started with Pipenv, first of all download it - assuming that there is a 'global' version of Python available on your system and on the PATH, then this can be achieved by running the following command,pip3installpipenvFor more information, including advanced configuration options, see theofficial pipenv documentation.Installing this Projects' DependenciesMake sure that you're in the project's root directory (the same one in whichPipfileresides), and then run,pipenvinstall--devThis will install all of the direct project dependencies as well as the development dependencies (the latter a consequence of the--devflag). To add and remove dependencies as required for your new project, usepipenv installandpipenv uninstallas required, using the--devflag for development-only dependencies.Running Python and IPython from the Project's Virtual EnvironmentIn order to open a Python REPL using within an environment that precisely mimics the one the project is being developed with, use Pipenv from the command line as follows,pipenvrunpython3Thepython3command could just as well beipython3.Automatic Loading of Environment VariablesPipenv will automatically pick-up and load any environment variables declared in the.envfile, located in the package's root directory. For example, adding,SPARK_HOME=applications/spark-2.3.1/binWill enable access to this variable within any Python program, via a call toos.environ['SPARK_HOME']. Note, that if any security credentials are placed here, then this filemustbe removed from source control - i.e. add.envto the.gitignorefile to prevent potential security risks.Pipenv ShellsPrependingpipenvto every command you want to run within the context of your Pipenv-managed virtual environment, can get (very) tedious. This can be avoided by entering into a Pipenv-managed shell,pipenvshellWhich is equivalent to 'activating' the virtual environment. Any command will now be executed within the virtual environment. Useexitto leave the shell session.Running Unit TestsAll test have been written using thePyTestpackage. Tests are kept in thetestsfolder and can be run from the command line by - e.g. by invoking,pipenvrunpytestThe test suite is structured as an independent Python package as follows:tests/|--test_data/||--supply_demand_data.json|__init__.py|conftest.py|test_curves.pyTheconftest.pymodule is used by PyTest - in this particular instance for loading test data and building objects that will then be used by potentially many other tests. These are referred to as 'fixtures' in PyTest - more details can be foundhere.Linting CodeI prefer to useflake8for style guide enforcement. This can be invoked from the command line by running,pipenvrunflake8py_pkgFlake8 could easily be swapped-out for another tool by using Pipenv as described above.Static Type CheckingWe have used the Python type annotation framework, together with theMyPy package, to perform static type checks on the codebase. Analogous to any linter or unit testing framework, MyPy can be run from the command line as follows,pipenvrunpython-mmypypy_pkg/*.pyMyPy options for this project can be defined in themypy.inifile that MyPy will look for by default. For more information on the full set of options, see themypy documentation.Examples of type annotation and type checking for library development can be found in thepy_pkg.curves.pymodule. This should also be cross-referenced with the improvement to readability (and usability) that this has on package documentation.DocumentationThe documentation in thedocsfolder has been built usingSphinx. We have used the default 'quickstart' automatic configuration, which was originally triggered by executing,pipenvrunsphinx-quickstartThe output is based primarily on the Docstrings in the source code, using theautodocextension within Sphinx (specified during the 'quickstart'). The contents for the entry point into the docs (index.html), is defined in theindex.rstfile, which itself imports themodules.rstfile that lists all of the modules to document. The documentation can be built by running the following command,pipenvrunsphinx-build-bhtmldocs/sourcedocs/build_htmlThe resulting HTML documentation can be accessed by openingdocs/build_html/index.htmlin a web browser.My preferred third party theme fromRead the Docshas also been used, by installing thesphinx_rtd_themeas a development dependency and modifyingdocs/source/config.pyas follows:importsphinx_rtd_themehtml_theme="sphinx_rtd_theme"html_theme_path=[sphinx_rtd_theme.get_html_theme_path()]Creating a PDF Version Using LaTeXSo long as a LaTex distribution is present on your system (e.g. MikTeX for Mac OS X), then it is possible to create a PDF version of the documentation, as well. Start by building the prerequisite LaTex version from the ReStructured Text originals,pipenvrunsphinx-build-blatexdocs/sourcedocs/build_latexThen, navigate todocs/build_latexand run,makeBoth LaTeX and PDF versions can then be found indocs/build_latex.Building Deployable DistributionsThe recommended (and most pragmatic) way of deploy this package is to build a Pythonwheeland to then to install it in a fresh virtual environment on the target system. The exact build configuration is determined by the parameters insetup.py. Note, that this requires that all package dependencies also be specified in theinstall_requiresdeclaration insetup.py,regardlessof their entry inPipfile. For more information on Python packaging refer to thePython Packaging User Guideand the accompanyingsample project. To create the Python wheel run,pipenvrunpythonsetup.pybdist_wheelThis will createbuild,py_package_template.egg-infoanddistdirectories - the wheel can be found in the latter. This needs to be copied to the target system (which we are assuming has Python and Pipenv available as a minimum), where it can be installed into a new virtual environment, together with all downstream dependencies, using,pipenvinstallpath/to/your-package.whlAutomated Testing and Deployment using Travis CIWe have chosen Travis for Continuous Integration (CI) as it integrates very easily with Python and GitHub (where I have granted it access to my public repositories). The configuration details are kept in the.travis.yamlfile in the root directory:ncsudo:requiredlanguage:pythonpython:-3.7-devinstall:-pip install pipenv-pipenv install --devscript:-pipenv run pytestdeploy:provider:pypiuser:alexioannidespassword:secure:my-encrypted-pypi-passwordon:tags:truedistributions:bdist_wheelBriefly, this instructs the Travis build server to:download, build and install Python 3.7;install Pipenvuse Pipenv andPipfile.lockto installalldependencies (dev dependencies are necessary for running PyTest);run all unit tests using PyTest;if the tests were run successfully and if we have pushed a new tag (i.e. a release) to the master branch then:build a Python wheel; and,push it to PyPI.org using my PyPI account credentials.Note that we provide Travis with an encrypted password, that was made using the Travis command line tool (downloaded using HomeBrew on OS X). For more details on this and PyPI deployment more generally see theTravis CI documentation.