pyproject-ops 0.8.1
Python project Ops automation.
License: MIT
Requires Python: >=3.7


.. .. image:: https://readthedocs.org/projects/pyproject_ops/badge/?version=latest
    :target: https://pyproject-ops.readthedocs.io/index.html
    :alt: Documentation Status

.. image:: https://github.com/MacHu-GWU/pyproject_ops-project/workflows/CI/badge.svg
    :target: https://github.com/MacHu-GWU/pyproject_ops-project/actions?query=workflow:CI

.. .. image:: https://codecov.io/gh/MacHu-GWU/pyproject_ops-project/branch/main/graph/badge.svg
    :target: https://codecov.io/gh/MacHu-GWU/pyproject_ops-project

.. image:: https://img.shields.io/pypi/v/pyproject_ops.svg
    :target: https://pypi.python.org/pypi/pyproject_ops

.. image:: https://img.shields.io/pypi/l/pyproject_ops.svg
    :target: https://pypi.python.org/pypi/pyproject_ops

.. image:: https://img.shields.io/pypi/pyversions/pyproject_ops.svg
    :target: https://pypi.python.org/pypi/pyproject_ops

.. image:: https://img.shields.io/badge/release_history!--None.svg?style=social
    :target: https://github.com/MacHu-GWU/pyproject_ops-project/blob/main/release-history.rst

.. image:: https://img.shields.io/badge/STAR_Me_on_GitHub!--None.svg?style=social
    :target: https://github.com/MacHu-GWU/pyproject_ops-project

------

.. .. image:: https://img.shields.io/badge/Link-Document-blue.svg
    :target: https://pyproject-ops.readthedocs.io/index.html

.. .. image:: https://img.shields.io/badge/Link-API-blue.svg
    :target: https://pyproject-ops.readthedocs.io/py-modindex.html

.. .. image:: https://img.shields.io/badge/Link-Source_Code-blue.svg
    :target: https://pyproject-ops.readthedocs.io/py-modindex.html

.. image:: https://img.shields.io/badge/Link-Install-blue.svg
    :target: `install`_

.. image:: https://img.shields.io/badge/Link-GitHub-blue.svg
    :target: https://github.com/MacHu-GWU/pyproject_ops-project

.. image:: https://img.shields.io/badge/Link-Submit_Issue-blue.svg
    :target: https://github.com/MacHu-GWU/pyproject_ops-project/issues

.. image:: https://img.shields.io/badge/Link-Request_Feature-blue.svg
    :target: https://github.com/MacHu-GWU/pyproject_ops-project/issues

.. image:: https://img.shields.io/badge/Link-Download-blue.svg
    :target: https://pypi.org/pypi/pyproject_ops#files


Welcome to ``pyproject_ops`` Documentation
==============================================================================
.. image:: https://github.com/MacHu-GWU/pyproject_ops-project/assets/6800411/a5c05a80-04ff-4a29-b637-021b7320f581
    :target: https://github.com/MacHu-GWU/pyproject_ops-project


What is this project?
------------------------------------------------------------------------------
There are various folder structures for Python projects, each with its own merits. Drawing from my extensive Python career experience, I have developed a personal best practice. To streamline the creation of the codebase skeleton according to this practice, I have introduced a tool called `cookiecutter-pyproject <https://github.com/MacHu-GWU/cookiecutter-pyproject>`_.

By adopting this folder structure setup, ``pyproject_ops`` is an automation tool, capable of handling common tasks throughout the Python project development lifecycle. These tasks include "creating virtual environments", "installing dependencies", "running tests", "building documentation sites", and more.

This tool is, in essence, my secret weapon for efficiently managing over 120 Python open source projects, all available on PyPI at https://pypi.org/user/machugwu/. Each of these projects is equipped with essential features such as continuous integration (CI), code coverage testing, matrix testing, and documentation websites.

A little history about this project:

    I previously had an automation tool called `pygitrepo <https://github.com/MacHu-GWU/pygitrepo-project>`_ that was tailored to my former development workflow. ``pygitrepo`` primarily relied on ``setup.py`` and ``requirements.txt`` files. Even today, ``pyproject_ops`` continues to support the old convention while also accommodating ``pyproject.toml`` and ``poetry``. This versatility enables me to write less code when automating Python development workflows.


How to Use
------------------------------------------------------------------------------


Use as a CLI
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: bash

    $ pip install pyproject_ops
    $ pyops --help
    NAME
        pyops - python project ops command line interface.

    SYNOPSIS
        pyops COMMAND | <flags>

    DESCRIPTION
        python project ops command line interface.

    FLAGS
        -v, --version=VERSION
            Type: bool
            Default: False

    COMMANDS
        COMMAND is one of the following:

         build_doc
           ** 📔 Build documentation website locally

         build_doc_only
           📔 Build documentation website locally without checking doc dependencies

         bump_version
           🔼 Bump semantic version.

         cov
           ** 🧪 Run code coverage test

         cov_only
           🧪 Run code coverage test without checking test dependencies

         deploy_latest_doc
           🚀 📔 Deploy Documentation Site To S3 as Latest Doc

         deploy_versioned_doc
           🚀 📔 Deploy Documentation Site To S3 as Versioned Doc

         install
           ** 💾 Install main dependencies and Package itself

         install_all
           ** 💾 💻 🧪 📔 🤖 Install All Dependencies

         install_automation
           💾 🤖 Install Dependencies for Automation Script

         install_dev
           💾 💻 Install Development Dependencies

         install_doc
           💾 📔 Install Document Dependencies

         install_test
           💾 🧪 Install Test Dependencies

         int
           ** 🧪 Run integration test

         int_only
           🧪 Run integration test without checking test dependencies

         poetry_export
           Export requirements-*.txt from poetry.lock file

         poetry_lock
           ** Resolve dependencies using poetry, update poetry.lock file

         publish
           📦 Publish package to PyPI

         test
           ** 🧪 Run test

         test_only
           🧪 Run test without checking test dependencies

         venv_create
           ** 🐍 Create Virtual Environment

         venv_remove
           ** 🗑 🐍 Remove Virtual Environment

         view_cov
           👀 🧪 View coverage test output html file locally in web browser.

         view_doc
           ** 👀 📔 View documentation website locally

         view_latest_doc
           👀 📔 View the latest documentation website on S3


Use as a Python library
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: python

    >>> from pyproject_ops.api import PyProjectOps
    >>> pyops = PyProjectOps.from_pyproject_toml("pyproject.toml")
    >>> pyops.create_virtualenv()
    >>> pyops.pip_install_all()
    >>> pyops.poetry_lock()
    >>> pyops.poetry_install_all()
    >>> pyops.run_cov_test()
    >>> pyops.view_cov()
    >>> pyops.build_doc()
    >>> pyops.view_cov()
    >>> pyops.python_build()
    >>> pyops.twine_upload()


Folder Structure
------------------------------------------------------------------------------
Below is the folder structured used in ``pyproject_ops``. The first item is the relative path from the project root directory. The second item is the attribute name that you can use to access the path in ``pyproject_ops``. The third item is the description of the path. You can find a concrete example at `HERE <https://github.com/MacHu-GWU/cookiecutter-pyproject/tree/main/%7B%7B%20cookiecutter.package_name%20%7D%7D-project>`_

- ``.venv``: ``PyProjectOps.dir_venv``, The virtualenv directory.
- ``.venv/bin``: ``PyProjectOps.dir_venv_bin``, The bin folder in virtualenv.
- ``.venv/bin/pip``: ``PyProjectOps.path_venv_bin_pip``, The pip command in virtualenv.
- ``.venv/bin/pytest``: ``PyProjectOps.path_venv_bin_pytest``, The pytest command in virtualenv.
- ``.venv/bin/python``: ``PyProjectOps.path_sys_executable``, The current Python interpreter path.
- ``.venv/bin/python``: ``PyProjectOps.path_venv_bin_python``, The python executable in virtualenv.
- ``.venv/bin/twine``: ``PyProjectOps.path_bin_twine``, The twine CLI command path.
- ``build``: ``PyProjectOps.dir_build``, The build folder for Python or artifacts build.
- ``build/glue``: ``PyProjectOps.dir_build_glue``, The AWS glue artifacts build folder.
- ``build/glue/extra_py_files``: ``PyProjectOps.dir_build_glue_extra_py_files``, The AWS glue extra Python files build folder.
- ``build/glue/extra_py_files.zip``: ``PyProjectOps.path_build_glue_extra_py_files_zip``, The AWS glue extra Python files zip file path.
- ``build/lambda``: ``PyProjectOps.dir_build_lambda``, The AWS Lambda artifacts build folder.
- ``build/lambda/layer.zip``: ``PyProjectOps.path_build_lambda_layer_zip``, The AWS Lambda layer zip file path.
- ``build/lambda/python``: ``PyProjectOps.dir_build_lambda_python``, The AWS Lambda layer build folder. This folder contains the dependencies.
- ``build/lambda/python/aws``: ``PyProjectOps.path_build_lambda_bin_aws``, This is the AWS CLI executable path in Lambda layer.
- ``build/lambda/source.zip``: ``PyProjectOps.path_build_lambda_source_zip``, The AWS Lambda source code deployment package zip file path.
- ``config``: ``PyProjectOps.dir_config``, The folder that stores the config files.
- ``config/config.json``: ``PyProjectOps.path_config_json``, Path to the JSON file that stores the non-sensitive config.
- ``dist``: ``PyProjectOps.dir_dist``, The dist folder for Python package distribution (.whl file).
- ``docs``: ``PyProjectOps.dir_sphinx_doc``, Sphinx docs folder.
- ``docs/build``: ``PyProjectOps.dir_sphinx_doc_build``, The temp Sphinx doc build folder.
- ``docs/build/html``: ``PyProjectOps.dir_sphinx_doc_build_html``, The built Sphinx doc build HTML folder.
- ``docs/build/html/index.html``: ``PyProjectOps.path_sphinx_doc_build_index_html``, The built Sphinx doc site entry HTML file path.
- ``docs/source``: ``PyProjectOps.dir_sphinx_doc_source``, Sphinx docs source code folder.
- ``docs/source/conf.py``: ``PyProjectOps.dir_sphinx_doc_source_conf_py``, Sphinx docs ``conf.py`` file path.
- ``docs/source/pyproject_ops``: ``PyProjectOps.dir_sphinx_doc_source_python_lib``, The generated Python library API reference Sphinx docs folder.
- ``htmlcov``: ``PyProjectOps.dir_htmlcov``, The code coverage test results HTML output folder.
- ``htmlcov/index.html``: ``PyProjectOps.path_htmlcov_index_html``, The code coverage test results HTML file.
- ``lambda_app``: ``PyProjectOps.dir_lambda_app``, The AWS Lambda app handler file and Lambda related code directory.
- ``lambda_app/.chalice/config.json``: ``PyProjectOps.path_chalice_config``, The AWS Chalice framework's config file path.
- ``lambda_app/.chalice/deployed``: ``PyProjectOps.dir_lambda_app_deployed``, The generated ``deployed.json`` file for AWS Chalice framework's.
- ``lambda_app/app.py``: ``PyProjectOps.path_lambda_app_py``, The app.py file for AWS Chalice framework.
- ``lambda_app/lambda_function.py``: ``PyProjectOps.path_lambda_function_py``, The lambda_function.py handler file for AWS Lambda, if you are not using
- ``lambda_app/update_chalice_config.py``: ``PyProjectOps.path_lambda_update_chalice_config_script``, Example: ``${dir_project_root}/lambda_app/update_chalice_config.py``
- ``lambda_app/vendor``: ``PyProjectOps.dir_lambda_app_vendor``, The vendor folder for AWS Chalice framework's packaging.
- ``lambda_app/vendor/pyproject_ops``: ``PyProjectOps.dir_lambda_app_vendor_python_lib``, The source python library folder in AWS Chalice framework's vendor folder.
- ``poetry-lock-hash.json``: ``PyProjectOps.path_poetry_lock_hash_json``, The poetry-lock-hash.json file path. It is the cache of the poetry.lock file hash.
- ``poetry.lock``: ``PyProjectOps.path_poetry_lock``, The poetry.lock file path.
- ``pyproject.toml``: ``PyProjectOps.path_pyproject_toml``, The pyproject.toml file path.
- ``pyproject_ops``: ``PyProjectOps.dir_python_lib``, The current Python library directory.
- ``pyproject_ops/_version.py``: ``PyProjectOps.path_version_py``, Path to the ``_version.py`` file where the package version is defined.
- ``requirements-automation.txt``: ``PyProjectOps.path_requirements_automation``, The requirements-automation.txt file path.
- ``requirements-dev.txt``: ``PyProjectOps.path_requirements_dev``, The requirements-dev.txt file path.
- ``requirements-doc.txt``: ``PyProjectOps.path_requirements_doc``, The requirements-doc.txt file path.
- ``requirements-test.txt``: ``PyProjectOps.path_requirements_test``, The requirements-test.txt file path.
- ``requirements.txt``: ``PyProjectOps.path_requirements``, The requirements.txt file path.
- ``tests``: ``PyProjectOps.dir_tests``, Unit test folder.
- ``tests_int``: ``PyProjectOps.dir_tests_int``, Integration test folder.
- ``tests_load``: ``PyProjectOps.dir_tests_load``, Load test folder.


Develop and Release Strategy
------------------------------------------------------------------------------
This project is a "meta" project for other projects, it is very hard to test. I keep using this project in many of my production projects, and continuously improving it. I will merge all the changes manually into this every three months.


.. _install:

Install
------------------------------------------------------------------------------

``pyproject_ops`` is released on PyPI, so all you need is:

.. code-block:: console

    $ pip install pyproject_ops

To upgrade to latest version:

.. code-block:: console

    $ pip install --upgrade pyproject_ops