Package: lib-cicd-github
lib_cicd_githubVersion v1.0.6 as of 2023-10-12 seeChangelogsmall utils for github actions:print colored bannerswrap commands into run/success/error banners, with automatic retryresolve the branch to test, based on the environment variablesautomated tests, Github Actions, Documentation, Badges, etc. are managed withPizzaCutter(cookiecutter on steroids)Python version required: 3.8.0 or newertested on recent linux with python 3.8, 3.9, 3.10, 3.11, 3.12-dev, pypy-3.9, pypy-3.10 - architectures: amd64100% code coverage, flake8 style checking ,mypy static type checking ,tested underLinux, macOS, Windows, automatic daily builds and monitoringTry it OnlineUsageUsage from CommandlineInstallation and UpgradeRequirementsAcknowledgementsContributeReport IssuesPull RequestCode of ConductLicenseChangelogTry it OnlineYou might try it right away in Jupyter Notebook by using the “launch binder” badge, or clickhereUsagerun a command passed as string# to be used in the github action YAML File
# run a command passed as string, wrap it in colored banners, retry 3 times, sleep 30 seconds when retry$>lib_cicd_github_clirun"description""command -some -options"--retry=3--sleep=30get the branch to work on from environment variables$>BRANCH=$(lib_cicd_github_cliget_branch)python methods:install, installs all needed dependencies to build and deploy the projectdefinstall(dry_run:bool=True)->None:"""
    upgrades pip, setuptools, wheel and pytest-pycodestyle


    Parameter
    ---------
    cPIP
        from environment, the command to launch pip, like "python -m pip"


    Examples
    --------

    >>> # Test
    >>> if is_github_actions_active():
    ...     install(dry_run=True)

    """script, run all testsdefscript(dry_run:bool=True)->None:"""
    jobs to run in CICD section "script":
    - run setup.py test
    - run pip with install option test
    - run pip standard install
    - test the CLI Registration
    - install the test requirements
    - install codecov
    - install pytest-codecov
    - run pytest coverage
    - run mypy strict
        - if MYPY_STRICT="True"
    - rebuild the rst files (resolve rst file includes)
        - needs RST_INCLUDE_SOURCE, RST_INCLUDE_TARGET set and BUILD_DOCS="True"
    - check if deployment would succeed, if setup.py exists and not a tagged build

    Parameter
    ---------
    cPREFIX
        from environment, the command prefix like 'wine' or ''
    cPIP
        from environment, the command to launch pip, like "python -m pip"
    cPYTHON
        from environment, the command to launch python, like 'python' or 'python3' on MacOS
    CLI_COMMAND
        from environment, must be set in CICD configuration file - the CLI command to test with option --version
    MYPY_STRICT
        from environment, if pytest with mypy --strict should run
    PACKAGE_NAME
        from environment, the package name to pass to mypy
    BUILD_DOCS
        from environment, if rst file should be rebuilt
    RST_INCLUDE_SOURCE
        from environment, the rst template with rst includes to resolve
    RST_INCLUDE_TARGET
        from environment, the rst target file
    DEPLOY_WHEEL
        from environment, if a wheel should be generated
        only if setup.py exists and on non-tagged builds (there we deploy for real)
    dry_run
        if set, this returns immediately - for CLI tests


    Examples
    --------
    >>> # test
    >>> script()

    """after_success, upload code coverage and codeclimate reportsdefafter_success(dry_run:bool=True)->None:"""
    jobs to run in CICD "after_success":
        - coverage report
        - codecov
        - codeclimate report upload

    it will not run on dry_run or on scheduled event - we dont need to upload
    coverage AGAIN on scheduled run.

    Parameter
    ---------
    cPREFIX
        from environment, the command prefix like 'wine' or ''
    cPIP
        from environment, the command to launch pip, like "python -m pip"
    CC_TEST_REPORTER_ID
        from environment, must be set in CICD configuration file
    dry_run
        if set, this returns immediately - for CLI tests



    Examples
    --------
    >>> # test
    >>> after_success()

    """deploy, deploy to pypidefdeploy(dry_run:bool=True)->None:"""
    uploads sdist and wheels to pypi on success


    Parameter
    ---------
    cPREFIX
        from environment, the command prefix like 'wine' or ''
    PYPI_UPLOAD_API_TOKEN
        from environment, passed as secure, encrypted variable via the GitHub repository secrets
    DEPLOY_SDIST, DEPLOY_WHEEL
        from environment, one of it needs to be true
    dry_run
        if set, this returns immediately - for CLI tests


    Examples
    --------
    >>> # test
    >>> deploy()

    """get_branch, determine the branch to work ondefget_branch()->str:"""
    Returns the branch to work on :
        <branch>    for push, pull requests, merge
        'release'   for tagged releases


    Parameter
    ---------
    github.ref, github.head_ref, github.event_name, github.job
        from environment

    Result
    ---------
    the branch


    Exceptions
    ------------
    none


    ==============  ===================  ===================  ===================  ===================
    Build           github.ref           github.head_ref      github.event_name    github.job
    ==============  ===================  ===================  ===================  ===================
    Push            refs/heads/<branch>  ---                  push                 build
    Custom Build    refs/heads/<branch>  ---                  push                 build
    Pull Request    refs/pull/xx/merge   <branch>             pull_request         build
    Merge           refs/heads/<branch>  ---                  push                 build
    Publish Tagged  refs/tags/<tag>      ---                  release              build
    ==============  ===================  ===================  ===================  ===================

    >>> # Setup
    >>> github_ref_backup = get_env_data('GITHUB_REF')
    >>> github_head_ref_backup = get_env_data('GITHUB_HEAD_REF')
    >>> github_event_name_backup = get_env_data('GITHUB_EVENT_NAME')

    >>> # test Push
    >>> set_env_data('GITHUB_REF', 'refs/heads/development')
    >>> set_env_data('GITHUB_HEAD_REF', '')
    >>> set_env_data('GITHUB_EVENT_NAME', 'push')
    >>> assert get_branch() == 'development'

    >>> # test Push without github.ref
    >>> set_env_data('GITHUB_REF', '')
    >>> set_env_data('GITHUB_HEAD_REF', '')
    >>> set_env_data('GITHUB_EVENT_NAME', 'push')
    >>> assert get_branch() == 'unknown branch, event=push'

    >>> # test PR
    >>> set_env_data('GITHUB_REF', 'refs/pull/xx/merge')
    >>> set_env_data('GITHUB_HEAD_REF', 'master')
    >>> set_env_data('GITHUB_EVENT_NAME', 'pull_request')
    >>> assert get_branch() == 'master'

    >>> # test Publish
    >>> set_env_data('GITHUB_REF', 'refs/tags/v1.1.15')
    >>> set_env_data('GITHUB_HEAD_REF', '')
    >>> set_env_data('GITHUB_EVENT_NAME', 'release')
    >>> assert get_branch() == 'release'

    >>> # test unknown event_name
    >>> set_env_data('GITHUB_REF', '')
    >>> set_env_data('GITHUB_HEAD_REF', '')
    >>> set_env_data('GITHUB_EVENT_NAME', 'unknown_event')
    >>> assert get_branch() == 'unknown branch, event=unknown_event'

    >>> # Teardown
    >>> set_env_data('GITHUB_REF', github_ref_backup)
    >>> set_env_data('GITHUB_HEAD_REF', github_head_ref_backup)
    >>> set_env_data('GITHUB_EVENT_NAME', github_event_name_backup)

    """run, usually used internallydefrun(description:str,command:str,retry:int=3,sleep:int=30,banner:bool=True,show_command:bool=True,)->None:"""
    runs and retries a command passed as string and wrap it in "success" or "error" banners


    Parameter
    ---------
    description
        description of the action, shown in the banner
    command
        the command to launch
    retry
        retry the command n times, default = 3
    sleep
        sleep for n seconds between the commands, default = 30
    banner
        if to use banner for run/success or just colored lines.
        Errors will be always shown as banner
    show_command
        if the command is shown - take care not to reveal secrets here !


    Result
    ---------
    none


    Exceptions
    ------------
    none


    Examples
    ------------

    >>> run('test', "unknown command", sleep=0)
    Traceback (most recent call last):
        ...
    SystemExit: ...

    >>> run('test', "unknown command", sleep=0, show_command=False)
    Traceback (most recent call last):
        ...
    SystemExit: ...

    >>> run('test', "echo test")
    >>> run('test', "echo test", show_command=False)

    """github actions yml File example# This workflow will install Python dependencies, run tests and lint with a variety of Python versions# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actionsname:Python packageon:push:branches:[master,development]pull_request:branches:[master,development]release:branches:[master,development]# release types see : https://docs.github.com/en/actions/reference/events-that-trigger-workflows#release# he prereleased type will not trigger for pre-releases published from draft releases, but the published type will trigger.# If you want a workflow to run when stable and pre-releases publish, subscribe to published instead of released and prereleased.types:[published]schedule:# * is a special character in YAML, so you have to quote this string# | minute | hour (UTC) | day of month (1-31) | month (1-2) | day of week (0-6 or SUN-SAT)# every day at 05:40 am UTC - avoid 05:00 because of high load at the beginning of every hour-cron:'405***'jobs:build:runs-on:${{ matrix.os }}env:# prefix before commands - used for wine, there the prefix is "wine"cPREFIX:""# command to launch python interpreter (it's different on macOS, there we need python3)cPYTHON:"python"# command to launch pip (it's different on macOS, there we need pip3)cPIP:"python-mpip"# switch off wine fix me messagesWINEDEBUG:fixme-all# PYTESTPYTEST_DO_TESTS:"True"# FLAKE8 testsDO_FLAKE8_TESTS:"True"# MYPY testsMYPY_DO_TESTS:"True"MYPY_OPTIONS:"--follow-imports=normal--ignore-missing-imports--implicit-reexport--install-types--no-warn-unused-ignores--non-interactive--strict"MYPYPATH:"./.3rd_party_stubs"# coverageDO_COVERAGE:"True"DO_COVERAGE_UPLOAD_CODECOV:"True"DO_COVERAGE_UPLOAD_CODE_CLIMATE:"True"# package namePACKAGE_NAME:"lib_cicd_github"# the registered CLI CommandCLI_COMMAND:"lib_cicd_github"# the source file for rst_include (rebuild rst file includes)RST_INCLUDE_SOURCE:"./.docs/README_template.rst"# the target file for rst_include (rebuild rst file includes)RST_INCLUDE_TARGET:"./README.rst"# make Code Coverage Secret available in EnvironmentCC_TEST_REPORTER_ID:${{ secrets.CC_TEST_REPORTER_ID }}# make PyPi API token available in EnvironmentPYPI_UPLOAD_API_TOKEN:${{ secrets.PYPI_UPLOAD_API_TOKEN }}strategy:matrix:include:# https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-software# https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#available-versions-of-python-and-pypy-os:windows-latestpython-version:"3.11"env:cEXPORT:"SET"BUILD_DOCS:"False"BUILD:"False"BUILD_TEST:"False"MYPY_DO_TESTS:"True"# Setup testsDO_SETUP_INSTALL:"False"DO_SETUP_INSTALL_TEST:"True"# Test registered CLI CommandDO_CLI_TEST:"True"-os:ubuntu-latestpython-version:"3.8"env:BUILD_DOCS:"False"BUILD:"True"BUILD_TEST:"True"MYPY_DO_TESTS:"True"DO_SETUP_INSTALL:"True"DO_SETUP_INSTALL_TEST:"True"DO_CLI_TEST:"True"-os:ubuntu-latestpython-version:"3.9"env:BUILD_DOCS:"False"BUILD:"True"BUILD_TEST:"True"MYPY_DO_TESTS:"True"DO_SETUP_INSTALL:"True"DO_SETUP_INSTALL_TEST:"True"DO_CLI_TEST:"True"-os:ubuntu-latestpython-version:"3.10"env:BUILD_DOCS:"False"BUILD:"True"BUILD_TEST:"True"MYPY_DO_TESTS:"True"DO_SETUP_INSTALL:"True"DO_SETUP_INSTALL_TEST:"True"DO_CLI_TEST:"True"-os:ubuntu-latestpython-version:"3.11"env:BUILD_DOCS:"True"BUILD:"True"BUILD_TEST:"True"MYPY_DO_TESTS:"True"DO_SETUP_INSTALL:"True"DO_SETUP_INSTALL_TEST:"True"DO_CLI_TEST:"True"-os:ubuntu-latestpython-version:"3.12-dev"env:BUILD_DOCS:"True"BUILD:"True"BUILD_TEST:"True"MYPY_DO_TESTS:"True"DO_SETUP_INSTALL:"True"DO_SETUP_INSTALL_TEST:"True"DO_CLI_TEST:"True"-os:ubuntu-latestpython-version:"pypy-3.9"env:BUILD_DOCS:"False"BUILD:"True"BUILD_TEST:"True"MYPY_DO_TESTS:"True"DO_SETUP_INSTALL:"True"DO_SETUP_INSTALL_TEST:"True"DO_CLI_TEST:"True"-os:ubuntu-latestpython-version:"pypy-3.10"env:BUILD_DOCS:"False"BUILD:"True"BUILD_TEST:"True"MYPY_DO_TESTS:"True"DO_SETUP_INSTALL:"True"DO_SETUP_INSTALL_TEST:"True"DO_CLI_TEST:"True"-os:macos-latestpython-version:"3.11"env:cPREFIX:""# prefix before commands - used for wine, there the prefix is "wine"cPYTHON:"python3"# command to launch python interpreter (it's different on macOS, there we need python3)cPIP:"python3-mpip"# command to launch pip (it's different on macOS, there we need pip3)BUILD_DOCS:"False"BUILD:"True"BUILD_TEST:"True"MYPY_DO_TESTS:"True"# Setup testsDO_SETUP_INSTALL:"True"DO_SETUP_INSTALL_TEST:"True"# Test registered CLI CommandDO_CLI_TEST:"True"name:"${{matrix.os}}Python${{matrix.python-version}}"steps:# see : https://github.com/actions/checkout-uses:actions/checkout@v3-name:Setting up Python ${{ matrix.python-version }}# see: https://github.com/actions/setup-pythonuses:actions/setup-python@v4with:python-version:${{ matrix.python-version }}-name:Install dependencies# see: https://github.community/t/github-actions-new-bug-unable-to-create-environment-variables-based-matrix/16104/3env:${{ matrix.env }}# make matrix env variables accessible# lib_cicd_github install: upgrades pip, setuptools, wheel and pytest-pycodestylerun:|${{ env.cPIP }} install git+https://github.com/bitranox/lib_cicd_github.gitlib_cicd_github install-name:Debug - printenv and colortestenv:# make matrix env variables accessible${{ matrix.env }}shell:bashrun:|# export for current stepexport "BRANCH=$(lib_cicd_github get_branch)"# export for subsequent stepsecho "BRANCH=$BRANCH" >> $GITHUB_ENVlog_util --level=SPAM  "working on branch $BRANCH"log_util --level=SPAM  "GITHUB_REF $GITHUB_REF"log_util --level=VERBOSE "github.base_ref: ${{ github.base_ref }}"log_util --level=VERBOSE "github.event: ${{ github.event }}"log_util --level=VERBOSE "github.event_name: ${{ github.event_name }}"log_util --level=VERBOSE "github.head_ref: ${{ github.head_ref }}"log_util --level=VERBOSE "github.job: ${{ github.job }}"log_util --level=VERBOSE "github.ref: ${{ github.ref }}"log_util --level=VERBOSE "github.repository: ${{ github.repository }}"log_util --level=VERBOSE "github.repository_owner: ${{ github.repository_owner }}"log_util --level=VERBOSE "runner.os: ${{ runner.os }}"log_util --level=VERBOSE "matrix.python-version: ${{ matrix.python-version }}"printenvlog_util --colortest-name:Run Testsenv:# make matrix env variables accessible${{ matrix.env }}shell:bashrun:|# export for current stepexport "BRANCH=$(lib_cicd_github get_branch)"# export for subsequent stepsecho "BRANCH=$BRANCH" >> $GITHUB_ENV# run the testslib_cicd_github script-name:After Successenv:${{matrix.env }}shell:bashcontinue-on-error:truerun:|lib_cicd_github after_success-name:Deployenv:# see: https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#github-context# see : https://github.com/rlespinasse/github-slug-action# make matrix env variables accessible${{matrix.env }}shell:bashrun:|lib_cicd_github deployUsage from CommandlineUsage: lib_cicd_github [OPTIONS] COMMAND [ARGS]...

  CI/CD (Continuous Integration / Continuous Delivery) - utils for github
  actions

Options:
  --version                     Show the version and exit.
  --traceback / --no-traceback  return traceback information on cli
  -h, --help                    Show this message and exit.

Commands:
  after_success  coverage reports
  deploy         deploy on pypi
  get_branch     get the branch to work on
  info           get program informations
  install        updates pip, setuptools, wheel, pytest-pycodestyle
  run            run string command wrapped in run/success/error banners
  script         updates pip, setuptools, wheel, pytest-pycodestyleInstallation and UpgradeBefore You start, its highly recommended to update pip and setup tools:python -m pip --upgrade pip
python -m pip --upgrade setuptoolsto install the latest release from PyPi via pip (recommended):python -m pip install --upgrade lib_cicd_githubto install the latest release from PyPi via pip, including test dependencies:python -m pip install --upgrade lib_cicd_github[test]to install the latest version from github via pip:python -m pip install --upgrade git+https://github.com/bitranox/lib_cicd_github.gitinclude it into Your requirements.txt:# Insert following line in Your requirements.txt:
# for the latest Release on pypi:
lib_cicd_github

# for the latest development version :
lib_cicd_github @ git+https://github.com/bitranox/lib_cicd_github.git

# to install and upgrade all modules mentioned in requirements.txt:
python -m pip install --upgrade -r /<path>/requirements.txtto install the latest development version, including test dependencies from source code:# cd ~
$ git clone https://github.com/bitranox/lib_cicd_github.git
$ cd lib_cicd_github
python -m pip install -e .[test]via makefile:
makefiles are a very convenient way to install. Here we can do much more,
like installing virtual environments, clean caches and so on.# from Your shell's homedirectory:$gitclonehttps://github.com/bitranox/lib_cicd_github.git$cdlib_cicd_github# to run the tests:$maketest# to install the package$makeinstall# to clean the package$makeclean# uninstall the package$makeuninstallRequirementsfollowing modules will be automatically installed :## Project Requirementsclickcli_exit_toolslib_detect_testenvlib_log_utilsrst_includeAcknowledgementsspecial thanks to “uncle bob” Robert C. Martin, especially for his books on “clean code” and “clean architecture”ContributeI would love for you to fork and send me pull request for this project.
-please ContributeLicenseThis software is licensed under theMIT license—Changelognew MAJOR version for incompatible API changes,new MINOR version for added functionality in a backwards compatible mannernew PATCH version for backwards compatible bug fixesv1.0.62023-10-12:do not upload codecov to codeclimate on scheduled buildsv1.0.52023-10-09:implement PyPI Upload API Tokenv1.0.42023-10-09:repair scheduled run detectionv1.0.32023-10-08:do not upload codecov on scheduled builds, because of error
‘Too many uploads to this commit.’ when upload codecov again and again.v1.0.22023-07-21:create mypy cache dir ‘.mypy_cache’require minimum python 3.8remove python 3.7 testsintroduce PEP517 packaging standardintroduce pyproject.toml build-systemremove mypy.iniremove pytest.iniremove setup.cfgremove setup.pyremove .bettercodehub.ymlremove .travis.ymlupdate black configclean ./tests/test_cli.pyadd codeql badgemove 3rd_party_stubs outside the src directory to./.3rd_party_stubsadd pypy 3.10 testsadd python 3.12-dev testsv1.0.1.22022-06-02: update to github actionscheckout@v3andsetup-python@v3v1.0.12022-03-29: remedy mypy Untyped decorator in cliv1.0.02022-03-25:initial pypi releaseupdate documentation and testslist ./dist dir if existingv0.0.12021-08-23: initial release