mpyl 1.8.7
Modular Pipeline Library
Requires Python: >=3.12

# Modular Pypeline Library
![build-and-test-module](https://github.com/Vandebron/pympl/actions/workflows/build-package.yml/badge.svg?branch=main)
![coverage](https://camo.githubusercontent.com/359bec0496fb67fc3a3deeb1917cc403a23890fc54c2efc37d59251a80e004d9/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f436f6465253230436f7665726167652d38302532352d79656c6c6f773f7374796c653d666c6174)
[![publish-documentation](https://github.com/Vandebron/mpyl/actions/workflows/docs.yml/badge.svg?branch=main)](https://vandebron.github.io/mpyl)
[![Checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)

[![linting: pylint](https://img.shields.io/badge/linting-pylint-yellowgreen)](https://github.com/PyCQA/pylint)
<a href="https://github.com/psf/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>

[![version](https://img.shields.io/github/v/tag/Vandebron/pympl.svg?color=blue&include_prereleases=&sort=semver)](https://pypi.org/project/mpyl/)
[![python](https://img.shields.io/badge/Python-3.9-3776AB.svg?style=flat&logo=python&logoColor=white)](https://www.python.org)
[![package downloads](https://img.shields.io/pypi/dw/mpyl.svg)](https://pypi.org/project/mpyl)
[![mpyl](https://snyk.io/advisor/python/mpyl/badge.svg)](https://snyk.io/advisor/python/mpyl)

# What is MPyL?

MPyL stands for Modular Pipeline Library (in Python).

This library is loosely based on the principles described in https://www.jenkins.io/blog/2019/01/08/mpl-modular-pipeline-library/
but completely independent of Jenkins or any other CI/CD platform.

## 🔖 Documentation
Detailed, *complete*, searchable documentation can be found at [https://vandebron.github.io/mpyl](https://vandebron.github.io/mpyl)

## 📚 Principles

MPyL is built with the following principles in mind.
- **Not a platform** It is not tied to any CI/CD platform. MPyL is a library, not a framework. It is not a runner,
  not a scheduler, and it doesn't have a GUI. It can be plugged into any CI/CD platform. Execution flows for
  Jenkins or Dagster are included as an example.
- **Minimal footprint**: It is self-contained and has very few external dependencies (e.g. Git, Docker).
  It can be run anywhere, also on your local machine.
- **Accessible and maintainable**
    - Written in Python, the most widely adopted scripting language with an extensive amount of client libraries relevant to CI/CD.
    - Strongly typed: `MyPy` type hints and schemas for all `YAML` files. Clearly defined interfaces for inputs
      and outputs of steps.
    - Focus on short feedback loop. Unit testable and everything can be run locally.
- **Self documented** by [docstrings](https://vandebron.github.io/mpyl) in the code, descriptions in the schemas
  and explanations in the CLI.
- **Python as YAML templating engine** Attempts to mix markup and logic tend to have a
  [worst of both worlds](https://anthonyhawkins.medium.com/is-python-the-perfect-json-yaml-templating-engine-c5c1b32418f6)
  outcome. MPyL uses `YAML` strictly for configuration. Where `YAML` needs to produced dynamically, like for CRDs via
  HELM charts, it is generated by Python code. The produced `YAML` is validated against the corresponding schemas.
- **Extensible** Support for the most common use cases (e.g. Docker, Helm, Kubernetes, AWS, etc.) is built-in.
  But it is easy to extend with your own `mpyl.steps`.

## 💻 Technologies

### Requirements
The following technologies are expected to be present on the local OS:
- [Python](https://www.python.org/) >= 3.9
- [Pip](https://pypi.org/project/pip/) >= 23.0.1
- [Docker](https://www.docker.com/) > 20
- [Docker compose](https://docs.docker.com/compose/install/linux/)
  installed as plugin (`docker compose version`) >= v2.2.3
- [Git](https://git-scm.com/) SCM

### Bundled
MPyL is extensible and has a minimal footprint. Having said that, batteries for the following technologies are included.

##### CI/CD
###### Build
- [Docker](https://www.docker.com/) `mpyl.steps.build.dockerbuild`
- [Scala (SBT)](https://www.scala-sbt.org/) `mpyl.steps.build.sbt`
- [Jenkins](https://www.jenkins.io/)  `mpyl.cli.commands.build.jenkins` and
  `mpyl.utilities.jenkins.runner.JenkinsRunner`

###### Testing
- [Junit](https://junit.org/) `mpyl.steps.models.ArtifactType.JUNIT_TESTS`

###### Deployment
- [K8S](https://kubernetes.io/) `mpyl.steps.deploy.kubernetes`
- [Helm](https://helm.sh/) `mpyl.steps.deploy.k8s.helm`

##### Reporting
- [Jira](https://www.atlassian.com) `mpyl.reporting.targets.jira`
- [Github](https://github.com/) `mpyl.reporting.targets.github`
- [Slack](https://slack.com/) `mpyl.reporting.targets.slack`