Package: coveo-stew
Upgrading from 2.x?The 3.0 update contains breaking changes:poetry,mypyandblackare no longer distributed withcoveo-stew.Please refer to theupgrade guidefor more information and resolution steps.coveo-stewcoveo-stew delivers a complete Continuous Integration (CI) and Continuous Delivery (CD) solution
usingpoetryas its backend.CI toolsConfig-free pytest, mypy and black runnersAdd your own linters and toolsJUnit report generationGitHub Action that runs all your CI toolsSimilar to: toxCD toolsGitHub Action for Continuous Delivery (CD) (publish to pypi)Automated "patch" version bumps (requires apypiserver)Can download locked dependencies into a folder, for offline distributionSimilar to: poetry, flit, pbr, setuptoolsMultiple projectsSupport for several isolated poetry projects in one GitHub repositorySupport for local path referencesA "one ring to rule them all" virtual environment that includes all subprojects within the repositoryBatch operationsNote: Single projects are also supported! 😅Similar to: nothing! it's unique! 😎PrerequisitesChanged in 3.0:poetryis no longer provided out-of-the-box.You needpoetryinstalled on your system, and it must be available through thePATH.The3.0version ofcoveo-stewis designed to work with modern versions ofpoetry.While it is compatible with older versions, old poetry issues
such asthisandthiswill causestewto misbehave.If you need to work with an older version of poetry,
consider usingcoveo-stew < 3.0which had workarounds implemented around these issues.InstallationJust like poetry,stewis a CLI tool that you install in your system.It is recommended to install usingpipxin order to isolate this into a nice little space:pip install pipx --user
pipx install coveo-stewIf you don't use pipx, make sure to isolate the installation into a virtual environment.GitHub ActionThis action checkouts the code, installs python, poetry and stew, and proceeds to run "stew ci" on a python project.Usagejobs:
  stew-ci:
    runs-on: ubuntu-latest
    steps:
      - uses: coveo/stew@main
        with:
          python-version: "3.10"
          project-name: your-project-nameSee additional options and documentation inthe action file.GitHub Action Step ReportWhen running in a GitHub context, a step summary will automatically be generated.
This summary can be seen in theSummarytab of a workflow run view in GitHub.You can disable this feature by launchingstew ciwith--no-github-step-report.Repository StructurePlease read these guides in order to learn how to organize your repository for maximum compatibility:For a single libraryFor multiple librariesCommandsGeneral command usageUnless a project name is specified, most commands will operate on all projects in a git repository based on the current working folder:stew <command>Perform a command on all projectsstew <command> --helpObtain help about a particular commandstew <command> <project-name>Perform the command on all projects with<project-name>in their name (partial match)stew <command> <project-name> --exact-matchDisable partial project name matchingThe main commands are explained below.stew ciThe main show; it runs all CI tools on one or multiple projects.Errors will show in the console, and junit xml reports will be generated inside the.cifolder.Without configuration, this command will run the following checks:mypy (using opinionated, strict rules)Note:mypyis not provided withcoveo-stew. Seebuiltin-runnersfor info.poetry checkstew check-outdatedOptions:--fixwill reformat the code ifblackfails. Additional fix routines may be added in the future.--check <runner>will launch only that runner. This option can be repeated.--skip <runner>will skip that runner. Takes precedence over--check. This option can be repeated.--quickskips runningpoetry install --remove-untrackedbefore running the checks.--no-github-step-reportcan be used to disable Step Report generation when running in a GitHub context.The configuration for this feature is explained in more details in therunnerssection.stew buildStore the project and itslocked dependenciesto disk, so it can be installed without contacting apypiserver.Optimally used to create truly repeatable builds and workflows (e.g.: containerized images, cloud storage, etc).The folder can later be installed offline withpip install --no-index --find-links <folder> <project-name>.Options:--directoryspecifies where the wheels should be downloaded.--pythonmay be used to target a different python. It's important to use the same python version, architecture and OS than the target system.Make sure your target<folder>is clean: Keep in mind thatpipwill still use thepyproject.tomlconstraints when installing, notpoetry.lock.
The locked version system works when the locked version is the only thing thatpipcan find in the<folder>.stew check-outdatedandstew fix-outdatedChecks for out-of-date files or automatically update them.Summary of actions:poetry lockifpyproject.tomlchanged but not thepoetry.lockstew pull-dev-requirementsif a pydev project's dev-requirements are out of syncstew pull-dev-requirementsOnly useful onpydevprojects (see aboutmultiple-libraries).
It pulls the dev requirements from the local projects in order to aggregate them into the dev requirements of the root project.Note: This command uses thedevgroup to identify test dependencies.
This can be defined astool.poetry.dev-dependencies(poetry 1.1.15) ortool.poetry.group.dev.dependencies.
The group name cannot be customized, feel free to contribute the feature if that's useful to you.stew bumpCallspoetry lockon all projects.stew refreshCallspoetry installon all projects.stew fresh-eggsClears the.egg-infofolder from your projects. Usually followed by apoetry installor astew refresh.Use this if you change a[tool.poetry.scripts]section, else the changes will not be honored.stew locate <project>Returns the path to a project:$ stew locate coveo-stew
/home/jonapich/code/stew/coveo-stewConfigurationConfiguration is done through eachpyproject.tomlfile; default values are shown:[tool.stew.ci]
mypy = true
pytest = false
black = false
poetry-check = true
check-outdated = true
offline-build = falseYou don't have to include the[tool.stew.ci]section at all if these defaults suit you!Runners (stew ci)Changed in coveo-stew 3.0: mypy and black are no longer provided out-of-the-box.In order to use a builtin or custom runner, you must have it installed. These locations are supported:Recommended: The runner is in the project's virtual environment (most likely as a dev dependency inpyproject.toml)Alternative: The runner is installed in your system and available through the PATHWe strongly suggest pinning them to yourpyproject.tomlfile in the[tool.poetry.dev-dependencies]section.This way, mypy won't surprise you with new failures when they release new versions! 😎Note: You can override and customize most runners byrewriting them as custom runners.Builtin RunnersmypyA strict mypy configuration is provided.You can provide your own mypy configuration, but you'll have to specify theset-configoption:[tool.stew.ci]

# disable stew's strict mypy config (i.e.: let mypy find its config)
mypy = { set-config = False }

# use a specific config (path relative to `pyproject.toml`'s folder)
mypy = { set-config = "mypy.ini" }Seehttps://mypy.readthedocs.io/en/stable/config_file.html#using-a-pyproject-tomlpytestPytest is configured to run with--doctest-modules --tb=short --durations=5as well as JUnit report generation.Some additional options are available:[tool.stew.ci]

# configure the markers to test (i.e.: `-m`)
pytest = { marker-expression = 'not docker_tests' }

# disable the doctests
pytest = { doctest-modules = False }blackBlack supports thepyproject.tomlfile natively:[tool.black]
line-length = 100Ref:black documentationpoetry-checkRunspoetry checkon each project.check-outdatedRunsstew check-outdated.Note: This runner cannot be overridden, but it can be disabled.offline-buildRunsstew buildto a temporary folder and ensures that pip is able to reinstall everything from there.Note: This runner cannot be overridden, but it can be disabled.Custom RunnersYou can add your own runners tostew ci.
You can also redefine a builtin runner completely.In this example, we create runners for flake8, bandit and isort. We also redefine the pytest runner:[tool.stew.ci.custom-runners]
flake8 = true
bandit = { check-args = ["--quiet", "--recursive", "."] }

# some may prefer this toml syntax:
[tool.stew.ci.custom-runners.isort]
check-args = ["--check", ".", "--profile black"]
autofix-args = [".", "--profile black"]

[tool.stew.ci.custom-runners.pytest]
check-args = ["--tb=long", "--junitxml=.ci/pytest-results.xml"]When a builtin runner such as pytest is redefined as a custom runner, you must provide all the arguments.
In this case, not passing--junitxmlwould mean that we lose the report that used to be in the.ci/directory.OptionsThe following options are supported for custom runners:name: You can specify the module name if it differs from the name of the tool.Important: Runners are called throughpython -m <name>, not through the shell!check-args: The arguments to invoke the check.autofix-args: The arguments to invoke the autofix. Provide the empty string "" in order to run without arguments.check-failed-exit-codes: A list of ints denoting the exit codes to consider "failed" (anything else will be "error"). 0 is always a success. default is[1].create-generic-report: Whether to create a generic pass/fail JUnit report for this check.working-directory: The default is "project" which corresponds to the project'spyproject.tomlfile. You can change it to "repository" in order to run from the root.Theargsandcheck-argscan be:A stringSuch as a single argument "--check"Such as a path "."Such as an option "--profile black"But NOT as a combo of the above: "--check . --profile black" will most likely not work.A list of string:Any combination of the "string" rules explained above.FAQ, Tips and Tricksconstraints vs locks - where do they apply?When you callpoetry install, you end up installing packages based on thepoetry.lockfile.
The resulting packages will always be the same, no matter what.
This is the dev scenario.When you callpip install, you are installing packages based on the constraints placed in apyproject.tomlor asetup.pyfile.
Unless the constraints are hard pinned versions, the resulting packages are not guaranteed and will depend on the point in time when the installation is performed, among other factors.
This is the shared library scenario.When you use poetry, you cover the two scenarios above.The third scenario is the private business use case: you want to freeze your dependencies in time so that everything from the developer to the CI servers to the production system is identical.
Essentially, you wantpoetry installwithout the dev requirements.This functionality is provided out of the box bystew build, which creates a pip-installable package from the lock file that you can then stash in a private storage of your choice or pass around your deployments.How to provision a business production system / how to freeze your project for "offline" distributionYou can keeppoetryandstewoff your production environment by creating a frozen archive of your application or library from your CI servers (docker used as example):Use thestew buildtool which:performs apoetry buildon your projectcallspip downloadbased on the content of the lock fileMoves the artifacts to the.wheelsfolder of your repoRecommended: Use the--pythonswitch when callingstew buildto specify which python executable to use! Make sure to use a python interpreter that matches the os/arch/bits of the system you want to provision.The content in.wheelscan then be zipped and moved around. A typical scenario is to push it into a Docker Container:Include the.wheelsfolder into your Docker build contextIn your Dockerfile:ADD the.wheelsfolderManage thepipversion! Either update it to latest, or pin it to something.Prepare a python environmentUsepython -m venv <location>to create a virtual environment natively.Note the executable location... typically (location/bin/pythonorlocation/Scripts/python.exe)Install your application into the python environment you just created:Use<venv-python-executable> -m pip install <your-package> --no-index --find-links <wheels-folder-location>You may delete the.wheelsfolder if you want. Consider keeping a copy of the lock file within the docker image, for reference.To make sure you use the python interpreter that matches the os/arch/bits of the system you want to provision, you can runstew builddirectly when building the container image.
In order to do so without packagingstewin production, you can usemulti-stage builds.How to hook your IDE with the virtual environmentIf your IDE supports poetry, it should detect and use thepyproject.tomlfile.To set it up manually:Callpoetry installfrom the location of thepyproject.tomlfileObtain the location of the virtual environment (i.e.:poetry env list --full-path)Configure your IDE to use the python interpreter from that locationYour IDE should proceed to analyze the environment and will pick up all imports automatically,
regardless of your PYTHONPATH or your working directory.
Since the local source is editable, any change to the source code will be reflected on the next interpreter run.If you use the multiple-projects approach, you should hook your IDE to thepydevenvironment. Seethis documentationfor more information.Using the virtual environment without activating itUsing the correct interpreter is all you need to do.
There is no activation script or environment variables to set up: the interpreter executable inside the virtual environment folder is a fully bootstrapped and isolated environment.A python dockerfile may call<venv-python-exec>directly in the dockerfile's CMDA service that spawns other processes should receive the path to the<venv-python-exec>Use the-mswitchin order to launch your app!