Package: octo-pipeline-python
Octo PipelineThe Octo pipeline gives the ability to work both with an actual pipeline such as jenkins / github actions etc
and locally on a machineThe pipeline consists of the following abbreviations:Backend - Some backend that can execute actionsAction - An action that needs to happen on the backend, such as consume, buildPipeline - A set of actions that run on different backendsWorkspace - A set of pipelines with dependencies between themEach component above can be controlled via the octo executableInstallingThe pipeline requires Python 3.8+In order to install octo-pipeline, you can install it directly from pypi:pip3installocto-pipeline-pythonDo notice that we use "extras" for our pipeline, each sub library of the pipeline is a specific backend that you can choose to install or not
Choosing "all" will install the pipeline along with all the backendspip3installinstallocto-pipeline-python[all]Once the above is done, the pipeline should be installed along with all the dependencies on the machineYou can remove the pip.conf if you don't want to use the index anymoreEnable AutoCompleteAuto-completion for CLI arguments can be enabled as described in theargcomplete#zsh-supportpackage which is part of the requirements for octo.Source the following in your.bashrcor.zshrc:eval"$(register-python-argcompleteocto)"Pipeline DescriptionThe pipeline is defined by a yaml file and an optional yaml parameters fileThe main pipeline yaml file can look for example as followingname:curlppscm:https://github.com/jpbarrette/curlpp.gitversion:0.8.1maintainers:-jpbarrettepipeline:-source:backend:gitsurroundings:-jenkins-local-consume:backend:conansurroundings:-jenkins-local-build:backend:conansurroundings:-jenkins-local-install:backend:conansurroundings:-local-package:backend:conansurroundings:-jenkins-on-demandThis will define a pipeline for the repository, and will run the actions sequentialEach action is defined by a set of backends it needs to run onEach backend can define the actions it supports accordinglyWe can also supply a "name" to the action such that:-build:name:build-winbackend:golangsurroundings:-jenkins-on-demand-build:name:build-linuxbackend:golangsurroundings:-jenkins-on-demandSo that we can support multiple actions of the same type for different use casesLasty, each action can define on which surroundings it can run onCurrently the supporting surroundings arejenkinslocalon-demandworkspaceAlong with the above file, you can define parameters for different backends by either specifiying backends key on the pipeline yaml file, or defining a backends.yml fileThe backends yaml file can look as followsgit:head:v0.8.1conan:artifactory:https://jfrog.comconfigurations:-debug-releaseFor each backend you will define a set of parameters that may or may not be usedIf a specific action settings need to apply, or settings for a platform, this can be done as follows:golang:actions:build-win:-platform:linuxsettings:env:GOOS:windowsGOARCH:amd64GO111MODULE:"on"CC:i686-w64-mingw32-gccCXX:i686-w64-mingw32-g++entrypoints:-cmd/provision_win.go-cmd/deprovision_win.goNotice that we can use "build" or "build-win" as action type or action name to define settings for our specific action
Those settings are only applied on the linux platformWorking with the pipelineOnce the pipeline is installed, you can either clone a repository of an existing pipeline and work on ur ownOr u can use the pipeline to help you outFor example, if you wish to work "curlpp" existing pipeline, you could perform the following commandoctopipelineinit--org=pas--name=curlppThe above will try and find inside pas organization the repository, check if it has a pipeline and clone it accordinglyOnce the above is done, you can execute the pipeline for that component by runningoctopipelineexecuteThis will trigger the pipeline defined for the componentYou could also run specific actions by running the followingoctopipelineexecute-actionconsumeAnd lastly, you could run the pipeline step by step and control the flow on ur ownThis can be done by using the following commands in a queue styleoctopipelinestepnext
octopipelinestepprevious
octopipelinestepcurrent
octopipelinestepexecute
octopipelinestepclean
octopipelinestepresetAll of the above will control each step you are running onAll of the pipeline actions can be listed with --help:usage:octo.pypipeline[-h]{init,execute,describe,describe-actions,clean,name,version,build_number,scm,execute-action,clean-action,step}...

positionalarguments:{init,execute,describe,describe-actions,clean,name,version,build_number,scm,execute-action,clean-action,step}initInitializesthepipelinebackendsandworkingdirectoryexecuteExecutestheentirepipelineactionsdescribePrintsoutadetaileddescriptionofthepipelinedescribe-actionsPrintsoutthelistofactionsonthepipelinecleanCleansupalltheactionsthatransofarnamePrintsoutthenameofthepipelineversionPrintsouttheversionofthepipelinebuild_numberPrintsoutthebuildnumberofthepipelinescmPrintsoutthescmofthepipelineexecute-actionExecutesaspecifictypeofactionclean-actionCleansupaspecifictypeofactionstepStepexecutionofthepipeline

optionalarguments:-h,--helpshowthishelpmessageandexitWorking with a specific backendYou can execute specific backend operations such as authenticationThe backend is responsible for keeping the state of authentication if working via the CLIIf you wish to authenticate to a backend, you may runoctobackendsconanauthenticate--username=john--secret=john--certificate="/path/to/cert"You may also add a target argument if you wish to authenticate to a specific resource in the backendYou may execute other operations on the backed as described in the helpusage:octobackendsconan[-h]{authenticate,describe,working-dir,get}...

positionalarguments:{authenticate,describe,working-dir,get}authenticateAuthenticatetoabackenddescribePrintsoutadescriptionofthebackendworking-dirPrintsouttheworkingdirectoryofthebackendgetGetsaspecifickeyfromthecontextofthebackend

optionalarguments:-h,--helpshowthishelpmessageandexitWorking with a workspaceA workspace is a set of pipelines that can run in certain orderThe workspace is defined by a yaml that can look similar to the followingname:coolspacescm:git@github.comorganizations:-coolorgworkspace:-3rdparties:-curlpp-nlohmann-json-stduuid-aws-sdk-cpp-core:-graphics-loggerWhere you can define which organizations in the scm to look for the repoAnd define the workspace layout, and who depends on whoAlong with that, you may also define settings that can propogate to the pipelines in a workspace levelThose settings can be divided by systemThe workspace can be initialized by running the following commandoctoworkspaceinit--org=pas--name=coolorg-wsThis will try to find the workspace and clone it, and along with it, clone the entire actual workspace definitionIf you do not wish to also sync the workspace on init, add --no-syncAnd you can later run the sync yourself withoctoworkspacesyncAfterwards, you may choose to run any of the following commandsoctoworkspaceexecuteall
octoworkspaceexecute-actionallbuild
octoworkspaceexecute3rdparties
octoworkspaceexecutegraphics
octoworkspaceexecute-actiongraphicsbuild
octoworkspacecleanall
octoworkspaceclean-actionallbuild
octoworkspaceclean3rdparties
octoworkspacecleangraphics
octoworkspaceclean-actiongraphicsbuild
octoworkspacedescribeThe above commands can execute the entire workspace pipeline, specific pipeline based on the workspace yml, or cleanup accordingly