Package: underpin
UnderpinningsUnderpinnings is an experimental utility to explore a two-repo pattern for migrating templated apps from an "applications" repo to a K8s "infrastructure" repo.Infrastructure repos (IRepos) are managed by Dev Ops people or people that are keen to dive into the management of K8s clusters while applications repos (ARepos) are the realm of application developers who build many applications (on many docker images).
Microservices systems lead to a profusion of apps and docker images that need to be managed and deployed into Kubernetes infrastructure. Application developers typically do not care about the details of how things are deployed once they have the resources they need to run their applications.
There are of course many engineers who are happy to work in either space and indeed many teams have arrived at good solutions to manage these environments.Underpinnings takes the perspective that special tooling could be useful sitting between ARepos and IRepos and that some patterns and abstractions are worth thinking about.How to create a separation of concerns between I and AHow to create abstract and simple application templates that can be translated into Kubernetes applicationsHow to manage docker and other container builds and their versions when mapping between A and IUsageUnderpinning creates a target repo for infrastructure. It is useful to think of underpinning running in the context of the source or ARepo, either inline in the repo are as part of builds, building the ARepo. Then the target repo has its own Git Context managed by underpin as a conduit to send transfer manifests.
Consider a basic config fileversion:underpin.io/alpha1metadata:namespace:defaultsource-root:samplesapp-uri:default/deploymentrepos:source:git@github.com:mr-saoirse/underpinnings.gittarget:git@github.com:mr-saoirse/cloud-stack.gitUnderpinnings will initialize the target repo into~/.underpin/cloned/<target>underpin initThen a pipeline can be run to generate all hydrated templates withunderpin runwhich will translates applications undersource-root: samplesand "hydrate" applications for K8s. There are some default templates in underpin or you additional ones can be added to the IRepo inrepo/.unpderpinnings. These are adapters that convert simple values files into more complex K8s app templates.Following this we build any docker images, update image manifests, write to the target IRepo and push. ArgoCD will then manage the applications in the IRepo, syncing to the cluster(s)We build containers and determine tagsunderpin build containersWe then tag manifests. However its assumed for simplicity that we can choose some super tag that we just write into the kustomize in the previous stage and we control the tags or SHA or whatever it is all docker images are re-pinned with for this build.
Allowing for other ways to get those tags is not hard its just unnecessary complexity for a PoC#we dont need this really yet 
underpin build kustomizeWe finally update the infraunderpin infra updateThese steps can all be tested locally but the intent is these run in a CI/CD pipeline runner.running in the workflowThe above is for testing. In the workflow the following occurswe clone the ARepowe Build any master docker images in the ARepowe init the target IRepo on a volumeWe fan out over changed apps with the parameters to process each app and write to the IRepowe rununderpin run -n=<app-name> -o tmp/context/app/buildwe build the docker app from that /tmp/context location, which is a self-contained appthis uses Kaniko as the build provider or Bentoml etc.we set kustomize image on the app for whatever build context and image is neededwe copy to app target in the IRepowe fan in and merge the IRepo changesrunning as a GitHub actionmode 1: trigger the workflowIn this mode, the cloud native in you may want to separate your build processes into something that can be run and tested on K8s. Alternatively see mode 2
In this first mode, we use an Argo Workflow that runs on our management cluster to build our ARepo and transfer the manifests into the IRepo. ArgoCD then takes these manifests and deploys them to all clusters under management.mode 2: run as inline action on app foldersIn mode 2 we use just run underpin in a git action (which runs on a docker image). Here we need to pass app changes into the underpin step and it will transform and generate manifests that get pushed to the target repo (for all app changes). For each app we run all the steps above to generate templates, build containers and tag manifests. We push each app to the target IRepo.other commandsFollowing are some commands for interacting with the Applications repo which is considered the source.#see what changes have been observed on the ARepo branch or in the context
underpin source --changes
#preview a template e..g all docker files and yaml files used for the app
underpin source template <app-name>Underpin also has a simple agent to helm with templatingunderpin agent <>