Portal:Toolforge/Admin/Build Service

Documentation of components and common admin procedures for Toolforge/Build Service.
== Components ==
* Builds client (source code): main entrypoint for users
* Builds API (source code): main entry-point for clients (users use the cli)
* Builds builder (source code): underlying building system
* Harbor (puppet code): Hosts the images the users create
** Tools: https://tools-harbor.wmcloud.org
** Toolsbeta: https://toolsbeta-harbor.wmcloud.org
== Alerts ==
* From the cloud UI: https://prometheus-alerts.wmcloud.org/?q=%40state%3Dactive&q=project%3D~^%28tools%7Ctoolsbeta%29
* From the prod UI: https://alerts.wikimedia.org/?q=team%3Dwmcs&q=project%3D~%28tools%7Ctoolsbeta%29
== Dashboards ==
You can find all the current dashboards here:
https://grafana.wmcloud.org/d/m9V1RQs4k/harbor-overview?orgId=1
== Administrative tasks ==
=== Starting a service ===
==== Harbor ====
Ssh to the harbor instance (ex. toolsbeta-harbor-1.toolsbeta.eqiad1.wikimedia.cloud):
==== Buildservice API ====
This lives in kubernetes, behind the API gateway. To start it you can try redepolying it, to do so follow Portal:Toolforge/Admin/Kubernetes#Deploy_new_version (the component is toolforge-builds-api).
You can monitor if it's coming up with the usual k8s commands:
==== Tekton ====
Similar to the builds api, tekton is a k8s component, you can try redepolying it too following Portal:Toolforge/Admin/Kubernetes/Components#Deploy (the component is buildservice).
You can monitor if it's coming up with the usual k8s commands:
=== Stopping a service ===
==== Harbor ====
Ssh to the harbor instance (ex. toolsbeta-harbor-1.toolsbeta.eqiad1.wikimedia.cloud):
==== Buildservice API ====
Being a k8s deployment, the quickest way might be just to remove the deployment itself (will require redeploying to start again).
For a full removal (CAREFUL! Only if you know what you are doing) you can use helm:
==== Tekton ====
This one is a bit more tricky, but it would be removing the tekton controller itself (the one that handles the PipelineRun and TaskRun resources).
NOTE: Tekton does not have yet a helm deployment associated with it
=== Checking all components are alive ===
We don't have a unified dashboard yet, but for now you can check each component individually.
You can check the dashboards as a starting point. For the rest keep reading:
==== Harbor ====
Ssh to the harbor instance (ex. toolsbeta-harbor-1.toolsbeta.eqiad1.wikimedia.cloud):
==== Buildservice API ====
You can monitor if it's coming up with the usual k8s commands:
==== Tekton ====
Same as before, different namespace:
=== Updating the builder image ===
You can find the latest info on the builds-builder git repo.
=== Adding a new buildpack ===
We keep a fork for the buildpacks we inject under https://gitlab.wikimedia.org/groups/repos/cloud/toolforge/buildpacks
Currently we are using the heroku builder and the heroku buildpacks, that usually are in the old heroku structure (as opposed to cloud-native). So we have to add a shim layer to make them cloud-native compatible.
You can see the latest examples in the repository, but a good start is to pull the buildpack from the cnb heroku url https://buildpack-registry.heroku.com/cnb/emk/rust where the last two parts of the path are the author and the name of the buildpack.
That will include a few scripts and files that are valid for cloud-native buildpack API 0.4, but we have to adapt them to API 0.6 at least as that's the minimum supported by the builder as of writting this.
For that, we have to:
* Add a project.toml file
* Change the api entry in the bulidpack.toml file
* Change everywhere where a layer toml file is created to have the newer structure (see the existing examples).
You can create a new branch with the changes.
Once the fork is ready, we can add the buildpack to the list of buildpacks to inject in the builds-builder (see examples there).
Note that this might change soon for a nicer/easier flow, as we are just starting to discover how to manage these things.
== History ==
See Help:Toolforge/Build Service#History.