Project: Linkerd (Orchestration & Management / Service Mesh)
File: BUILD.md

<!-- markdownlint-disable-file code-block-style --> :balloon: Welcome to the Linkerd2 development guide! :wave: This document will help you build and run Linkerd2 from source. More information about testing from source can be found in the guide. - - - - - - - - - - - - [Updating ServiceProfile generated code](#updating-serviceprofile-generated-code) - - - Linkerd2 is primarily written in Rust, Go, and React. At its core is a high-performance data plane written in Rust. The control plane components and its extensions are written in Go. The dashboard UI is a React application. : Command-line `linkerd` utility, view and drive the control plane. - : Accepts requests from `proxy` instances and serves service discovery information. : Mutating webhook triggered by pods creation, that injects the proxy container as a sidecar. : Provides a CA to distribute certificates to proxies for them to establish mTLS connections between them. - : Accepts requests from API clients such as cli and web, serving metrics from the proxies in the cluster through Prometheus queries. : Provides a live pipeline of requests. : Mutating webhook triggered by pods creation, that injects metadata into the proxy container in order to enable tap. : Provides a UI dashboard to view and drive the control plane. - [`linkerd-gateway`]: Accepts requests from other clusters and forwards them to the appropriate destination in the local cluster. : Controller observing the labeling of exported services in the target cluster, each one for which it will create a mirrored service in the local cluster. - : Mutating webhook triggered by pods creation, that expands the proxy container for it to produce tracing spans. : Rust source code for the proxy lives in the linkerd2-proxy repo. : Protobuf definitions for the data plane APIs live in the linkerd2-proxy-api repo. <!-- markdownlint-disable no-inline-html --> <details> <summary></summary> linkerd2_components digraph G { rankdir=LR; node [style=filled, shape=rect]; "cli" [color=lightblue]; "destination" [color=lightblue]; "identity" [color=lightblue]; "metrics-api" [color=lightblue]; "tap" [color=lightblue]; "web" [color=lightblue]; "proxy" [color=orange]; "cli" -> "metrics-api"; "cli" -> "tap"; "web" -> "metrics-api"; "web" -> "tap"; "web" -> "grafana"; "metrics-api" -> "prometheus"; "tap" -> "proxy"; "proxy" -> "destination"; "proxy" -> "identity"; "identity" -> "kubernetes api" "destination" -> "kubernetes api"; "grafana" -> "prometheus"; "prometheus" -> "proxy"; } linkerd2_components </details> <!-- markdownlint-enable no-inline-html --> Depending on use case, there are several configurations with which to develop and run Linkerd2: : Integrated configuration using k3d, most closely matches release. : Development of the Linkerd2 Dashboard. This configuration builds all Linkerd2 components in Docker images, and deploys them onto a k3d cluster. This setup most closely parallels our recommended production installation, documented in [Getting Started](https://linkerd.io/2/getting-started/). Note that you need to have first installed docker buildx, as explained . ```bash bin/k3d cluster create bin/docker-build bin/image-load --k3d bin/linkerd install --crds | kubectl apply -f - bin/linkerd install | kubectl apply -f - bin/linkerd viz install | kubectl apply -f - kubectl -n linkerd rollout restart deploy bin/linkerd version bin/linkerd check --expected-version $(bin/root-tag) bin/linkerd viz dashboard curl https://run.linkerd.io/emojivoto.yml | bin/linkerd inject - | kubectl apply -f - kubectl -n emojivoto port-forward svc/web-svc 8080:80 bin/linkerd viz -n emojivoto stat deployments bin/linkerd viz -n emojivoto tap deploy voting ``` Control Plane components have the `trace-collector` flag used to enable for development purposes. It can be enabled globally i.e Control plane components and their proxies by using the `--set controlPlaneTracing=true` installation flag. This will configure all the components to send the traces at

```bash linkerd install --set controlPlaneTracing=true | kubectl apply -f - linkerd jaeger install | kubectl apply -f - kubectl -n linkerd rollout restart deploy ``` The example above builds and loads the docker images into k3d. For testing your built images outside your local environment, you need to publish your images so they become accessible in those external environments. To signal `bin/docker-build` or any of the more specific scripts `bin/docker-build-*` what registry to use, just set the environment variable `DOCKER_REGISTRY` (which defaults to the official registry `cr.l5d.io/linkerd`). After having pushed those images through the usual means (`docker push`) you'll have to pass the `--registry` flag to `linkerd install` with a value matching your registry. Extensions don't have that flag and instead you need to use the equivalent Helm value; e.g. for Viz `linkerd viz install --set defaultRegistry=...`. Our instructions use a script in lieu `go run`. This is a convenience script that leverages caching via `go build` to make your build/run/debug loop faster. In general, replace commands like this: ```bash go run cli/main.go check ``` with this: ```bash bin/go-run cli check ``` That is equivalent to running `linkerd check` using the code on your branch. To analyze and lint the Go code using golangci-lint, run: ```bash golangci-lint run ``` All Go source code is formatted with `goimports`. The version of `goimports` used by this project is specified in `go.mod`. To ensure you have the same version installed, run `go install -mod=readonly golang.org/x/tools/cmd/goimports`. It's recommended that you set your IDE or other development tools to use `goimports`. Formatting is checked during CI by the `bin/fmt` script. The script for building the CLI binaries using docker is `bin/docker-build-cli-bin`. This will also be called indirectly when calling `bin/docker-build`. By default it creates binaries for your current host's OS/arch. To cross-build targeting a different OS or architecture, set the environment variable `DOCKER_TARGET` according to any of the final stages available in . For local development and a faster edit-build-test cycle you can build directly without going through a docker container by calling `bin/build-cli-bin`. If you set the environment variable `LINKERDLOCALBUILD_CLI=1` then `bin/docker-build` will use this last method for the step that builds the CLI. Linkerd2's control plane is composed of several Go microservices. You can run these components in a Kubernetes cluster, or even locally. To run an individual component locally, you can use the `go-run` command, and pass in valid Kubernetes credentials via the `-kubeconfig` flag. For instance, to run the destination service locally, run: ```bash bin/go-run controller/cmd destination -kubeconfig ~/.kube/config -log-level debug ``` You can send test requests to the destination service using the `destination-client` in the `controller/script` directory. For instance: ```bash bin/go-run controller/script/destination-client -path hello.default.svc.cluster.local:80 ``` The Tap APIService is a Kubernetes extension API server, so it can be challenging to run outside the cluster. The most straightforward workflow is to simply test changes by building and loading the container image as explained in the section above (in order to just build this component use `bin/docker-build-tap`). The for the CLI tool is partially generated from YAML. This can be generated by running the `linkerd doc` command. This is a React app fronting a Go process. It uses webpack to bundle assets, and postcss to transform css. These commands assume working and environments. Install and use it to install JS dependencies: ```bash brew install yarn bin/web setup ``` Install Linkerd on a Kubernetes cluster. ```bash bin/web run ``` The web server will be running on `localhost:7777`. To develop with a webpack dev server: Start the development server. ```bash bin/web dev ``` Note: this will start up: `web` on :7777. This is the golang process that serves the dashboard. `webpack-dev-server` on :8080 to manage rebuilding/reloading of the

`metrics-api` is port-forwarded from the Kubernetes cluster via `kubectl` on :8085 Go to to see everything running. To add a JS dependency: ```bash cd web/app yarn add [dep] ``` To add a locale: ```bash cd web/app yarn lingui add-locale [locales...] # will create a messages.json file for new locale(s) ``` To extract message keys from existing components: ```bash cd web/app yarn lingui extract ... yarn lingui compile # done automatically in bin/web run ``` Finally, make sure the new locale is also referred in the following places: Under the `lingui` section in `package.json` In the `make-plural/plurals` import in `index.js` In the `langOptions` object in `index.js` All Rust development happens in the repo. The `bin/docker-build-proxy` script builds the proxy by pulling a pre-published proxy binary: ```bash bin/docker-build-proxy ``` If you want to deploy a locally built proxy, you can build it in the repo by running: ```bash DOCKER_TAG=cr.l5d.io/linkerd/proxy:dev make docker ``` Then, in this repo, run: ```bash ./bin/k3d image import cr.l5d.io/linkerd/proxy:dev ``` Now, to make a pod use your image, add the following annotations to it: ```yaml config.linkerd.io/proxy-version: dev ``` If you make Protobuf changes, run: ```bash bin/protoc-go.sh ``` The is generated by , which depends on [K8s code-generator](https://github.com/kubernetes/code-generator), which does not yet support Go Modules. To re-generate this code, check out this repo into your `GOPATH`: ```bash go get -u github.com/linkerd/linkerd2 cd $GOPATH/src/github.com/linkerd/linkerd2 bin/update-codegen.sh ``` The Linkerd control plane chart is located in the folder. The chart consists of the Linkerd proxy specification, which is used by the proxy injector to inject the proxy container. Both charts depend on the partials subchart which can be found in the folder. Note that the `charts/linkerd2/values.yaml` file contains a placeholder `linkerdVersionValue` that you need to replace with an appropriate string (like `edge-20.2.2`) before proceeding. During development, please use the wrapper script to invoke the Helm commands. For example, ```bash bin/helm install linkerd2 charts/linkerd2 ``` This ensures that you use the same Helm version as that of the Linkerd CI system. For general instructions on how to install the charts check out the . You also need to supply or generate your own certificates to use the chart, as explained . Extensions provide each their own chart: Viz: Multicluster: Jaeger: Whenever you make changes to the files under or its dependency , make sure to run which will refresh the dependencies and lint the templates. Whenever a new chart is created or updated a README should be generated from the chart's `values.yaml`. This can be done by utilizing the bundled binary. For adding additional information, such as specific installation instructions a README template is required to be created. Check existing charts for examples. Example usage: ```sh bin/helm-docs bin/helm-docs --dry-run #Prints to cli instead bin/helm-docs --chart-search-root=./charts #Sets search root for charts bin/helm-docs --template-files=README.md.gotmpl #Sets the template file used ``` Note: The tool searches through the current directory and sub-directories by default. For additional information checkout their repo above. To allow helm-docs to properly document the values in `values.yaml` a descriptive comment is required. This can be done in two ways. Either comment the value directly above with `# -- This is a really nice value` where the double dashes automatically annotates the value. Another explicit usage is to type out the value name. `# global.MyNiceValue -- I really like this value` In order to accommodate for extra data that might not have a proper place in the values.yaml file the corresponding README.md.gotmpl can be modified for each chart. This template allows the standard markdown syntax as well as the go templating functions. Checkout for more info.