Project: Inspektor Gadget (Observability and Analysis / Observability)
File: ci.md

title: Continuous Integration weight: 210 description: > Inspektor Gadget continuous integration workflow This repository uses GitHub Actions to create a custom Continuous Integration (CI) workflow named `Inspektor Gadget CI`. This workflow compiles and uploads the Inspektor Gadget CLI (`kubectl-gadget` and `ig`) and gadget container images and runs their unit and integration tests. In addition, it verifies that the documentation is up-to-date and runs a static code analysis using Golang linters. A fork of this project should enable GitHub Actions in the repo settings page and add the proper to be able to use this workflow. The following sections describes what secrets need to be added to enable all the CI tests. To run the integration tests, it is necessary to have the gadget container image available so that it can be installed/loaded in the Kubernetes cluster where the tests will run. As a default, images are shared via artifacts between workflow jobs for all CI pipeline. When the target branch correspond to the main or the push refers to a tag, the images are also pushed to `ghcr.io/${{ github.repository }}`. This permits a clear separation between "in development" images and production ones. Note that, you need to to allow anonymous pull. Optionally, we can add the secrets described in this section so that the integration tests will also run on a pre-created [Azure Red Hat OpenShift (ARO)](https://docs.microsoft.com/en-us/azure/openshift/intro-openshift) cluster. Consider that the Inspektor Gadget workflow will still success even if no ARO cluster is provided through these secrets. These are the steps to create an ARO cluster using the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/): ```bash $ export SUBSCRIPTION=<mySubscription> $ export RESOURCEGROUP=<myResourceName> $ export LOCATION=<myLocation> $ export CLUSTER=<myCluster> $ export VNET=<myVNET> $ export MASTSUB=<myMASTSUB> $ export WORKSUB=<myWORKSUB> $ az account set --subscription $SUBSCRIPTION $ az provider register -n Microsoft.RedHatOpenShift --wait $ az provider register -n Microsoft.Compute --wait $ az provider register -n Microsoft.Storage --wait $ az provider register -n Microsoft.Authorization --wait $ az group create --name $RESOURCEGROUP --location $LOCATION $ az network vnet create --resource-group $RESOURCEGROUP --name $VNET --address-prefixes 10.0.0.0/22 $ az network vnet subnet create --resource-group $RESOURCEGROUP --vnet-name $VNET --name $MASTSUB --address-prefixes 10.0.0.0/23 --service-endpoints Microsoft.ContainerRegistry $ az network vnet subnet create --resource-group $RESOURCEGROUP --vnet-name $VNET --name $WORKSUB --address-prefixes 10.0.2.0/23 --service-endpoints Microsoft.ContainerRegistry $ az network vnet subnet update --name $MASTSUB --resource-group $RESOURCEGROUP --vnet-name $VNET --disable-private-link-service-network-policies true $ az aro create --resource-group $RESOURCEGROUP --name $CLUSTER --vnet $VNET --master-subnet $MASTSUB --worker-count 3 --worker-subnet $WORKSUB ``` Considerations: After executing the `az aro create` command, it normally takes about 35 minutes to create a cluster. For the sake of simplicity, we are not providing a [Red Hat pull secret](https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#get-a-red-hat-pull-secret-optional) during the cluster creation, so our cluster will not include samples or operators from Red Hat or certified partners. However, it is not a requirement to run the Inspektor Gadget integration tests on

Creating an ARO cluster requires specific permissions, check [the documentation](https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#verify-your-permissions) to be sure you have them. If we need to delete our cluster, it is enough to execute: ```bash $ az group delete --name $RESOURCEGROUP ``` Take into account that it will remove the entire resource group and all resources inside it. Further details about creating an ARO cluster can be found in the [Azure Red Hat OpenShift documentation](https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster). Fist of all, to be able to connect to our cluster, we need the following information: ```bash $ az aro show --subscription $SUBSCRIPTION -g $RESOURCEGROUP -n $CLUSTER --query apiserverProfile.url https://api.server.example.io:1234 $ az aro list-credentials --subscription $SUBSCRIPTION -g $RESOURCEGROUP -n $CLUSTER { "kubeadminPassword": "myPassword", "kubeadminUsername": "myUsername" } ``` The `test-integration` job is already configured to authenticate and set the kubeconf context to the ARO cluster configured in the GitHub repository. So all we need to do is to add the following actions secrets: `OPENSHIFT_SERVER`: The API server URL: `https://api.server.example.io:1234`. `OPENSHIFT_USER`: The `kubeadminUsername` from the JSON output of the `list-credentials` command. `OPENSHIFT_PASSWORD`: The `kubeadminPassword` from the JSON output of the `list-credentials` command. Further details about connect to an ARO cluster from GitHub actions can be found in the [Azure Red Hat OpenShift documentation](https://docs.microsoft.com/en-us/azure/openshift/tutorial-connect-cluster#connect-using-the-openshift-cli) and the [redhat-actions/oc-login documentation](https://github.com/redhat-actions/oc-login). For debugging, it might be necessary to connect to the cluster from a host. We can do it by using the `oc` tool: ```bash $ oc login $apiServer -u $kubeadminUsername -p $kubeadminPassword ``` Notice that it configures the kubectl configuration with a new context. Please take into account that any change done on this cluster could cause issues with the integration tests running on GitHub actions at that moment. When secrets described in the below sections are set, the integration tests will also run on . As Inspektor Gadget support both `amd64` and `arm64`, two clusters will be created when the CI is triggered. To be able to create them in the CI, you need to follow : ```bash $ subscription_id=<mySubscriptionID> $ resourcegroup=<myResourceName> $ location=<myLocation> $ app_name=<myAppName> $ federated_name=<myFederatedCredentialName> $ organization=<myGitHubOrganization> $ repository=<myGitHubRepository> $ environment=<myCIJobEnvironment> $ az account set --subscription $subscription_id $ az group create --name $resourcegroup --location $location $ az ad app create --display-name $app_name { } $ appid=$(az ad app list --display-name $appname --query [0].id | tr -d '"') $ az ad sp create --id $app_id { } $ spid=$(az ad sp list --display-name $appname --query [0].id | tr -d '"') $ az role assignment create --role contributor --assignee-object-id $spid --assignee-principal-type ServicePrincipal --scope /subscriptions/$subscriptionid/resourceGroups/$resourcegroup { } $ az ad app federated-credential create --id $app_id --parameters <(echo "{ \"name\": \"${federated_name}\", \"issuer\":

\"subject\": \"repo:${organization}/${repository}:environment:${environment}\", \"description\": \"AKS federated credentials for CI\", \"audiences\": [ \"api://AzureADTokenExchange\" ] }") { } ``` After doing this, you will need to create several secrets: `AZUREAKSCLIENTID`: The application ID as given by `az ad app list --display-name $appname --query [0].appId`. `AZUREAKSTENANT_ID`: The tenant ID as given by `az account show --query tenantId`. `AZUREAKSSUBSCRIPTION_ID`: The subscription used to create the federated credentials as given by `az account show --query id`. `AZUREAKSRESOURCE_GROUP`: It stores the name of the resource group where the clusters will be created. The workflow will create the `amd64` and `arm64` clusters for you by using the above information. By default, each of this cluster features . Once created, the integration tests will be run on these clusters. Finally, the clusters are deleted, whatsoever is the result of the tests. We use OpenID Connect to authenticate GitHub Actions against AWS. If you want to run the integration tests on your fork, follow to configure your AWS account. You need to create a role with the [minimum IAM policies](https://eksctl.io/usage/minimum-iam-policies/) needed by eksctl. Save the name of this role (including the rull `arn:aws..` prefix). in a [repository variable](https://docs.github.com/en/actions/learn-github-actions/variables#creating-configuration-variables-for-a-repository) named `AWS_ROLE`. In order to run the integration tests on GKE on a fork, you need to set the following secrets: `GKE_PROJECT`: The GCP project where the GKE cluster is located. `GKESERVICEACCOUNT`: The service account used to authenticate to GCP. `GKEWORKLOADIDENTITY_PROVIDER`: The workload identity provider used to authenticate to GCP. these secrets use OpenID Connect to authenticate GitHub Actions against GCP for more information on how it works you can check the GitHub documentation . The steps to create a service account and a workload identity provider can be found in the repository till step 6. In terms of permissions, the service account needs to have the following roles: `roles/container.admin` `roles/container.clusterAdmin` `roles/iam.serviceAccountUser` You can see the guide to see how these roles can be assigned to the service account. Inspektor Gadget has that are automatically executed and published by . You can see the results on: https://inspektor-gadget.github.io/ig-benchmarks/dev/bench/index.html This requires a GitHub API token (secret `BENCHMARKS_TOKEN`) configured with read and write access to two repositories: : required to allow the bot to post comments , see the `gh-pages` branch. The GitHub Action is disabled for pushes on forks or PR from forks, so the result page will not be updated by forks. In this way, forks can still use other parts of the CI without failing, even without the `BENCHMARKS_TOKEN` secret. We compute hash sum of all our release artifacts and the file containing these checksums is signed using . To sign this file, you will need to create a private key with an associated password: ```bash $ cosign generate-key-pair Enter password for private key: Enter password for private key again: Private key written to cosign.key Public key written to cosign.pub ``` You will then need to store the content of `cosign.key` in the `COSIGNPRIVATEKEY` and the password you used to create the key in `COSIGN_PASSWORD`. Without these secrets, the release job will not be run.