Package: runwhen-cli-keywords
Troubleshooting Tasks in Codecollection:137Codebundles in Codecollection:50RunWhen Public CodecollectionThis repository is a codecollection that is to be used within the RunWhen platform. It contains codebundles that can be used in SLIs, and TaskSets.Please see thecontributingandcode of conductfor details on adding your contributions to this project.Documentation for each codebundle is maintained in the README.md alongside the robot code and is published athttps://docs.runwhen.com/public/v/codebundles/. Please see thereadme howtofor details on crafting a codebundle readme that can be indexed.Getting StartedHead on over to our centralized documentationherefor detailed information on getting started.File Structure overview of devcontainer:-/app/
    |- auth/ # store secrets here, it should already be properly gitignored for you
    |- codecollection/
    |   |- codebundles/ # stores codebundles that can be run during development
    |   |- libraries/ # stores python keyword libraries used by codebundles
    |- dev_facade/ # provides interfaces equivalent to those used on the platform, but just dry runs the keywords to assist with development
    ...The included scriptrowraps therobotRobotFramework binary, and includes some extra functionality to write files to a consistent location for viewing in a HTTP server athttp://localhost:3000/that is always running as part of the devcontainer.QuickstartNavigate to the codebundle directorycd codecollection/codebundles/curl-http-ok/Run the codebundlero runbook.robotCodebundle IndexNameSupported IntegrationsTasksDocumentationAWS CloudWatch Overutlized EC2 InspectionAWS,CloudWatchCheck For Overutilized Ec2 InstancesQueries AWS CloudWatch for a list of EC2 instances with a high amount of resource utilization, raising issues when overutilized instances are found.DocsAWS EKS Nodegroup Status CheckAWS,EKSCheck EKS Nodegroup StatusQueries a node group within a EKS cluster to check if the nodegroup has degraded service, indicating ongoing reboots or other issues.DocsAzure Internal LoadBalancer TriageKubernetes,AKS,AzureCheck Activity Logs for Azure Load Balancer${AZ_LB_NAME}``Triages issues related to a Azure Loadbalancers and its activity logs.DocsAzure Monitor Activity Log SLIKubernetes,AKS,AzureRun Azure Monitor Activity Log TriageMeasures the count of error activity log entries as a SLI metric for the Azure tenancy.DocsAzure Monitor Event TriageKubernetes,AKS,AzureRun Azure Monitor Activity Log TriageTriages issues related to a Azure Loadbalancers, Kubernetes ingress objects and services.DocsGCP Cloud Function HealthGCPCount unhealthy GCP Cloud Functions in GCP Project${GCP_PROJECT_ID}``Count the number of Cloud Functions in an unhealthy state for a GCP Project.DocsGCP Gcloud Log InspectionGCP,Gcloud,Google MonitoringInspect GCP Logs For Common ErrorsFetches logs from a GCP using a configurable query and raises an issue with details on the most common issues.DocsGCP Node Prempt ListGCP,GKECount the number of nodes in active prempt operationCounts nodes that have been preempted within the defined time interval.DocsGKE Kong Ingress Host TriageGCP,GMP,Ingress,Kong,MetricsCheck If Kong Ingress HTTP Error Rate Violates HTTP Error Threshold,Check If Kong Ingress HTTP Request Latency Violates Threshold,Check If Kong Ingress Controller Reports Upstream ErrorsCollects Kong ingress host metrics from GMP on GCP and inspects the results for ingress with a HTTP error code rate greater than zero over a configurable duration and raises issues based on the number of ingress with error codes.DocsGKE Nginx Ingress Host TriageGCP,GMP,Ingress,Nginx,MetricsFetch Nginx HTTP Errors From GMP for Ingress${INGRESS_OBJECT_NAME}, `Find Owner and Service Health for Ingress `${INGRESS_OBJECT_NAME}Collects Nginx ingress host controller metrics from GMP on GCP and inspects the results for ingress with a HTTP error code rate greater than zero over a configurable duration and raises issues based on the number of ingress with error codes.DocsKubeprometheus Operator TroubleshootKubernetes,AKS,EKS,GKE,OpenShift,PrometheusCheck Prometheus Service Monitors,Check For Successful Rule Setup,Verify Prometheus RBAC Can Access ServiceMonitors,Identify Endpoint Scraping Errors,Check Prometheus API HealthyThis taskset investigates the logs, state and health of Kubernetes Prometheus operator.DocsKubernetes Application MonitorKubernetes,AKS,EKS,GKE,OpenShiftMeasure Application ExceptionsMeasures the number of exception stacktraces present in an application's logs over a time period.DocsKubernetes Application TroubleshootKubernetes,AKS,EKS,GKE,OpenShiftGet${WORKLOAD_NAME}Application Logs,Scan${WORKLOAD_NAME}For Misconfigured Environment,Troubleshoot${WORKLOAD_NAME}Application LogsPerforms application-level troubleshooting by inspecting the logs of a workload for parsable exceptions, and attempts to determine next steps.DocsKubernetes ArgoCD Application Health & TroubleshootKubernetes,AKS,EKS,GKE,OpenShift,ArgoCDFetch ArgoCD Application Sync Status & Health,Fetch ArgoCD Application Last Sync Operation Details,Fetch Unhealthy ArgoCD Application Resources,Scan For Errors in Pod Logs Related to ArgoCD Application Deployments,Fully Describe ArgoCD ApplicationThis taskset collects information and runs general troubleshooting checks against argocd application objects within a namespace.DocsKubernetes ArgoCD HelmRelease TaskSetKubernetes,AKS,EKS,GKE,OpenShift,ArgoCDFetch all available ArgoCD Helm releases,Fetch Installed ArgoCD Helm release versionsThis codebundle runs a series of tasks to identify potential helm release issues related to ArgoCD managed Helm objects.DocsKubernetes Artifactory TriageKubernetes,AKS,EKS,GKE,OpenShift,ArtifactoryCheck Artifactory Liveness and Readiness EndpointsPerforms a triage on the Open Source version of Artifactory in a Kubernetes cluster.DocsKubernetes CertManager HealthcheckKubernetes,AKS,EKS,GKE,OpenShiftGet Health Score of CertManager WorkloadsCheck the health of pods deployed by cert-manager.DocsKubernetes Daemonset TriageKubernetes,AKS,EKS,GKE,OpenShiftGet DaemonSet Log Details For Report,Get Related Daemonset Events,Check Daemonset ReplicasTriages issues related to a Daemonset and its available replicas.DocsKubernetes Deployment TriageKubernetes,AKS,EKS,GKE,OpenShiftCheck Deployment Log For Issues with${DEPLOYMENT_NAME}, `Check Liveness Probe Configuration for Deployment `${DEPLOYMENT_NAME},Check Readiness Probe Configuration for Deployment${DEPLOYMENT_NAME}, `Troubleshoot Deployment Warning Events for `${DEPLOYMENT_NAME},Get Deployment Workload Details For${DEPLOYMENT_NAME}and Add to Report,Troubleshoot Deployment Replicas for${DEPLOYMENT_NAME}, `Check Deployment Event Anomalies for `${DEPLOYMENT_NAME}Triages issues related to a deployment and its replicas.DocsKubernetes Flux Choas TestingKubernetes,AKS,EKS,GKE,OpenShiftSuspend the Flux Resource Reconciliation,Find Random FluxCD Workload as Chaos Target,Execute Chaos Command,Execute Additional Chaos Command,Resume Flux Resource ReconciliationThis taskset is used to suspend a flux resource for the purposes of executing chaos tasks.DocsKubernetes FluxCD HelmRelease TaskSetKubernetes,AKS,EKS,GKE,OpenShift,FluxCDList all available FluxCD Helmreleases in Namespace${NAMESPACE}, `Fetch Installed FluxCD Helmrelease Versions in Namespace `${NAMESPACE},Fetch Mismatched FluxCD HelmRelease Version in Namespace${NAMESPACE}, `Fetch FluxCD HelmRelease Error Messages in Namespace `${NAMESPACE},Check for Available Helm Chart Updates in Namespace${NAMESPACE}``This codebundle runs a series of tasks to identify potential helm release issues related to Flux managed Helm objects.DocsKubernetes FluxCD Kustomization TaskSetKubernetes,AKS,EKS,GKE,OpenShift,FluxCDList all available Kustomization objects in Namespace${NAMESPACE}, `Get details for unready Kustomizations in Namespace `${NAMESPACE}This codebundle runs a series of tasks to identify potential Kustomization issues related to Flux managed Kustomization objects.DocsKubernetes GitOps GitHub RemediationKubernetes,AKS,EKS,GKE,OpenShift,FluxCD,ArgoCDRemediate Readiness and Liveness Probe GitOps Manifests Namespace${NAMESPACE}``Provides a list of tasks that can remediate configuraiton issues with deployment manifests in GitHub based gitops repositories.DocsKubernetes Grafana Loki Health Checkk8sCheck Loki Ring API,Check Loki API ReadyThis taskset checks the health of Grafana Loki and its hash ring.DocsKubernetes Image CheckKubernetes,AKS,EKS,GKE,OpenShiftCheck Image Rollover Times for Namespace${NAMESPACE}, `List Images and Tags for Every Container in Running Pods for Namespace `${NAMESPACE},List Images and Tags for Every Container in Failed Pods for Namespace${NAMESPACE}, `List ImagePullBackOff Events and Test Path and Tags for Namespace `${NAMESPACE}This taskset provides detailed information about the images used in a Kubernetes namespace.DocsKubernetes Ingress GCE & GCP HTTP Load Balancer HealthcheckKubernetes,GKE,GCE,GCPSearch For GCE Ingress Warnings in GKE,Identify Unhealthy GCE HTTP Ingress Backends,Validate GCP HTTP Load Balancer Configurations,Fetch Network Error Logs from GCP Operations Manager for Ingress Backends,Review GCP Operations Logging DashboardTroubleshoot GCE Ingress Resources related to GCP HTTP Load Balancer in GKEDocsKubernetes Ingress HealthcheckKubernetes,AKS,EKS,GKE,OpenShiftFetch Ingress Object Health in Namespace${NAMESPACE}, `Check for Ingress and Service Conflicts in Namespace `${NAMESPACE}Triages issues related to a ingress objects and services.DocsKubernetes Jenkins HealthcheckKubernetes,AKS,EKS,GKE,OpenShift,JenkinsQuery The Jenkins Kubernetes Workload HTTP Endpoint,Query For Stuck Jenkins JobsThis taskset collects information about perstistent volumes and persistent volume claims to validate health or help troubleshoot potential issues.DocsKubernetes Labeled Pod CountKubernetes,AKS,EKS,GKE,OpenShiftMeasure Number of Running Pods with LabelThis codebundle fetches the number of running pods with the set of provided labels, letting you measure the number of running pods.DocsKubernetes Namespace HealthcheckKubernetes,AKS,EKS,GKE,OpenShiftGet Event Count and Score,Get Container Restarts and Score,Get NotReady Pods,Generate Namspace ScoreThis SLI uses kubectl to score namespace health. Produces a value between 0 (completely failing thet test) and 1 (fully passing the test). Looks for container restarts, events, and pods not ready.DocsKubernetes Namespace TroubleshootKubernetes,AKS,EKS,GKE,OpenShiftTroubleshoot Warning Events in Namespace${NAMESPACE}, `Troubleshoot Container Restarts In Namespace `${NAMESPACE},Troubleshoot Pending Pods In Namespace${NAMESPACE}, `Troubleshoot Failed Pods In Namespace `${NAMESPACE},Troubleshoot Workload Status Conditions In Namespace${NAMESPACE}, `Get Listing Of Resources In Namespace `${NAMESPACE},Check Event Anomalies in Namespace${NAMESPACE}, `Check Missing or Risky PodDisruptionBudget Policies in Namepace `${NAMESPACE}This taskset runs general troubleshooting checks against all applicable objects in a namespace. Looks for warning events, odd or frequent normal events, restarting containers and failed or pending pods.DocsKubernetes Persistent Volume HealthcheckKubernetes,AKS,EKS,GKE,OpenShiftFetch Events for Unhealthy Kubernetes PersistentVolumeClaims in Namespace${NAMESPACE}, `List PersistentVolumeClaims in Terminating State in Namespace `${NAMESPACE},List PersistentVolumes in Terminating State in Namespace${NAMESPACE}, `List Pods with Attached Volumes and Related PersistentVolume Details in Namespace `${NAMESPACE},Fetch the Storage Utilization for PVC Mounts in Namespace${NAMESPACE}, `Check for RWO Persistent Volume Node Attachment Issues in Namespace `${NAMESPACE}This taskset collects information about storage such as PersistentVolumes and PersistentVolumeClaims to validate health or help troubleshoot potential storage issues.DocsKubernetes Pod Resources ScanKubernetes,AKS,EKS,GKE,OpenShiftShow Pods Without Resource Limit or Resource Requests Set in Namespace${NAMESPACE}, `Get Pod Resource Utilization with Top in Namespace `${NAMESPACE}Inspects the resources provisioned for a given set of pods, selected by their labels and raises issues if no resources were specified.DocsKubernetes Postgres TriageAKS,EKS,GKE,Kubernetes,Patroni,PostgresGet Standard Postgres Resource Information,Describe Postgres Custom Resources,Get Postgres Pod Logs & Events,Get Postgres Pod Resource Utilization,Get Running Postgres Configuration,Get Patroni Output,Run DB QueriesRuns multiple Kubernetes and psql commands to report on the health of a postgres cluster.DocsKubernetes Redis HealthcheckKubernetes,AKS,EKS,GKE,OpenShift,RedisPing${DEPLOYMENT_NAME}Redis Workload,Verify${DEPLOYMENT_NAME}Redis Read Write OperationThis taskset collects information on your redis workload in your Kubernetes cluster and raises issues if any health checks fail.DocsKubernetes Restart resourceKubernetes,AKS,EKS,GKE,OpenShiftGet Current Resource State,Get Resource Logs,Restart ResourceThis taskset restarts a resource with a given set of labels, typically used with other tasksets.DocsKubernetes Service Account CheckKubernetes,AKS,EKS,GKE,OpenShiftTest Service Account Access to Kubernetes API Server in Namespace${NAMESPACE}``This taskset provides tasks to troubleshoot service accounts in a Kubernetes namespace.DocsKubernetes StatefulSet TriageKubernetes,AKS,EKS,GKE,OpenShiftCheck Readiness Probe Configuration for StatefulSet${STATEFULSET_NAME}, `Check Liveness Probe Configuration for StatefulSet `${STATEFULSET_NAME},Troubleshoot StatefulSet Warning Events for${STATEFULSET_NAME}, `Check StatefulSet Event Anomalies for `${STATEFULSET_NAME},Fetch StatefulSet Logs for${STATEFULSET_NAME}, `Get Related StatefulSet `${STATEFULSET_NAME}` Events`, `Fetch Manifest Details for StatefulSet `${STATEFULSET_NAME},List StatefulSets with Unhealthy Replica Counts In Namespace${NAMESPACE}``Triages issues related to a StatefulSet and its replicas.DocsKubernetes Vault TriageAKS,EKS,GKE,Kubernetes,VaultFetch Vault CSI Driver Logs,Get Vault CSI Driver Warning Events,Check Vault CSI Driver Replicas,Fetch Vault Logs,Get Related Vault Events,Fetch Vault StatefulSet Manifest Details,Fetch Vault DaemonSet Manifest Details,Verify Vault Availability,Check Vault StatefulSet ReplicasA suite of tasks that can be used to triage potential issues in your vault namespace.DocsTerraform Cloud Workspace Lock CheckTerraform CloudChecking whether the Terraform Cloud Workspace is in a locked stateCheck whether the Terraform Cloud Workspace is in a locked state.DocsTest IssuesTestRaise Full IssueA codebundle for testing the issues feature. Purely for testing flow.DocscURL HTTP OKLinux macOS Windows HTTPChecking HTTP URL Is Available And TimelyThis taskset uses curl to validate the response code of the endpoint. Returns ascore of 1 if healthy, an 0 if unhealthy.Docscli-test-tasksetcliRun CLI and Parse Output For Issues,Exec Test,Local Process TestThis taskset smoketests the CLI codebundle setup and run processDocscmd-test-tasksetcmdRun CLI Command,Run Bash File,Log SuggestionThis taskset smoketests the CLI codebundle setup and run process by running a bare commandDocsk8s-kubectl-cmd-slik8sRun User Provided Kubectl CommandThis taskset runs a user provided kubectl command and pushes the metric. The supplied command must result in distinct single metric. Command line tools like jq are available.Docsk8s-kubectl-cmd-tasksetk8sRun User Provided Kubectl CommandThis taskset runs a user provided kubectl command andadds the output to the report. Command line tools like jq are available.Docs