Project: Litmus (Observability and Analysis / Chaos Engineering)
File: ebs-loss-by-tag.md

It causes chaos to disrupt state of ebs volume by detaching it from the node/ec2 instance for a certain chaos duration using volume tags. In case of EBS persistent volumes, the volumes can get self-attached and experiment skips the re-attachment step. Tests deployment sanity (replica availability & uninterrupted service) and recovery workflows of the application pod. !!! tip "Scenario: Detach EBS Volume" ??? info "View the uses of the experiment" coming soon ??? info "Verify the prerequisites" Ensure that Kubernetes Version > 1.16 Ensure that the Litmus Chaos Operator is running by executing <code>kubectl get pods</code> in operator namespace (typically, <code>litmus</code>).If not, install from <a href="https://v1-docs.litmuschaos.io/docs/getstarted/#install-litmus">here</a> Ensure that the <code>ebs-loss-by-tag</code> experiment resource is available in the cluster by executing <code>kubectl get chaosexperiments</code> in the desired namespace. If not, install from <a href="https://hub.litmuschaos.io/api/chaos/master?file=faults/aws/ebs-loss-by-tag/fault.yaml">here</a> Ensure that you have sufficient AWS access to attach or detach an ebs volume for the instance. Ensure to create a Kubernetes secret having the AWS access configuration(key) in the `CHAOS_NAMESPACE`. A sample secret file looks like: ```yaml apiVersion: v1 kind: Secret metadata: name: cloud-secret type: Opaque stringData: cloud_config.yml: |- [default] awsaccesskey_id = XXXXXXXXXXXXXXXXXXX awssecretaccess_key = XXXXXXXXXXXXXXX ``` If you change the secret key name (from `cloudconfig.yml`) please also update the `AWSSHAREDCREDENTIALSFILE` ENV value on `experiment.yaml`with the same name. ??? info "View the default validations" EBS volume is attached to the instance. !!! tip "NOTE" If you are using this experiment as part of a litmus workflow scheduled constructed & executed from chaos-center, then you may be making use of the RBAC, which is pre installed in the cluster as part of the agent setup. ??? note "View the Minimal RBAC permissions" ```yaml apiVersion: v1 kind: ServiceAccount metadata: name: ebs-loss-by-tag-sa namespace: default labels: name: ebs-loss-by-tag-sa app.kubernetes.io/part-of: litmus apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: ebs-loss-by-tag-sa labels: name: ebs-loss-by-tag-sa app.kubernetes.io/part-of: litmus rules: apiGroups: [""] resources: ["pods"] verbs: ["create","delete","get","list","patch","update", "deletecollection"] apiGroups: [""] resources: ["events"] verbs: ["create","get","list","patch","update"] apiGroups: [""] resources: ["secrets","configmaps"] verbs: ["get","list",] apiGroups: [""] resources: ["pods/log"] verbs: ["get","list","watch"] apiGroups: [""] resources: ["pods/exec"] verbs: ["get","list","create"] apiGroups: ["batch"] resources: ["jobs"] verbs: ["create","list","get","delete","deletecollection"] apiGroups: ["litmuschaos.io"] resources: ["chaosengines","chaosexperiments","chaosresults"] verbs: ["create","list","get","patch","update","delete"] apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: ebs-loss-by-tag-sa labels: name: ebs-loss-by-tag-sa app.kubernetes.io/part-of: litmus roleRef: apiGroup:

kind: ClusterRole name: ebs-loss-by-tag-sa subjects: kind: ServiceAccount name: ebs-loss-by-tag-sa namespace: default ``` Use this sample RBAC manifest to create a chaosServiceAccount in the desired (app) namespace. This example consists of the minimum necessary role permissions to execute the experiment. ??? info "check the experiment tunables" <h2>Mandatory Fields</h2> <table> <tr> <th> Variables </th> <th> Description </th> <th> Notes </th> </tr> <tr> <td> EBSVOLUMETAG </td> <td> Provide the common tag for target volumes. It'll be in form of <code>key:value</code> (Ex: 'team:devops')</td> <td> </td> </tr> <tr> <td> REGION </td> <td> The region name for the target volumes</td> <td> </td> </tr> </table> <h2>Optional Fields</h2> <table> <tr> <th> Variables </th> <th> Description </th> <th> Notes </th> </tr> <tr> <td> VOLUMEAFFECTEDPERC </td> <td> The Percentage of total ebs volumes to target </td> <td> Defaults to 0 (corresponds to 1 volume), provide numeric value only </td> </tr> <tr> <td> TOTALCHAOSDURATION </td> <td> The time duration for chaos insertion (sec) </td> <td> Defaults to 30s </td> </tr> <tr> <td> CHAOS_INTERVAL </td> <td> The time duration between the attachment and detachment of the volumes (sec) </td> <td> Defaults to 30s </td> </tr> <tr> <td> SEQUENCE </td> <td> It defines sequence of chaos execution for multiple volumes</td> <td> Default value: parallel. Supported: serial, parallel </td> </tr> <tr> <td> RAMP_TIME </td> <td> Period to wait before and after injection of chaos in sec </td> <td> </td> </tr> </table> Refer the and to tune the common tunables for all experiments and aws specific tunables. It will detach a random single ebs volume with the given `EBSVOLUMETAG` tag and `REGION` region. Use the following example to tune this: ```yaml apiVersion: litmuschaos.io/v1alpha1 kind: ChaosEngine metadata: name: engine-nginx spec: engineState: "active" annotationCheck: "false" chaosServiceAccount: ebs-loss-by-tag-sa experiments: name: ebs-loss-by-tag spec: components: env: name: EBSVOLUMETAG value: 'key:value' name: REGION value: '<region for EBSVOLUMETAG>' name: TOTALCHAOSDURATION value: '60' ``` It will detach the `VOLUMEAFFECTEDPERC` percentage of ebs volumes with the given `EBSVOLUMETAG` tag and `REGION` region. Use the following example to tune this: ```yaml apiVersion: litmuschaos.io/v1alpha1 kind: ChaosEngine metadata: name: engine-nginx spec: engineState: "active" annotationCheck: "false" chaosServiceAccount: ebs-loss-by-tag-sa experiments: name: ebs-loss-by-tag spec: components: env: name: VOLUMEAFFECTEDPERC value: '100' name: EBSVOLUMETAG value: 'key:value' name: REGION value: '<region for EBSVOLUMETAG>' name: TOTALCHAOSDURATION value: '60' ```