Package: chaosimp
Chaos ImpChaos Imp is a framework for creating, executing, and runningchaos engineering(CE) experiments on AWS. It provides shorthand syntax to express experiment templates, executions, and automations. With just a few lines, you can define the experiment you want and model it using YAML and shell scripts. During deployment, Chaos Imp transforms and expands your YAML and shell scripts into AWS CloudFormation syntax, enabling you to run chaos experiments faster.Chaos Imp uses a plethora of AWS services under the hood. It glues Systems Manager Agent (SSM), Failure Injection Simulator (FIS), Events, and Lambda APIs to create an easy-to-use tool around the following components of the CE process:Defining infrastructure, application, and security failure injection templates.Running CE experiments in a controlled way by using AWS access capabilities.Automating experiments to be run continuously.What benefits does Chaos Imp bring to organizations when compared to SSM/FIS/Lambda?Experiment scripts are decoupled from YAML, which means that they are much more easily editable and can be re-used across multiple experiments.Templates and automations are automatically managed via CloudFormation templates, which makes it easy to control and cleanup.CLI API is very minimalist. It has three namespaces for creatingtemplates, runningexperiments, and setting upautomations. No need to worry about gluing different services together and resolving IAM shenanigans.Chaos Imp uses unified config file syntax. Think of it asAWS SAMfor chaos engineering.InstallationChaos Imp is a Python package. To install it run:pipinstallchaosimpNow, you can start using scripts and classes from thechaosimppackage. You can also run CLI commands. Chaos Imp supports four namespaces:config:get,list, andsetoperations.templates:list,show,create,update, anddeleteoperations.experiments:get,get-by-id,list,start, andstopoperations.automations:list,show,create,update, anddeleteoperations.For example, to list all of your templates run:imptemplateslistThe CLI is self-documenting, so you can learn about any command by running:imp<COMMAND_NAME>--helpCreating Experiment TemplatesCheck outChaos Imp example templatesthat include resource, network, and state chaos experiments.Let's create a simple experiment that stresses CPUs of several EC2 instances.Experiment BoundariesYou can perform experiments on a variety of different AWS resources. Chaos Imp automatically translates resources defined in the YAML experiment template toAWS FIS targets.For example, to target a subset of EC2 instances tagged withimp: ec2-experimentdefine the following target in `imp.yml:Targets:-Name:"ec2-instances"ResourceType:"aws:ec2:instance"ResourceTags:-Key:"imp"Value:"ec2-experiment"SelectionMode:"ALL"This defines a FIS target that experiment actions can be applied to.ActionsNow, let's define a custom Chaos Imp action that runs a script withstress-ngstressing CPUs:Actions:-Name:"stress-cpus"Type:"imp:run-script"Target:"ec2-instances"Parameters:Duration:"PT1M"Document:Path:"stress-cpu.sh"This defines a Chaos Imp action that is later translated into a FIS action. To avoid confusion, you can use all FIS action types defined in theofficial documentation.Chaos Imp introduces its own namespace and action type into the mix:imp:run-script. This action functions just likeaws:ssm:send-commandexcept for you can reference a local file instead ofdocumentArnanddocumentVersion.Now, we just add an experiment script filestress-cpu.sh:#!/bin/bashsudoyum-yinstallstress-ng
stress-ng--cpu0--cpu-methodmatrixprod--cpu-load100-t20sThis will installstress-ngand apply 100% load on all CPUs for 20 seconds.Creating a TemplateBefore creating a template, you have to create a role with apolicythat allows FIS to run actions.You can reference this role with every template creation call by using--role-arnbut it's much more convenient to store it in the local config:impconfigsetTemplateRoleArn<ROLE_ARN>We are finally ready to create our first template:imptemplatescreate--path.cpu-stressRunning ExperimentsBefore running an experiment on EC2 instances those instances have to be assigned a role witha policythat allows them to interact with SSM. This is required for all FIS SSM actions as well as Chaos Imp special actions.Once instances are ready, you can run an experiment based on the template we created:impexperimentsstart--templatecpu-stressmy-cpu-experimentThis will create and run an experiment in FIS. If you run subsequent experiments with the same name you can list all experiment executions by running:impexperimentsgetmy-cpu-experimentIf you are interested in the specific instance of an experiment then run:impexperimentsget-by-id<EXPERIMENT_ID>Automating ExperimentsExperiment automation is a work in progress. Chaos Imp uses a combination of CloudWatch Events and Lambda Functions to create automations.Unfortunately, AWS SDK is out of date in the Lambda runtime and doesn't support FIS yet, so you'll have to create a Docker image with an updated AWS SDK in it.First,downloadDockerfileandapp.pyon your machine. Then run the following commands to create and push an image to your private AWS ECR:awsecrget-login-password|dockerlogin--usernameAWS--password-stdin<AWS_ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com
dockerbuild-timp-automation.
dockertagimp-automation:latest<REPO_URL>/imp-automation:latest
dockerpush<REPO_URL>/imp-automation:latestThis will become unnecessary once Lambda supports a more recent SDK.To create an automation run:impautomationscreate\--schedule="rate(30 minutes)"\--template="cpu-stress"\--image=<AWS_ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com/imp-automation:latest\cpu-stress-automationThis will create a CloudWatch Event Rule that will kickoff a Lambda every 30 minutes. The Lambda starts a FIS experiment.