Package: critter
critter- AWSConfigRuleIntegrationTesTERcritterenables AWS Config rule integration testing. Use it to validate that AWS Config rules evaluate resources as expected.Customers useAWS Config rulesto evaluate their AWS resources against their own unique compliance and governance requirements.critteris a command line tool that enables a continuous integration workflow to validate that Config rules evaluate resources as expected. This is essential to guaranteeing compliance within AWS accounts, especially when you consider the potential impact of unexpected behavior from Config rule auto-remediations.This project is in MVP phase - it is functional but the api may change significantly before release1.0.OverviewUsage ofcritterwithin the Config rule development workflow looks like this:Developer writesConfig custom rule evaluation Lambda function codethat evaluates AWS resources.Example Config custom rules Python codecrittercan also testConfigmanagedrulesDeveloper deploys their Config rule and Lambda function.CloudFormation,the RDK (Rule Development Kit), orTerraformcan be used to deploy Config rules.Example Config rules CloudFormation templatesDeveloper creates a CloudFormation template that defines test resources to be evaluated by the Config ruleExamplecrittertest CloudFormation templatesThe template declares a few outputs that will be used bycritterResources:CompliantResource1:# ...CompliantResource2:# ...NonCompliantResource1:# ...NonCompliantResource2:# ...Outputs:ConfigRuleName:Value:my-custom-config-ruleCompliantResourceIds:Value:!Sub"${CompliantResource1.Id},${CompliantResource2.Id}"NonCompliantResourceIds:Value:!Sub"${NonCompliantResource1.Id},${NonCompliantResource2.Id}"Developer triggerscritterfrom the command line (or from within a CI/CD system).critterthen ...Deploys thecrittertest CloudFormation stackWaits for the resources to be evaluated by the Config ruleValidates that resources declared in outputCompliantResourceIdswere evaluated asCOMPLIANTand resources declared in outputNonCompliantResourceIdswere evaluated asNON_COMPLIANTDeletes thecrittertest CloudFormation stackDeveloper is confident their Config rule will evaluate resources as expected!InstallInstall withPythonpipfromthe GitHub projectusing git + ssh:pipinstallgit+ssh://git@github.com:awslabs/critter.gitAftercritteris installed, you can verify the installation by showing the help:critter -h.UsageDeploy an AWS Config rule. The rule can be a managed rule or a custom rule that utilizes an AWS Lambda function for resource valuation.Create acrittertest CloudFormation template (i.e.my-test-template.yml) that deploys at least one resource you expect the rule to evaluate asCOMPLIANTand at least one resource you expect the rule to evaluate asNON_COMPLIANT.Declare the followingOutputsin thecrittertest CloudFormation template (documentation below):ConfigRuleName: The name of the Config rule to testCompliantResourceIds: A comma separated list of one or more AWS Config resource IDs expected to be evaluated asCOMPLIANT.NonCompliantResourceIds: A comma separated list of one or more AWS Config resource IDs expected to be evaluated asNON_COMPLIANT.critter TEMPLATE(i.e.critter ./my-test-template.yml) deploys thecrittertest CloudFormation stack and validates the resource evaluations.critterAWS integration is configured withstandardboto3configuration (environment variables and the~/.aws/configfile).Thecrittertest CloudFormation stack will be deleted after testingOnSuccessby default (i.e. if all tests pass). This behavior can be controlled with--delete-stack.Continuous IntegrationTo understand howcrittercan be utilized in a Continuous Integration (CI) workflow to automatically test changes to AWS Config rules, seethe AWS CodeBuild CI example inexamples/ci-pipelines/aws-codebuild/.AWS Config Resource IDsMost AWS resources have anidattribute (or similar) that is used as the AWS Config resource ID. For EC2 instances, resource IDs are the EC2 instance IDs (i.e.i-111111111aaaaaaaa,i-222222222bbbbbbbb). For VPC security groups, the resource ID is the security group ID (i.e.sg-333333333cccccccc). For IAM roles, the resource ID is the role ID (i.e.AROAJI4AVVEXAMPLE, which can be retrieved in a CloudFormation template usingFn::Sub '${MyIamRole.RoleId}').Generally, resource IDs can be obtained using the CloudFormation intrinsicReffunction with the resource's logical name. To confirm which attribute is used as a resource's Config resource ID, there are a few options:Browse resources inthe AWS Config consoleBrowse resources usingthe AWS CLI:aws configservice list-discovered-resources --resource-type AWS::EC2::InstanceSee theresourceIdfield on example configuration itemshere.critterTest CloudFormation Stack OutputsSeeexamplecrittertest CloudFormation templates in the./examples/test-stacks/. Below is an explanation of each of the supported test stack outputs. Note that CloudFormation stack outputs are always strings.ConfigRuleNameAWS Config rule name to be tested. Be sure the rule is already deployed within the account and Region thecrittertest CloudFormation stack will be deployed to.Example values:"my-custom-config-rule"CompliantResourceIdsA comma separated list of one or more AWS Config resource IDs expected to be evaluated asCOMPLIANT. At least one ofCompliantResourceIdsandNonCompliantResourceIdsmust be output.Typically generated with!Sub "${CompliantResource1.Id},${CompliantResource2.Id}"Example values:"i-11111111111111111""sg-22222222222222222,sg-33333333333333333"NonCompliantResourceIdsA comma separated list of one or more AWS Config resource IDs expected to be evaluated asNON_COMPLIANT. At least one ofCompliantResourceIdsandNonCompliantResourceIdsmust be output.Typically generated with!Sub "${NonCompliantResource1.Id},${NonCompliantResource2.Id}"Example values:"sg-11111111111111111""i-22222222222222222,i-33333333333333333"DelayAfterDeploySeconds to delay aftercrittertest CloudFormation stack create or update. If the test stack is already deployed and no update occurs, the delay will be skipped.Example values:"60"SkipWaitForResourceRecordingSkip waiting for resources to be recorded in Config. Useful if Config rule is testing resources not natively supported by Config.Allowed values:"True""False"(default behavior)CLI OptionsWarning - This documentation may become out of date until the api stabilizes. Show the up to date help withcritter -h.$ critter -h
usage: critter [-h] [--trigger-rule-evaluation] [--stack-name STACK-NAME] [--stack-tags '[{"Key": "TagKey", "Value": "TagValue"}, ...]']
               [--capabilities CAPABILITY [CAPABILITY ...]] [--delete-stack {Always,OnSuccess,Never}]
               TEMPLATE

critter - AWS Config Rule Integration TesTER

positional arguments:
  TEMPLATE              CloudFormation template(s) to test already deployed Config rule

optional arguments:
  -h, --help            show this help message and exit
  --trigger-rule-evaluation
                        Trigger Config rule evaluation after CloudFormation stack deployment. Useful for periodic evaluation rules. Also ensures the rule
                        evaluation occured after stack deployment.
  --stack-name STACK-NAME
                        CloudFormation stack name (default is generated from TEMPLATE file name)
  --stack-tags '[{"Key": "TagKey", "Value": "TagValue"}, ...]'
                        Tags to associate with the CloudFormation stack formatted as a JSON string
  --capabilities CAPABILITY [CAPABILITY ...]
                        CloudFormation capabilities needed to deploy the stack (i.e. CAPABILITY_IAM, CAPABILITY_NAMED_IAM)
  --delete-stack {Always,OnSuccess,Never}
                        Test outcome that should trigger CloudFormation stack delete (default: OnSuccess)Contributing and SecuritySeeCONTRIBUTINGfor more information.LicenseThis project is licensed under the Apache-2.0 License.