Package: cloudwatch-log-retention-manager
CloudWatch Logs Retention Manager is a configurable tool that can be used to validate or enforce CloudWatch log retention rules.Unlike other tools in this space, thecwlrmprovides flexibility through the use of configuration, this configuration can apply to a subset of log groups - useful when working in shared environments.UsageInstallcwlrmthrough pip and create aconfig.ymlfile within your project.➜ cwlrm --help
usage: cwlrm [-h] [--version] [-u] [-c CONFIGFILE] [-s] [-sc] [-ic] [-v] [-vv]

CloudWatch logs retention manager

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -u, --update          update settings in AWS
  -c CONFIGFILE, --config CONFIGFILE
                        location of config.yaml
  -s, --show_all        show all log groups in filter
  -sc, --show_cost      total cost per groups
  -ic, --show_individual_cost
                        show storage cost for each log group in filter
  -v, --verbose         set loglevel to INFO
  -vv, --very-verbose   set loglevel to DEBUGOnce installed, all that is needed is aconfig.ymlfile containing the log group patterns and retention periods to set.
The tool supports aCompliancemode, where the live AWS environment will be validated
against the described config file.When you are ready to apply the changes, runcwlrm -uto enforce Compliance as defined.The format of theconfig.ymlfile is as follows:A single root object names retentionPatternsAn array of child objects consisting of the following attributes:name- mandatory - names a section of log groups to be processedlogPrefix- optional - the pattern of logs to apply the group to - default: all logsretentionInDays- optional - the log retention period to set - default: Foreveroverride- optional - if set will require the exact retentionInDays values to be compliant, otherwise any log retention is considered compliant - default: falseshowAlways- optional - when set, will show all log groups that match irrespective of their compliance - default: falseOptionally thelogPrefixcan be an array of patterns to be collected into the matching group.Example:retentionPatterns:
  - name: CodeBuild jobs
    logPrefix: /aws/codebuild/
    retentionInDays: 14
    override: true
    showAlways: true

  - name: AWS Glue Crawlers
    logPrefix: /aws-glue/crawlers
    retentionInDays: 14

  - name: Specific application lambda logs
    logPrefix:
     - /aws/lambda/Application1
     - /aws/lambda/Application2
     - /aws/lambda/Application3
    retentionInDays: 45

  - name: Lambda logs
    logPrefix: /aws/lambda
    retentionInDays: 30

  - name: API Gateway access logs
    logPrefix: /aws/api-gateway/
    retentionInDays: 120

  - name: API Gateway Execution Logs
    logPrefix: API-Gateway-Execution-Logs
    retentionInDays: 120

  - name: All remaining log groups
    showAlways: true
    retentionInDays: 90IMPORTANT Note:For each run a log group will only be processed by a single control - the first control to process the log group will be the active one, if subsequent patterns are to match that same log group it will be ignored.ExamplesSome example config files.Simple retention patternretentionPatterns:
  - name: AWS Glue Crawlers
    logPrefix: /aws-glue/crawlers
    retentionInDays: 14

  - name: CloudFront Lambda Function logs
    logPrefix: /aws/cloudfront/LambdaEdge
    retentionInDays: 30This configuration will ensure that all glue-crawler and lambdaedge function logs have a retention set, when applied will set 15 and 30 days respectivly.Specific compliance retention periodretentionPatterns:
  - name: CodeBuild jobs
    logPrefix: /aws/codebuild/
    retentionInDays: 14
    override: trueThis configuration will enforce that all codebuild log groups have specifically a 14 day retention period - any other value will be considered “Non Compliant”Logs that should be retained ForeverretentionPatterns:
  - name: Application audit logs
    logPrefix: /application/audit/
    showAlways: trueIf a log group contains some sort of audit required for long term archive, Compliance can be achieved by specifiying the logGroup pattern and omitting theretentionInDaysparameter.
In this scenario log groups without retention periods are considered Compliant, and are configured to be shown on each execution.Show all remaining log groupsretentionPatterns:
  - name: API Gateway access logs
    logPrefix: /aws/api-gateway/
    retentionInDays: 30

  - name: Lambda API logs
    logPrefix: /aws/lambda/
    retentionInDays: 30

  - name: API Gateway Execution Logs
    logPrefix: API-Gateway-Execution-Logs
    retentionInDays: 30

  - name: All remaining log groups
    showAlways: true
    retentionInDays: 90A normal operation ofcwlrmwill only process the specified
log groups (as selected by the logPrefix) - in some cases you may
want to apply a default to the entire account, this can be done
through the omission of thelogPrefixattribute.  Any log group
matching the previous conditions willNoteThis project has been set up using PyScaffold 4.0.1. For details and usage
information on PyScaffold seehttps://pyscaffold.org/.