Package: awsinsights
awsinsightsGet and filter logs from multiple log groups of AWS CloudWatch and filter CloudWatch logs using predefined regular expressions.This script usesAWS CloudWatch Insightsservice.Installawsinsightsis released to public PyPI -awsinsights. It can be installed using this command:pip install awsinsightsBasic UsageSet up AWS Credentials.Define apps(sets of CloudWatch log groups assigned to app name) in$HOME/.awsinsights.jsonfile. Seeexample filebelow.Get all logs from thesimplebookapp (2 log groups pre-defined) since last 30 minutes:awsinsights --timedelta 30m --appname simplebookFilter logs fromsimplebookapp since last 7d containing words 'Monday' or
'Tuesday'(you can use any Regular Expression in )awsinsights --timedelta 7d --appname simplebook --filter "Monday|Tuesday"Advanced UsageGet logs fromsimplebookfrom 1 Jan 2021 10:00am to 2 Jan 2021 9:00am
which contain 'Exception' or 'ERROR' on PROD environmentawsinsights --env prod --start 2021-01-01 10:00:00 --end 2021-01-02 09:00:00 --appname simplebook --filter "Exception|ERROR"Get all logs from CloudWatch log groupsgroup-one-devand/aws/lambda/group-two-devsince last 2 hours:awsinsights --timedelta 2h --log_groups "group-one-dev" "/aws/lambda/group-two-dev"Tail modeawsinsights allows to listen CloudWatch in live mode which is calledtail mode.It can be activated using--tailoption.Example - listening for ERRORs and Exceptions in tail mode:awsinsights --timedelta 30m --appname simplebook --filter "ERROR|Exception" --tailNOTE: Please notice that there might befew mins delaybetween the time when log really happened
and the time when it will appear in output of awsinsights'tail mode.Example of config fileConfig file should be placed in$HOME/.awsinsights.jsonThis example file contains 2 apps:simplebookandsecondapp.
Each app consits of 2 CloudWatch log groups.{
    "simplebook": [
        "/aws/lambda/simple-books-catalog-api-$ENV",
        "/aws/lambda/api-task-2-ad"
    ],
    "secondapp": [
        "first-log-group",
        "/aws/lambda/second-log-group"
    ]
}Output fileLogs will be written to output file. Output file will be:/tmp/{appname}.logif app name is definedusing--appnameoption OR/tmp/awsinsights.logif app name is NOT definedHelpawsinsights [-h] [--timedelta TIMEDELTA] [--start START] [--end END]
                   [--filter FILTER]
                   (--appname APPNAME | --log_groups LOG_GROUPS [LOG_GROUPS ...])
                   [--env ENV] [--query QUERY]

optional arguments:
  -h, --help            show this help message and exit
  --timedelta TIMEDELTA
                        delta time since now when logs should be filtered ex.
                        120m, 3h, 2d. Default: 60m
  --start START         start time of grabbing logs. Format: YYYY-MM-DD
                        HH:MM:SS
  --end END             end time of grabbing logs. Format: YYYY-MM-DD HH:MM:SS
  --filter FILTER       Regular expression for filtering logs
  --appname APPNAME     name of the app which logs should be analysed. App
                        names should have logs groups configured in
                        .awsinsightsrc file. See README.md file.
  --log_groups LOG_GROUPS [LOG_GROUPS ...]
                        list of the log groups " "to analyse (up to 20)
  --env ENV             env name. It can be used to resolve "{env}" var in log
                        groups names. Default: dev
  --query QUERY         Custom full AWS CloudWatch Insights query. " "Default:
                        fields @timestamp, @message | filter @message like //
                        | sort @timestamp
  --tail                TAIL MODE. If set to "true", It will listen for live
                        logs forever