Package: watchtower2
watchtower_2 is a django python2 adaptation of watchtower by kislyukWatchtower is a log handler forAmazon Web Services CloudWatch Logs.CloudWatch Logs is a log management service built into AWS. It is conceptually similar to services like Splunk and
Loggly, but is more lightweight, cheaper, and tightly integrated with the rest of AWS.Watchtower, in turn, is a lightweight adapter between thePython logging systemand CloudWatch Logs. It uses theboto3 AWS SDK, and lets you plug your application logging directly into CloudWatch without the need
to install a system-wide log collector likeawscli-cwlogsand round-trip
your logs through the instance’s syslog. It aggregates logs into batches to avoid sending an API request per each log
message, while guaranteeing a delivery deadline (60 seconds by default).Installationpip install watchtowerSynopsisInstallawscliand set your AWS credentials (runaws configure).importwatchtower,logginglogging.basicConfig(level=logging.INFO)logger=logging.getLogger(__name__)logger.addHandler(watchtower.CloudWatchLogHandler())logger.info("Hi")logger.info(dict(foo="bar",details={}))After running the example, you can see the log output in yourAWS console.Example: Django logging with WatchtowerThis is an example of Watchtower integration with Django. In your Django project, add the following tosettings.py:fromboto3.sessionimportSessionAWS_ACCESS_KEY_ID='your access key'AWS_SECRET_ACCESS_KEY='your secret access key'AWS_REGION_NAME='your region'boto3_session=Session(aws_access_key_id=AWS_ACCESS_KEY_ID,aws_secret_access_key=AWS_SECRET_ACCESS_KEY,region_name=AWS_REGION_NAME)LOGGING={'version':1,'disable_existing_loggers':False,'root':{'level':logging.ERROR,'handlers':['console'],},'formatters':{'simple':{'format':"%(asctime)s[%(levelname)-8s]%(message)s",'datefmt':"%Y-%m-%d%H:%M:%S"},'aws':{# you can add specific format for aws here'format':"%(asctime)s[%(levelname)-8s]%(message)s",'datefmt':"%Y-%m-%d%H:%M:%S"},},'handlers':{'watchtower':{'level':'DEBUG','class':'watchtower.CloudWatchLogHandler','boto3_session':boto3_session,'log_group':'MyLogGroupName','stream_name':'MyStreamName','formatter':'aws',},},'loggers':{'django':{'level':'INFO','handlers':['watchtower'],'propagate':False,},# add your other loggers here...},}Using this configuration, every log statement from Django will be sent to Cloudwatch in the log groupMyLogGroupNameunder the stream nameMyStreamName. Instead of setting credentials viaAWS_ACCESS_KEY_IDand other variables,
you can also assign an IAM role to your instance and omit those parameters, prompting boto3 to ingest credentials from
instance metadata.(See also theDjango logging documentation).Examples: Querying CloudWatch logsThis section is not specific to Watchtower. It demonstrates the use of awscli and jq to read and search CloudWatch logs
on the command line.For the Flask example above, you can retrieve your application logs with the following two commands:aws logs get-log-events --log-group-name watchtower --log-stream-name loggable | jq '.events[].message'
aws logs get-log-events --log-group-name watchtower --log-stream-name werkzeug | jq '.events[].message'CloudWatch Logs supports alerting and dashboards based onmetric filters, which are pattern
rules that extract information from your logs and feed it to alarms and dashboard graphs. The following example shows
logging structured JSON data using Watchtower, setting up a metric filter to extract data from the log stream, a dashboard to
visualize it, and an alarm that sends an email:TODOExamples: Python Logging ConfigPython has the ability to provide a configuration file that can be loaded in order to separate the logging
configuration from the code. Historically, Python has used thelogging.config.fileConfigfunction to do
so, however, that feature lacks the ability to use keyword args. Python 2.7 introduced a new feature to
handle logging that is more robust -logging.config.dictConfigwhich provides the ability to do more
advanced Filters, but more importantly adds keyword args, thus allowing thelogging.configfunctionality
to instantiate Watchtower.The following are two example YAML configuration files that can be loaded usingPyYaml. The resultingdictobject can then be loaded intologging.config.dictConfig. The first example is a basic example
that relies on the default configuration provided byboto3:# Default AWS Configversion:1formatters:json:format:"[%(asctime)s]%(process)d%(levelname)s%(name)s:%(funcName)s:%(lineno)s-%(message)s"plaintext:format:"[%(asctime)s]%(process)d%(levelname)s%(name)s:%(funcName)s:%(lineno)s-%(message)s"handlers:console:():logging.StreamHandlerlevel:DEBUGformatter:plaintextstream:sys.stdoutwatchtower:formatter:jsonlevel:DEBUG():watchtower.CloudWatchLogHandlerlog_group:loggerstream_name:loggablesend_interval:1create_log_group:Falseloggers:root:handlers:[console,watchtower,logfile]boto:handlers:[console]boto3:handlers:[console]botocore:handlers:[console]requests:handlers:[console]The above works well if you can use the default configuration, or rely on environmental variables.
However, sometimes one may want to use different credentials for logging than used for other functionality;
in this case theboto3_profile_nameoption to Watchtower can be used to provide a profile name:# AWS Config Profileversion:1formatters:json:format:"[%(asctime)s]%(process)d%(levelname)s%(name)s:%(funcName)s:%(lineno)s-%(message)s"plaintext:format:"[%(asctime)s]%(process)d%(levelname)s%(name)s:%(funcName)s:%(lineno)s-%(message)s"handlers:console:():logging.StreamHandlerlevel:DEBUGformatter:plaintextstream:sys.stdoutwatchtower:formatter:jsonlevel:DEBUG():watchtower.CloudWatchLogHandlerlog_group:loggerstream_name:loggableboto3_profile_name:watchtowerloggersend_interval:1create_log_group:Falseloggers:root:handlers:[console,watchtower,logfile]boto:handlers:[console]boto3:handlers:[console]botocore:handlers:[console]requests:handlers:[console]For the more advanced configuration, the following configuration file will provide
the matching credentials to thewatchtowerloggerprofile:[profile watchtowerlogger]aws_access_key_id=MyAwsAccessKeyaws_secret_access_key=MyAwsSecretAccessKeyregion=us-east-1Finally, the following shows how to load the configuration into the working application:AuthorsAndrey KislyukMoshe FromerLinksProject home page (GitHub)DocumentationPackage distribution (PyPI)AWS CLI CloudWatch Logs pluginDocker awslogs adapterBugsPlease report bugs, issues, feature requests, etc. onGitHub.LicenseLicensed under the terms of theApache License, Version 2.0.