Package: aws-cdk.aws-logs
Amazon CloudWatch Logs Construct Library---AWS CDK v1 has reached End-of-Support on 2023-06-01.
This package is no longer being updated, and users should migrate to AWS CDK v2.For more information on how to migrate, see theMigrating to AWS CDK v2guide.This library supplies constructs for working with CloudWatch Logs.Log Groups/StreamsThe basic unit of CloudWatch is aLog Group. Every log group typically has the
same kind of data logged to it, in the same format. If there are multiple
applications or services logging into the Log Group, each of them creates a newLog Stream.Every log operation creates a "log event", which can consist of a simple string
or a single-line JSON object. JSON objects have the advantage that they afford
more filtering abilities (see below).The only configurable attribute for log streams is the retention period, which
configures after how much time the events in the log stream expire and are
deleted.The default retention period if not supplied is 2 years, but it can be set to
one of the values in theRetentionDaysenum to configure a different
retention period (including infinite retention).# Configure log group for short retentionlog_group=LogGroup(stack,"LogGroup",retention=RetentionDays.ONE_WEEK)# Configure log group for infinite retentionlog_group=LogGroup(stack,"LogGroup",retention=Infinity)LogRetentionTheLogRetentionconstruct is a way to control the retention period of log groups that are created outside of the CDK. The construct is usually
used on log groups that are auto created by AWS services, such asAWS
lambda.This is implemented using aCloudFormation custom
resourcewhich pre-creates the log group if it doesn't exist, and sets the specified log retention period (never expire, by default).By default, the log group will be created in the same region as the stack. ThelogGroupRegionproperty can be used to configure
log groups in other regions. This is typically useful when controlling retention for log groups auto-created by global services that
publish their log group to a specific region, such as AWS Chatbot creating a log group inus-east-1.Resource PolicyCloudWatch Resource Policies allow other AWS services or IAM Principals to put log events into the log groups.
A resource policy is automatically created whenaddToResourcePolicyis called on the LogGroup for the first time:log_group=logs.LogGroup(self,"LogGroup")log_group.add_to_resource_policy(iam.PolicyStatement(actions=["logs:CreateLogStream","logs:PutLogEvents"],principals=[iam.ServicePrincipal("es.amazonaws.com")],resources=[log_group.log_group_arn]))Or more conveniently, write permissions to the log group can be granted as follows which gives same result as in the above example.log_group=logs.LogGroup(self,"LogGroup")log_group.grant_write(iam.ServicePrincipal("es.amazonaws.com"))Be aware that any ARNs or tokenized values passed to the resource policy will be converted into AWS Account IDs.
This is because CloudWatch Logs Resource Policies do not accept ARNs as principals, but they do accept
Account ID strings. Non-ARN principals, like Service principals or Any princpals, are accepted by CloudWatch.Encrypting Log GroupsBy default, log group data is always encrypted in CloudWatch Logs. You have the
option to encrypt log group data using a AWS KMS customer master key (CMK) should
you not wish to use the default AWS encryption. Keep in mind that if you decide to
encrypt a log group, any service or IAM identity that needs to read the encrypted
log streams in the future will require the same CMK to decrypt the data.Here's a simple example of creating an encrypted Log Group using a KMS CMK.importaws_cdk.aws_kmsaskmslogs.LogGroup(self,"LogGroup",encryption_key=kms.Key(self,"Key"))See the AWS documentation for more detailed information aboutencrypting CloudWatch
Logs.Subscriptions and DestinationsLog events matching a particular filter can be sent to either a Lambda function
or a Kinesis stream.If the Kinesis stream lives in a different account, aCrossAccountDestinationobject needs to be added in the destination account which will act as a proxy
for the remote Kinesis stream. This object is automatically created for you
if you use the CDK Kinesis library.Create aSubscriptionFilter, initialize it with an appropriatePattern(see
below) and supply the intended destination:importaws_cdk.aws_logs_destinationsasdestinations# fn: lambda.Function# log_group: logs.LogGrouplogs.SubscriptionFilter(self,"Subscription",log_group=log_group,destination=destinations.LambdaDestination(fn),filter_pattern=logs.FilterPattern.all_terms("ERROR","MainThread"))Metric FiltersCloudWatch Logs can extract and emit metrics based on a textual log stream.
Depending on your needs, this may be a more convenient way of generating metrics
for you application than making calls to CloudWatch Metrics yourself.AMetricFiltereither emits a fixed number every time it sees a log event
matching a particular pattern (see below), or extracts a number from the log
event and uses that as the metric value.Example:MetricFilter(self,"MetricFilter",log_group=log_group,metric_namespace="MyApp",metric_name="Latency",filter_pattern=FilterPattern.exists("$.latency"),metric_value="$.latency")Remember that if you want to use a value from the log event as the metric value,
you must mention it in your pattern somewhere.A very simple MetricFilter can be created by using thelogGroup.extractMetric()helper function:# log_group: logs.LogGrouplog_group.extract_metric("$.jsonField","Namespace","MetricName")Will extract the value ofjsonFieldwherever it occurs in JSON-structed
log records in the LogGroup, and emit them to CloudWatch Metrics under
the nameNamespace/MetricName.Exposing Metric on a Metric FilterYou can expose a metric on a metric filter by calling theMetricFilter.metric()API.
This has a default ofstatistic = 'avg'if the statistic is not set in theprops.# log_group: logs.LogGroupmf=logs.MetricFilter(self,"MetricFilter",log_group=log_group,metric_namespace="MyApp",metric_name="Latency",filter_pattern=logs.FilterPattern.exists("$.latency"),metric_value="$.latency")# expose a metric from the metric filtermetric=mf.metric()# you can use the metric to create a new alarmcloudwatch.Alarm(self,"alarm from metric filter",metric=metric,threshold=100,evaluation_periods=2)PatternsPatterns describe which log events match a subscription or metric filter. There
are three types of patterns:Text patternsJSON patternsSpace-delimited table patternsAll patterns are constructed by using static functions on theFilterPatternclass.In addition to the patterns above, the following special patterns exist:FilterPattern.allEvents(): matches all log events.FilterPattern.literal(string): if you already know what pattern expression to
use, this function takes a string and will use that as the log pattern. For
more information, see theFilter and Pattern
Syntax.Text PatternsText patterns match if the literal strings appear in the text form of the log
line.FilterPattern.allTerms(term, term, ...): matches if all of the given terms
(substrings) appear in the log event.FilterPattern.anyTerm(term, term, ...): matches if all of the given terms
(substrings) appear in the log event.FilterPattern.anyTermGroup([term, term, ...], [term, term, ...], ...): matches if
all of the terms in any of the groups (specified as arrays) matches. This is
an OR match.Examples:# Search for lines that contain both "ERROR" and "MainThread"pattern1=logs.FilterPattern.all_terms("ERROR","MainThread")# Search for lines that either contain both "ERROR" and "MainThread", or# both "WARN" and "Deadlock".pattern2=logs.FilterPattern.any_term_group(["ERROR","MainThread"],["WARN","Deadlock"])JSON PatternsJSON patterns apply if the log event is the JSON representation of an object
(without any other characters, so it cannot include a prefix such as timestamp
or log level). JSON patterns can make comparisons on the values inside the
fields.Strings: the comparison operators allowed for strings are=and!=.
String values can start or end with a*wildcard.Numbers: the comparison operators allowed for numbers are=,!=,<,<=,>,>=.Fields in the JSON structure are identified by identifier the complete object as$and then descending into it, such as$.fieldor$.list[0].field.FilterPattern.stringValue(field, comparison, string): matches if the given
field compares as indicated with the given string value.FilterPattern.numberValue(field, comparison, number): matches if the given
field compares as indicated with the given numerical value.FilterPattern.isNull(field): matches if the given field exists and has the
valuenull.FilterPattern.notExists(field): matches if the given field is not in the JSON
structure.FilterPattern.exists(field): matches if the given field is in the JSON
structure.FilterPattern.booleanValue(field, boolean): matches if the given field
is exactly the given boolean value.FilterPattern.all(jsonPattern, jsonPattern, ...): matches if all of the
given JSON patterns match. This makes an AND combination of the given
patterns.FilterPattern.any(jsonPattern, jsonPattern, ...): matches if any of the
given JSON patterns match. This makes an OR combination of the given
patterns.Example:# Search for all events where the component field is equal to# "HttpServer" and either error is true or the latency is higher# than 1000.pattern=logs.FilterPattern.all(logs.FilterPattern.string_value("$.component","=","HttpServer"),logs.FilterPattern.any(logs.FilterPattern.boolean_value("$.error",True),logs.FilterPattern.number_value("$.latency",">",1000)))Space-delimited table patternsIf the log events are rows of a space-delimited table, this pattern can be used
to identify the columns in that structure and add conditions on any of them. The
canonical example where you would apply this type of pattern is Apache server
logs.Text that is surrounded by"..."quotes or[...]square brackets will
be treated as one column.FilterPattern.spaceDelimited(column, column, ...): construct aSpaceDelimitedTextPatternobject with the indicated columns. The columns
map one-by-one the columns found in the log event. The string"..."may
be used to specify an arbitrary number of unnamed columns anywhere in the
name list (but may only be specified once).After constructing aSpaceDelimitedTextPattern, you can use the following
two members to add restrictions:pattern.whereString(field, comparison, string): add a string condition.
The rules are the same as for JSON patterns.pattern.whereNumber(field, comparison, number): add a numerical condition.
The rules are the same as for JSON patterns.Multiple restrictions can be added on the same column; they must all apply.Example:# Search for all events where the component is "HttpServer" and the# result code is not equal to 200.pattern=logs.FilterPattern.space_delimited("time","component","...","result_code","latency").where_string("component","=","HttpServer").where_number("result_code","!=",200)Logs Insights Query DefinitionCreates a query definition for CloudWatch Logs Insights.Example:logs.QueryDefinition(self,"QueryDefinition",query_definition_name="MyQuery",query_string=logs.QueryString(fields=["@timestamp","@message"],sort="@timestamp desc",limit=20))NotesBe aware that Log Group ARNs will always have the string:*appended to
them, to match the behavior ofthe CloudFormationAWS::Logs::LogGroupresource.