Question: Which log group is causing a sudden increase in my CloudWatch Logs bill?

My Amazon CloudWatch Logs bill is unusually high, and I want to determine which log group is increasing my CloudWatch Logs costs.

Short descriptionSudden increases in CloudWatch Logs bills often result from an increase in ingested or storage data in a particular log group. Use CloudWatch Logs metrics to check data usage, and review your AWS bill to identify the log group that's responsible for bill increases.

ResolutionCheck the amount of data that you're ingestingThe IncomingBytes metric shows you the amount of ingested data in your CloudWatch log groups in near real time. This metric can help you to determine the following points:Which log group is the highest contributor towards your billWhether there's a spike in the incoming data to your log groups or a gradual increase due to new applicationsHow much data is pushed during a particular period of timeQuery a small set of log groups using the CloudWatch console1.    Open the Amazon CloudWatch console.2.    In the navigation pane, choose Metrics.3.    For each of your log groups, select the individual IncomingBytes metrics. Then, choose the Graphed metrics tab.4.    For Statistic, choose Sum.5.    For Period, choose 30 Days.6.    Choose the Graph options tab, and then choose Number.7.    At the top right of the graph, choose Custom, and then choose Absolute. Select a start and end date that corresponds with the last 30 days.Query a large set of log groups using the CloudWatch console1.    Open the Amazon CloudWatch console.2.    In the navigation pane, choose All metrics.3.    Choose the Graphed metrics tab. Then, from the Add metric dropdown list, choose Start with an empty expression.4.    Copy the following math expression, and then paste it into the Edit math expression field:SORT(REMOVE_EMPTY(SEARCH('{AWS/Logs,LogGroupName} MetricName="IncomingBytes"', 'Sum', 2592000)),SUM, DESC)After you paste the expression, choose Apply.5.    Choose the Graph options tab, and then choose Number.6.    At the top right of the graph, choose Custom. Then, choose Absolute. Select a start and end date that corresponds with the last 30 days.Note: You can graph up to 500 metrics using this method.Query a large set of log groups using an API callNote: Before you run the following API calls, review the associated costs with making API calls. It's a best practice to distribute the ListMetrics call to avoid throttling. The default limit for ListMetrics is 25 transactions per second. However, you can request a limit increase, if necessary.1.    Make a ListMetrics call. This call finds all log group names that ingested data in the past 14 days. Use the following parameters:Namespace: AWS/LogsMetricName: IncomingBytes2.    Make a GetMetricData call. This call finds the sum of all incoming bytes in a month for every log group name that you get from the ListMetrics call. Use the following parameters:Namespace: AWS/LogsMetricName: IncomingBytesDimensions: As received from the ListMetrics callStartTime: [Date and time 14 days before the current date]EndTime: [Current date and time]Period: [EndTime - StartTime, in seconds]Statistics: Sum3.    To display the log group names with the highest Ingested data amounts, sort the resulting data points in descending order.To be sure that ingested data charges don't exceed a specified limit in the future, create a CloudWatch alarm.Review your storage data usageCheck your most recent AWS bill to see how much storage data you used in the previous billing cycle.

Related informationCloudWatch billing and cost