Repository: aws-cloudformation/cfn-lint

Issue:
E2529 error with multiple `AWS::Logs::SubscriptionFilter` resources.
*cfn-lint version: 0.38.0*

*[`E2529`](https://github.com/aws-cloudformation/cfn-python-lint/blob/master/docs/rules.md#E2529) error with multiple [`AWS::Logs::SubscriptionFilter`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-logs-subscriptionfilter.html) resources.*

I have a CloudFormation template (using AWS SAM) that has multiple SubscriptionFilters. These SubscriptionFilters have the same `LogGroupName`, which causes an `E2529` error despite these filters having separate FilterPatterns. The template passes the [`aws cloudformation validate-template`](https://docs.aws.amazon.com/cli/latest/reference/cloudformation/validate-template.html) command.

[`src/cfnlint/rules/resources/lmbd/EventsLogGroupName.py`](https://github.com/aws-cloudformation/cfn-python-lint/blob/master/src/cfnlint/rules/resources/lmbd/EventsLogGroupName.py)

```
  MainFunctionLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      RetentionInDays: 14
      LogGroupName: !Join ["", [/aws/lambda/, !Ref MainFunction]]

  MainFunctionLogFilter:
    Type: AWS::Logs::SubscriptionFilter
    Properties:
      DestinationArn: !Ref LogIngestionARN
      FilterPattern: "FilterPattern1"
      LogGroupName: !Ref MainFunctionLogGroup

  SecondaryLogFilter:
    Type: AWS::Logs::SubscriptionFilter
    Properties:
      DestinationArn: !Ref LogIngestionARN
      FilterPattern: "FilterPattern2"
      LogGroupName: !Ref MainFunctionLogGroup
```


Discussion:
Few things here.
- `aws cloudformation validate-template` isn't detailed enough to catch this error.  I'm assuming this template deploys correctly?  That is the most important thing.
- It looks like the limit for Subscription Filters has increased to 2 from 1. See [current docs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/cloudwatch_limits_cwl.html) and what is in the [docs repo](https://github.com/awsdocs/amazon-cloudwatch-logs-user-guide/blob/master/doc_source/cloudwatch_limits_cwl.md).

The error message for this rule isn't well written.  It has to do with Serverless but also applies to any SubscriptionFilter so it should be cleaned up.  Additionally I believe this rule is testing the limits of SubscriptionFilters which has been changed so we need to change this rule to the new standard..

Resolving patch:
diff --git a/src/cfnlint/rules/resources/lmbd/EventsLogGroupName.py b/src/cfnlint/rules/resources/lmbd/EventsLogGroupName.py
--- a/src/cfnlint/rules/resources/lmbd/EventsLogGroupName.py
+++ b/src/cfnlint/rules/resources/lmbd/EventsLogGroupName.py
@@ -2,6 +2,7 @@
 Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
 SPDX-License-Identifier: MIT-0
 """
+import json
 from cfnlint.rules import CloudFormationLintRule
 from cfnlint.rules import RuleMatch
 
@@ -9,45 +10,41 @@
 class EventsLogGroupName(CloudFormationLintRule):
     """Check if the settings of multiple subscriptions are included for one LogGroup"""
     id = 'E2529'
-    shortdesc = 'Check for duplicate Lambda events'
-    description = 'Check if there are any duplicate log groups in the Lambda event trigger element.'
+    shortdesc = 'Check for SubscriptionFilters have beyond 2 attachments to a CloudWatch Log Group'
+    description = 'The current limit for a CloudWatch Log Group is they can have 2 subscription filters. ' \
+        'We will look for duplicate LogGroupNames inside Subscription Filters and make sure they are within 2. ' \
+        'This doesn\'t account for any other subscription filters getting set.'
     source_url = 'https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#user-content-cloudwatchlogs'
     tags = ['resources', 'lambda']
+    limit = 2
 
     def check_events_subscription_duplicated(self, cfn):
         """Check if Lambda Events Subscription is duplicated"""
         matches = []
-        message = 'You must specify the AWS::Serverless::Function event correctly. ' \
-                  'LogGroups are duplicated. '
-
-        log_group_name_list = self.__get_log_group_name_list(cfn)
-
-        if self.__is_duplicated(log_group_name_list):
-            matches.append(
-                RuleMatch(
-                    'path', message.format()
+        message = 'You can only have {} Subscription Filters per CloudWatch Log Group'.format(self.limit)
+
+        log_group_paths = self.__get_log_group_name_list(cfn)
+        for _, c in log_group_paths.items():
+            if len(c) > self.limit:
+                matches.append(
+                    RuleMatch(
+                        ['Resources', c[2]], message.format()
+                    )
                 )
-            )
 
         return matches
 
-    def __is_duplicated(self, duplicate_list):
-        unique_list = self.__remove(duplicate_list)
-        return len(unique_list) != len(duplicate_list)
-
-    def __remove(self, duplicate):
-        final_list = []
-        for ele in duplicate:
-            if ele not in final_list:
-                final_list.append(ele)
-        return final_list
-
     def __get_log_group_name_list(self, cfn):
-        log_group_name_list = []
+        log_group_paths = {}
         for value in cfn.get_resources('AWS::Logs::SubscriptionFilter').items():
             prop = value[1].get('Properties')
-            log_group_name_list.append(prop.get('LogGroupName'))
-        return log_group_name_list
+            log_group_name = json.dumps(prop.get('LogGroupName'))
+
+            if log_group_name not in log_group_paths:
+                log_group_paths[log_group_name] = []
+
+            log_group_paths[log_group_name].append(value[0])
+        return log_group_paths
 
     def match(self, cfn):
         """Check if Lambda Events Subscription is duplicated"""


Test patch:
diff --git a/test/fixtures/templates/bad/some_logs_stream_lambda.yaml b/test/fixtures/templates/bad/some_logs_stream_lambda.yaml
--- a/test/fixtures/templates/bad/some_logs_stream_lambda.yaml
+++ b/test/fixtures/templates/bad/some_logs_stream_lambda.yaml
@@ -70,6 +70,11 @@ Resources:
           Properties:
             LogGroupName: !Ref FunctionALogGroup
             FilterPattern: ""
+        FunctionDLogGroup:
+          Type: CloudWatchLogs
+          Properties:
+            LogGroupName: !Ref FunctionALogGroup
+            FilterPattern: ""
   LogSubscriptionFunctionLogGroup:
     Type: AWS::Logs::LogGroup
     Properties: