internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: aws-cloudformation/cfn-lint

Issue:
E3002 error thrown when using parameter lists
*cfn-lint version cfn-lint 0.8.3

*Description of issue. When using SSM parameters that are type list, the linter doesn't properly recognize them as a list and thows an error E3002 

Specifically E3002 Property PreferredAvailabilityZones should be of type List or Parameter should be a list for resource cacheRedisV1
us-east-1.yaml:249:7

The parameter section for this is:
Parameters:
  azList:
    Type: "AWS::SSM::Parameter::Value<List<String>>"
    Description: "The list of AZs from Parameter Store"
    Default: '/regionSettings/azList'

The resource is defined as:
cacheForumRedisV1:
    Type: AWS::ElastiCache::CacheCluster
    Properties:
      PreferredAvailabilityZones: !Ref azList



Discussion:

Resolving patch:
diff --git a/src/cfnlint/rules/resources/properties/Properties.py b/src/cfnlint/rules/resources/properties/Properties.py
--- a/src/cfnlint/rules/resources/properties/Properties.py
+++ b/src/cfnlint/rules/resources/properties/Properties.py
@@ -225,7 +225,7 @@ def propertycheck(self, text, proptype, parenttype, resourcename, path, root):
                                     if ref in parameternames:
                                         param_type = self.cfn.template['Parameters'][ref]['Type']
                                         if param_type:
-                                            if not param_type.startswith('List<') and not param_type == 'CommaDelimitedList':
+                                            if 'List<' not in param_type and '<List' not in param_type and not param_type == 'CommaDelimitedList':
                                                 message = 'Property {0} should be of type List or Parameter should ' \
                                                           'be a list for resource {1}'
                                                 matches.append(


Test patch:
diff --git a/test/fixtures/templates/good/resource_properties.yaml b/test/fixtures/templates/good/resource_properties.yaml
--- a/test/fixtures/templates/good/resource_properties.yaml
+++ b/test/fixtures/templates/good/resource_properties.yaml
@@ -218,6 +218,10 @@ Outputs:
           - ElasticLoadBalancer
           - DNSName
 Parameters:
+  azList:
+    Type: "AWS::SSM::Parameter::Value<List<String>>"
+    Description: "The list of AZs from Parameter Store"
+    Default: '/regionSettings/azList'
   InstanceType:
     AllowedValues:
     - t1.micro
@@ -606,3 +610,10 @@ Resources:
       DistributionConfig:
         Aliases: !Ref Aliases
         Enabled: True
+  cacheForumRedisV1:
+    Type: AWS::ElastiCache::CacheCluster
+    Properties:
+      PreferredAvailabilityZones: !Ref azList
+      CacheNodeType: String
+      Engine: String
+      NumCacheNodes: 1
Repository: aws-cloudformation/cfn-lint

Issue:
Error: 'NoneType' object is not iterable
*cfn-lint version: (`cfn-lint --version`)*
`cfn-lint 0.9.0`

NOTE: version 0.8.3 works fine, so the issue seems to have been introduced in this latest version. 

*Description of issue.*
```
$ cfn-lint test.yaml
2018-11-13 18:40:28,869 - cfnlint - ERROR - Tried to process rules on file test.yaml but got an error: 'NoneType' object is not iterable
```

Please provide as much information as possible:
* Template linting issues:

test.yaml:
```
---
AWSTemplateFormatVersion: '2010-09-09'
Metadata:
  cfn-lint:
    config:
      ignore_checks:
        - E0002

Parameters:
  NodeCount:
    Type: String
    AllowedPattern: '[1-7]'
    Default: 1
```


Thanks.


Discussion:

Resolving patch:
diff --git a/src/cfnlint/config.py b/src/cfnlint/config.py
--- a/src/cfnlint/config.py
+++ b/src/cfnlint/config.py
@@ -340,9 +340,9 @@ def set_template_args(self, template):
                             defaults['regions'] = config_value
                     if config_name == 'append_rules':
                         if isinstance(config_value, list):
-                            defaults['override_spec'] = config_value
+                            defaults['append_rules'] = config_value
                     if config_name == 'override_spec':
-                        if isinstance(config_value, (six.string_types, six.text_type)):
+                        if isinstance(config_value, (six.string_types)):
                             defaults['override_spec'] = config_value
                     if config_name == 'ignore_bad_template':
                         if isinstance(config_value, bool):
@@ -377,7 +377,6 @@ def _get_argument_value(self, arg_name, is_template, is_config_file):
             return template_value
         if file_value and is_config_file:
             return file_value
-
         return cli_value
 
     @property
diff --git a/src/cfnlint/rules/parameters/Default.py b/src/cfnlint/rules/parameters/Default.py
--- a/src/cfnlint/rules/parameters/Default.py
+++ b/src/cfnlint/rules/parameters/Default.py
@@ -33,8 +33,7 @@ def check_allowed_pattern(self, allowed_value, allowed_pattern, path):
             Check allowed value against allowed pattern
         """
         message = 'Default should be allowed by AllowedPattern'
-
-        if not re.match(allowed_pattern, allowed_value):
+        if not re.match(allowed_pattern, str(allowed_value)):
             return([RuleMatch(path, message)])
 
         return []


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
cfn-lint 0.9.0 broke compatibility with python 3.4
cfn-lint version: 0.9.2

cfn-lint 0.9.0 broke compatibility with python 3.4.

cfn-lint is now using the home() method from pathlib. That method was introduced in python 3.5 and causing the following error when running cfn-lint with python 3.4.

```[etrochim@localhost]~% cfn-lint
Traceback (most recent call last):
  File "/home/etrochim/.local/bin/cfn-lint", line 11, in <module>
    sys.exit(main())
  File "/home/etrochim/.local/lib/python3.4/site-packages/cfnlint/__main__.py", line 28, in main
    (args, filenames, formatter) = cfnlint.core.get_args_filenames(sys.argv[1:])
  File "/home/etrochim/.local/lib/python3.4/site-packages/cfnlint/core.py", line 111, in get_args_filenames
    config = cfnlint.config.ConfigMixIn(cli_args)
  File "/home/etrochim/.local/lib/python3.4/site-packages/cfnlint/config.py", line 366, in __init__
    ConfigFileArgs.__init__(self)
  File "/home/etrochim/.local/lib/python3.4/site-packages/cfnlint/config.py", line 68, in __init__
    self.load()
  File "/home/etrochim/.local/lib/python3.4/site-packages/cfnlint/config.py", line 120, in load
    user_config, project_config = self._find_config()
  File "/home/etrochim/.local/lib/python3.4/site-packages/cfnlint/config.py", line 82, in _find_config
    self.__user_config_file = Path.home().joinpath(config_file_name)
AttributeError: type object 'Path' has no attribute 'home'
```



Discussion:

Resolving patch:
diff --git a/src/cfnlint/config.py b/src/cfnlint/config.py
--- a/src/cfnlint/config.py
+++ b/src/cfnlint/config.py
@@ -19,8 +19,9 @@
 import logging
 import glob
 import json
-import yaml
+import os
 import six
+import yaml
 import jsonschema
 from cfnlint.version import __version__
 try:  # pragma: no cover
@@ -79,7 +80,10 @@ def _find_config(self):
             > user_config, project_config = self._find_config()
         """
         config_file_name = '.cfnlintrc'
-        self.__user_config_file = Path.home().joinpath(config_file_name)
+        if six.PY34:
+            self.__user_config_file = Path(os.path.expanduser('~')).joinpath(config_file_name)
+        else:
+            self.__user_config_file = Path.home().joinpath(config_file_name)
         self.__project_config_file = Path.cwd().joinpath(config_file_name)
 
         user_config_path = ''


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
Does not lint deeply nested template paths
*cfn-lint version: 0.9.2*

Using `**` path globs does not behave as expected. Assume the following directory structure

```yaml
project:
  templates:
    - a:
      - b:
        - c:
          - foo.yaml
          - bar.yaml
        - d:
          - baz.yaml
          - quux.yaml
      - e:
        - ipsum.yaml
        - lorem.yaml
    - f:
      - zyx.yaml
      - quuz.yaml
    - main.yaml
```

If I use the glob format in `.cfnlintrc`:

```yaml
templates:
- project/templates/**/*.yaml
```

The only files that are checked are `project/templates/f/zyx.yaml` and `project/templates/f/quux.yaml`. All others are ignored. I would expect `project/templates/**/*yaml` to match all deeply nested templates as well as `project/templates/main.yaml`.



Discussion:
The easy fix for Python 3.5 or higher is to add `, recursive=True` to https://github.com/awslabs/cfn-python-lint/blob/master/src/cfnlint/config.py#L439 

I'm debating punting on the backwards compatibility of this and just saying we support what glob can do based on the version of Python available.  

Resolving patch:
diff --git a/src/cfnlint/config.py b/src/cfnlint/config.py
--- a/src/cfnlint/config.py
+++ b/src/cfnlint/config.py
@@ -436,7 +436,11 @@ def templates(self):
         # some shells don't expand * and configparser won't expand wildcards
         all_filenames = []
         for filename in filenames:
-            add_filenames = glob.glob(filename)
+            if sys.version_info >= (3, 5):
+                # pylint: disable=E1123
+                add_filenames = glob.glob(filename, recursive=True)
+            else:
+                add_filenames = glob.glob(filename)
             # only way to know of the glob failed is to test it
             # then add the filename as requested
             if not add_filenames:


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
E3020 - ACM Validation - Linter error
*cfn-lint version: 0.11.1*

Hello!

So I have the below snippet of code:
```
  r7XXXZZb04efXXXYYYZZ2b8cc0b2aa1f132XXXYYYZZZCNAMERecordSet:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneId: !Ref rHostedZone
      Comment: domain CNAME record.
      Name: !Join ["", ["_75XXXTTT04efa8c2052b8ccXXXYYZZZ.XXXYYYZZZ.", !Ref pDomainName, "."]]
      Type: CNAME
      TTL: '300'
      ResourceRecords:
        - _14062XXXXYYY6b1f37eb361e9XXXYYYZZZ.acm-validations.aws
```

Unfortunately linter reports it as a E3020 error.

```
E3020 CNAME record (_14062XXXXYYY6b1f37eb361e9XXXYYYZZZ.acm-validations.aws) does not contain a valid domain name
dns/dns-production.template:645:7
```

Is that a proper behaviour? Resource is created properly - and the record works as it should - why then linter complains about it?



Discussion:

Resolving patch:
diff --git a/src/cfnlint/rules/resources/route53/RecordSet.py b/src/cfnlint/rules/resources/route53/RecordSet.py
--- a/src/cfnlint/rules/resources/route53/RecordSet.py
+++ b/src/cfnlint/rules/resources/route53/RecordSet.py
@@ -30,6 +30,7 @@ class RecordSet(CloudFormationLintRule):
 
     REGEX_DOMAINNAME = re.compile(r'^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])(.)$')
     REGEX_TXT = re.compile(r'^("[^"]{1,255}" *)*"[^"]{1,255}"$')
+    REGEX_CNAME_VALIDATIONS = re.compile(r'^.*\.acm-validations\.aws\.?$')
 
     def check_a_record(self, path, recordset):
         """Check A record Configuration"""
@@ -116,10 +117,10 @@ def check_cname_record(self, path, recordset):
             for index, record in enumerate(resource_records):
                 if not isinstance(record, dict):
                     tree = path[:] + ['ResourceRecords', index]
-                    if (not re.match(self.REGEX_DOMAINNAME, record)
-                            # ACM Route 53 validation uses invalid CNAMEs starting with `_`,
-                            # special-case them rather than complicate the regex.
-                            and not record.endswith('.acm-validations.aws.')):
+                    if (not re.match(self.REGEX_DOMAINNAME, record) and
+                            not re.match(self.REGEX_CNAME_VALIDATIONS, record)):
+                        # ACM Route 53 validation uses invalid CNAMEs starting with `_`,
+                        # special-case them rather than complicate the regex.
                         message = 'CNAME record ({}) does not contain a valid domain name'
                         matches.append(RuleMatch(tree, message.format(record)))
 


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
cfn-lint throwing E3002 instead of W3002 on Lambda::LayerVersion
*cfn-lint version: `0.12.1`*

*Description of issue.*

cfn-lint is throwing `E3002 Expecting an object at Resources/LambdaLayerVersion/Properties/Content` when linting a template where a path is passed for a `AWS::Lambda::LayerVersion`'s `Content` similarly to how we would do that for a `AWS::Lambda::Function`'s `Code`. 

This is intended to work with the `aws cloudformation package` command which now supports `AWS::Lambda::LayerVersion`. 

Please provide as much information as possible:
* Template linting issues:
  * Please provide a CloudFormation sample that generated the issue.
```yaml
AWSTemplateFormatVersion: 2010-09-09

Resources:

  LambdaLayerVersion:
    Type: AWS::Lambda::LayerVersion
    Properties:
      Description: A layer version which will contain static content
      CompatibleRuntimes:
        - python3.6
        - python3.7
      Content: ../static-content
```
  * If present, please add links to the (official) documentation for clarification.

    The documentation doesn't mention `aws cloudformation package` support for `AWS::Lambda::LayerVersion` yet, but support was added in aws-cli version [1.16.93](https://github.com/aws/aws-cli/blob/1.16.93/awscli/customizations/cloudformation/artifact_exporter.py#L373)

* Feature request:
  * Please provide argumentation about the missing feature. Context is key!

We're packaging static files in lambda layers and would like for cfn-lint to provide the expected behavior of throwing a warning like `W3002 This code may only work with `package` cli command as the property (Resources/LambdaLayerVersion/Properties/Content) is a string`



Discussion:

Resolving patch:
diff --git a/src/cfnlint/rules/resources/properties/Properties.py b/src/cfnlint/rules/resources/properties/Properties.py
--- a/src/cfnlint/rules/resources/properties/Properties.py
+++ b/src/cfnlint/rules/resources/properties/Properties.py
@@ -154,6 +154,7 @@ def check_exceptions(self, parenttype, proptype, text):
         templated_exceptions = {
             'AWS::ApiGateway::RestApi': ['BodyS3Location'],
             'AWS::Lambda::Function': ['Code'],
+            'AWS::Lambda::LayerVersion': ['Content'],
             'AWS::ElasticBeanstalk::ApplicationVersion': ['SourceBundle'],
         }
 
diff --git a/src/cfnlint/rules/resources/properties/PropertiesTemplated.py b/src/cfnlint/rules/resources/properties/PropertiesTemplated.py
--- a/src/cfnlint/rules/resources/properties/PropertiesTemplated.py
+++ b/src/cfnlint/rules/resources/properties/PropertiesTemplated.py
@@ -53,6 +53,7 @@ def match_resource_properties(self, properties, resourcetype, path, cfn):
         templated_exceptions = {
             'AWS::ApiGateway::RestApi': ['BodyS3Location'],
             'AWS::Lambda::Function': ['Code'],
+            'AWS::Lambda::LayerVersion': ['Content'],
             'AWS::ElasticBeanstalk::ApplicationVersion': ['SourceBundle'],
         }
 


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
!GetAtt reference lint error
Hello,

Getting validation error on `AWS::ElasticLoadBalancingV2::Listener` property `LoadBalancerArn` when the `LoadBalancerArn` is referenced using !GetAtt nested-stack-name.Outputs.LoadbalancerArn

`[cfn-lint] E3008:CloudFormation stack outputs need to be strings not lists at Resources/ApiGwNlbListener/Properties/LoadBalancerArn/Fn::GetAtt
`


Discussion:

Resolving patch:
diff --git a/src/cfnlint/rules/resources/properties/ValueRefGetAtt.py b/src/cfnlint/rules/resources/properties/ValueRefGetAtt.py
--- a/src/cfnlint/rules/resources/properties/ValueRefGetAtt.py
+++ b/src/cfnlint/rules/resources/properties/ValueRefGetAtt.py
@@ -140,8 +140,7 @@ def check_value_getatt(self, value, path, **kwargs):
             # Nested Stack Outputs
             # if its a string type we are good and return matches
             # if its a list its a failure as Outputs can only be strings
-
-            if is_value_a_list:
+            if is_value_a_list and property_type == 'List':
                 message = 'CloudFormation stack outputs need to be strings not lists at {0}'
                 matches.append(RuleMatch(path, message.format('/'.join(map(str, path)))))
 


Test patch:
diff --git a/test/fixtures/templates/good/resources/properties/value.yaml b/test/fixtures/templates/good/resources/properties/value.yaml
--- a/test/fixtures/templates/good/resources/properties/value.yaml
+++ b/test/fixtures/templates/good/resources/properties/value.yaml
@@ -125,3 +125,7 @@ Resources:
         Value: '50'
       SecurityGroups:
       - !GetAtt SubStack.Outputs.SecurityGroups
+  Listener:
+    Type: AWS::ElasticLoadBalancingV2::Listener
+    Properties:
+      LoadBalancerArn: !GetAtt SubStack.Outputs.LoadBalancerArn
Repository: aws-cloudformation/cfn-lint

Issue:
E2504 enforces incorrect boundary 100-2000, should be 100-20000
*cfn-lint version: (`cfn-lint --version`)* 0.15.0 (also tested with 0.14.1)

**Description:**
AWS EC2 Instance with block device mapping does not enforce the correct `Ebs/Iops` boundary of 100-20000 as specified in the [documentation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-blockdev-template.html)

It looks like it actually enforces 100-2000, as if you set `Iops` to 2000 it passes the linter.

**Sample to reproduce:**
```yaml
Resources:
  Machine:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: "ami-79fd7eee"
      KeyName: "testkey"
      BlockDeviceMappings: 
        - DeviceName: /dev/sdm
          Ebs: 
            VolumeType: io1
            Iops: 3000 
            DeleteOnTermination: false
            VolumeSize: 20
```

Output:
```bash
> cfn-lint minimal-stack.yaml

E2504 Property Iops should be Int between 100 to 20000 Resources/Machine/Properties/BlockDeviceMappings/0/Ebs/Iops
minimal-stack.yaml:11:13
```



Discussion:
See line 53 for the issue https://github.com/awslabs/cfn-python-lint/blob/f5c3504f593d4311388a047fa4b6f18f792c919a/src/cfnlint/rules/resources/ectwo/Ebs.py#L50-L55

Resolving patch:
diff --git a/src/cfnlint/rules/resources/ectwo/Ebs.py b/src/cfnlint/rules/resources/ectwo/Ebs.py
--- a/src/cfnlint/rules/resources/ectwo/Ebs.py
+++ b/src/cfnlint/rules/resources/ectwo/Ebs.py
@@ -50,7 +50,7 @@ def _checkEbs(self, cfn, ebs, path):
                                         iops = iops_obj[0]['Value']
                                         if isinstance(iops, (six.string_types, int)) and not iops_obj[0]['Path']:
                                             iops_value = int(iops)
-                                            if iops_value < 100 or iops_value > 2000:
+                                            if iops_value < 100 or iops_value > 20000:
                                                 pathmessage = path[:] + ['Iops']
                                                 message = 'Property Iops should be Int between 100 to 20000 {0}'
                                                 matches.append(


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
False positive E1029 with EMR autoscaling policies
*cfn-lint version: 0.16.0*

E1029 is raised for the following valid template:

    AWSTemplateFormatVersion: '2010-09-09'
    Description: Launch an EMR cluster
    Parameters:
      EC2KeyName:
        Type: AWS::EC2::KeyPair::KeyName
    
    Resources:
      EMRCluster:
        Type: "AWS::EMR::Cluster"
        Properties:
          Applications:
            - Name: Hadoop
          AutoScalingRole: EMR_AutoScaling_DefaultRole
          JobFlowRole: EMR_EC2_DefaultRole
          ServiceRole: EMR_DefaultRole
          Name: Test-E1029
          ReleaseLabel: emr-5.21.0
          VisibleToAllUsers: true
          Instances:
            Ec2KeyName: !Ref EC2KeyName
            MasterInstanceGroup:
              InstanceCount: 1
              InstanceType: m4.large
              Name: Master - 1
            CoreInstanceGroup:
              Name: Core - 2
              InstanceCount: 3
              InstanceType: m4.large
              AutoScalingPolicy:
                Constraints:
                  MaxCapacity: 10
                  MinCapacity: 3
                Rules:
                  - Action:
                      SimpleScalingPolicyConfiguration:
                        AdjustmentType: CHANGE_IN_CAPACITY
                        CoolDown: 300
                        ScalingAdjustment: 1
                    Description: ''
                    Name: Default-scale-out-1
                    Trigger:
                      CloudWatchAlarmDefinition:
                        ComparisonOperator: LESS_THAN
                        Dimensions:
                          - Key: JobFlowId
                            Value: ${emr.clusterId}
                        EvaluationPeriods: 1
                        MetricName: YARNMemoryAvailablePercentage
                        Namespace: AWS/ElasticMapReduce
                        Period: 300
                        Statistic: AVERAGE
                        Threshold: 15
                        Unit: PERCENT

The `${emr.clusterId}` syntax is normally interpreted by EMR when building out the cluster.

Note: if you want to try this template, be sure to run `aws emr create-default-roles` first.


Discussion:

Resolving patch:
diff --git a/src/cfnlint/rules/functions/SubNeeded.py b/src/cfnlint/rules/functions/SubNeeded.py
--- a/src/cfnlint/rules/functions/SubNeeded.py
+++ b/src/cfnlint/rules/functions/SubNeeded.py
@@ -28,7 +28,7 @@ class SubNeeded(CloudFormationLintRule):
 
     # Free-form text properties to exclude from this rule
     # content is part of AWS::CloudFormation::Init
-    excludes = ['UserData', 'ZipFile', 'Condition', 'AWS::CloudFormation::Init']
+    excludes = ['UserData', 'ZipFile', 'Condition', 'AWS::CloudFormation::Init', 'CloudWatchAlarmDefinition']
     api_excludes = ['Uri', 'Body']
 
     # IAM Policy has special variables that don't require !Sub, Check for these


Test patch: