internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: aws-cloudformation/cfn-lint

Issue:
E0000 found unknown escape character ‘/’
version:1:1

cfn-lint --template vpc.cf.json
E0000 found unknown escape character ‘/’
vpc.cf.json:12:135

this is the string that it says container the escape character error. this however works fine when deployed to the CFN service. 

"^([0-9]{1,3}\\.){3}[0-9]{1,3}(\\\/([0-9]|[1-2][0-9]|3[0-2]))?$"

![image](https://user-images.githubusercontent.com/42137702/54364854-a11a3000-4665-11e9-8454-9ab8033fc1e7.png)



Discussion:

Resolving patch:
diff --git a/src/cfnlint/decode/__init__.py b/src/cfnlint/decode/__init__.py
--- a/src/cfnlint/decode/__init__.py
+++ b/src/cfnlint/decode/__init__.py
@@ -57,11 +57,12 @@ def decode(filename, ignore_bad_template):
     except cfnlint.decode.cfn_yaml.CfnParseError as err:
         err.match.Filename = filename
         matches = [err.match]
-
     except ParserError as err:
         matches = [create_match_yaml_parser_error(err, filename)]
     except ScannerError as err:
-        if err.problem == 'found character \'\\t\' that cannot start any token':
+        if err.problem in [
+                'found character \'\\t\' that cannot start any token',
+                'found unknown escape character']:
             try:
                 template = cfnlint.decode.cfn_json.load(filename)
             except cfnlint.decode.cfn_json.JSONDecodeError as json_err:


Test patch:
diff --git a/test/fixtures/templates/good/decode/parsing.json b/test/fixtures/templates/good/decode/parsing.json
new file mode 100644
--- /dev/null
+++ b/test/fixtures/templates/good/decode/parsing.json
@@ -0,0 +1,9 @@
+{
+  "Parameters": {
+    "VPN": {
+      "Type": "String",
+      "AllowedPattern": "^\\\/"
+    }
+  },
+  "Resources": {}
+}
diff --git a/test/module/cfn_json/test_cfn_json.py b/test/module/cfn_json/test_cfn_json.py
--- a/test/module/cfn_json/test_cfn_json.py
+++ b/test/module/cfn_json/test_cfn_json.py
@@ -73,6 +73,17 @@ def test_success_parse(self):
             matches.extend(self.rules.run(filename, cfn))
             assert len(matches) == failures, 'Expected {} failures, got {} on {}'.format(failures, len(matches), filename)
 
+    def test_success_escape_character(self):
+        """Test Successful JSON Parsing"""
+        failures = 1
+        filename = 'test/fixtures/templates/good/decode/parsing.json'
+        template = cfnlint.decode.cfn_json.load(filename)
+        cfn = Template(filename, template, ['us-east-1'])
+
+        matches = []
+        matches.extend(self.rules.run(filename, cfn))
+        assert len(matches) == failures, 'Expected {} failures, got {} on {}'.format(failures, len(matches), filename)
+
     def test_success_parse_stdin(self):
         """Test Successful JSON Parsing through stdin"""
         for _, values in self.filenames.items():
Repository: aws-cloudformation/cfn-lint

Issue:
E0002 Unknown exception while processing rule E3012: 'int' object has no attribute 'start_mark'
**cfn-lint version:**
0.17.0

**Exception:**
E0002 Unknown exception while processing rule E3012: 'int' object has no attribute 'start_mark'
template.yml:1:1

**Example template:**
```yaml
AWSTemplateFormatVersion: 2010-09-09
Resources:
  emrStep:
    Type: 'AWS::EMR::Step'
    Properties:
      ActionOnFailure: CANCEL_AND_WAIT
      HadoopJarStep:
        Args:
          - string
          - 2
          - otherstring
        Jar: command-runner.jar
      JobFlowId: 'foo'
      Name: 'bar'
```

**Explanation:**
All items in the list for Resources.emrStep.Properties.HadoopJarStep.Args should be of type string per https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-emr-step-hadoopjarstepconfig.html, when a non-string value is provided cfn-lint generates an exception instead of noting the error.

Changing the args to the correct format by removing the `2` or putting quotes around it resolves the exception.

The (actual) template deploys successfully even though it has some ints instead of a strings in the args like the example does


Discussion:

Resolving patch:
diff --git a/src/cfnlint/__init__.py b/src/cfnlint/__init__.py
--- a/src/cfnlint/__init__.py
+++ b/src/cfnlint/__init__.py
@@ -824,7 +824,10 @@ def get_location_yaml(self, text, path):
             # If the last item of the path is an integer, and the vaue is an array,
             # Get the location of the item in the array
             if isinstance(text, list) and isinstance(path[0], int):
-                result = self._loc(text[path[0]])
+                try:
+                    result = self._loc(text[path[0]])
+                except AttributeError as err:
+                    LOGGER.debug(err)
             else:
                 try:
                     for key in text:


Test patch:
diff --git a/test/module/cfn_json/test_cfn_json.py b/test/module/cfn_json/test_cfn_json.py
--- a/test/module/cfn_json/test_cfn_json.py
+++ b/test/module/cfn_json/test_cfn_json.py
@@ -48,7 +48,7 @@ def setUp(self):
             },
             "vpc_management": {
                 "filename": 'test/fixtures/templates/quickstart/vpc-management.json',
-                "failures": 33
+                "failures": 35
             },
             "vpc": {
                 "filename": 'test/fixtures/templates/quickstart/vpc.json',
Repository: aws-cloudformation/cfn-lint

Issue:
E2532 when Step Functions Pass state has Parameters key
*cfn-lint version: 0.17.0*

Using latest version of cfn-lint and updated spec files.

*Description of issue.*

Linting a CF template fails when a `AWS::StepFunctions::StateMachine`'s `DefinitionString` has a `Pass` state with the `Parameters` key.

Example template:

```yaml
AWSTemplateFormatVersion: 2010-09-09
Resources:
    ExampleStateMachine:
        Type: AWS::StepFunctions::StateMachine
        Properties:
            DefinitionString: |-
                {
                    "StartAt": "Pop Element from List",
                    "States": {
                        "Pop Element from List": {
                            "Type": "Pass",
                            "Parameters": {
                                "List.$": "$.List[1:]"
                            },
                            "End": true
                        }
                    }
                }
            RoleArn: redacted
```

Error:

```
E2532 State Machine Definition key (Parameters) for State (Pop Element from List) of Type (Pass) is not valid
example.yaml:6:13
```

However, the above `Pass` state conforms to the ASL [spec](https://states-language.net/spec.html#state-type-table) and [docs](https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-pass-state.html) and comes directly from the [Transfer Data Records sample project](https://docs.aws.amazon.com/step-functions/latest/dg/sample-project-transfer-data-sqs.html).


Discussion:
Can you add the error you get when you try to deploy the template? This makes it more understandable and is a double check to make sure our rule is off, and not the documentation. 
(I assume the docs and the sample project is right, but I've seen code samples from AWS that "invalid" 😉 )
The template deploys successfully (after substituting a real role ARN, of course).

Resolving patch:
diff --git a/src/cfnlint/rules/resources/stepfunctions/StateMachine.py b/src/cfnlint/rules/resources/stepfunctions/StateMachine.py
--- a/src/cfnlint/rules/resources/stepfunctions/StateMachine.py
+++ b/src/cfnlint/rules/resources/stepfunctions/StateMachine.py
@@ -51,7 +51,7 @@ def _check_state_json(self, def_json, state_name, path):
             'Type',
         ]
         state_key_types = {
-            'Pass': ['Result', 'ResultPath'],
+            'Pass': ['Result', 'ResultPath', 'Parameters'],
             'Task': ['Resource', 'ResultPath', 'Retry', 'Catch', 'TimeoutSeconds', 'Parameters', 'HeartbeatSeconds'],
             'Choice': ['Choices', 'Default'],
             'Wait': ['Seconds', 'Timestamp', 'SecondsPath', 'TimestampPath'],


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
Double in mapping thrown E7001 error
*cfn-lint version: cfn-lint 0.20.1*

*Description of issue.*
When a mapping value is a double (ex. 1.1) it returns the error `E7001:Mapping [map] has invalid property at [property]`

Examples:
With double value:
![image](https://user-images.githubusercontent.com/2495780/57731384-887bc580-7668-11e9-998c-f983a1029716.png)

Changed to Int:
![image](https://user-images.githubusercontent.com/2495780/57731469-b19c5600-7668-11e9-9521-20eadd2885a4.png)

Example CFT: [environment.yaml.txt](https://github.com/aws-cloudformation/cfn-python-lint/files/3179852/environment.yaml.txt)



Discussion:
Thanks. I’ll take a look

Resolving patch:
diff --git a/src/cfnlint/rules/mappings/Configuration.py b/src/cfnlint/rules/mappings/Configuration.py
--- a/src/cfnlint/rules/mappings/Configuration.py
+++ b/src/cfnlint/rules/mappings/Configuration.py
@@ -32,6 +32,8 @@ def match(self, cfn):
 
         matches = []
 
+        valid_map_types = (six.string_types, list, six.integer_types, float)
+
         mappings = cfn.template.get('Mappings', {})
         if mappings:
             for mapname, mapobj in mappings.items():
@@ -53,8 +55,7 @@ def match(self, cfn):
                         else:
                             for secondkey in firstkeyobj:
                                 if not isinstance(
-                                        firstkeyobj[secondkey],
-                                        (six.string_types, list, six.integer_types)):
+                                        firstkeyobj[secondkey], valid_map_types):
                                     message = 'Mapping {0} has invalid property at {1}'
                                     matches.append(RuleMatch(
                                         ['Mappings', mapname, firstkey, secondkey],


Test patch:
diff --git a/test/fixtures/templates/good/mappings/configuration.yaml b/test/fixtures/templates/good/mappings/configuration.yaml
--- a/test/fixtures/templates/good/mappings/configuration.yaml
+++ b/test/fixtures/templates/good/mappings/configuration.yaml
@@ -13,4 +13,9 @@ Mappings:
         - ap-northeast-1a
         - ap-northeast-1b
         - ap-northeast-1c
+  DoubleMap:
+    us-east-1:
+      MinCapacity: 1
+      MaxCapacity: 2
+      TargetValue: 15.0
 Resources: {}
Repository: aws-cloudformation/cfn-lint

Issue:
pkg_resources (setuptools) requirement not declared in setup.py
*cfn-lint version:* 0.20.1

*Description of issue.*
While trying to package cfn-lint for conda-forge, I ran into the issue that pkg_resources is [imported in a few places](https://github.com/aws-cloudformation/cfn-python-lint/search?q=pkg_resources&unscoped_q=pkg_resources) but that this requirement (setuptools) is not specified in setup.py https://github.com/aws-cloudformation/cfn-python-lint/blob/master/setup.py#L75-L82

Is setuptools desired to be a run time requirement? If so, install_requires should probably list it. 


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -79,7 +79,8 @@ def get_version(filename):
         'aws-sam-translator>=1.10.0',
         'jsonpatch',
         'jsonschema~=2.6',
-        'pathlib2>=2.3.0;python_version<"3.4"'
+        'pathlib2>=2.3.0;python_version<"3.4"',
+        'setuptools',
     ],
     python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*',
     entry_points={


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
Parameter parsing doesn't match help text
$ cfn-lint --version
cfn-lint 0.13.1

The help documentation suggests that passing the template as the last parameter without a switch should be valid. However, it doesn't work correctly if you pass an option in the middle. For instance,

```
$ cfn-lint -i E3012 template.yml
```

Generates help output.

However, this works even though it doesn't seem to conform to the specified usage,
```
$ cfn-lint template.yml -i E3012
```

Current usage output is as follows:
```
$ cfn-lint -h
usage: cfn-lint [-h] [-t TEMPLATE [TEMPLATE ...]] [-b] [-d]
                [-f {quiet,parseable,json}] [-l] [-r REGIONS [REGIONS ...]]
                [-a APPEND_RULES [APPEND_RULES ...]]
                [-i IGNORE_CHECKS [IGNORE_CHECKS ...]]
                [-c INCLUDE_CHECKS [INCLUDE_CHECKS ...]] [-o OVERRIDE_SPEC]
                [-v] [-u]
                [TEMPLATE [TEMPLATE ...]]
```


Discussion:
Yeah, that doesn't match up. I tend to put config at the end so I hadn't noticed. I'll flag this as a bug so we can dig into it. Thanks!
Yea, this one may be tricky.  It has a little to do with  `-i` will take many rules and so then it believes `template.yaml` is an `i` based parameter.  Another option is `$ cfn-lint -i E3012 -- template.yml`

Resolving patch:
diff --git a/src/cfnlint/config.py b/src/cfnlint/config.py
--- a/src/cfnlint/config.py
+++ b/src/cfnlint/config.py
@@ -305,7 +305,16 @@ def __call__(self, parser, namespace, values, option_string=None):
                         items.append(value)
                 setattr(namespace, self.dest, items)
 
-        parser = ArgumentParser(description='CloudFormation Linter')
+        usage = (
+            '\nBasic: cfn-lint test.yaml\n'
+            'Ignore a rule: cfn-lint -I E3012 -- test.yaml\n'
+            'Configure a rule: cfn-lint -x E3012:strict=false -t test.yaml\n'
+            'Lint all yaml files in a folder: cfn-lint dir/**/*.yaml'
+        )
+
+        parser = ArgumentParser(
+            description='CloudFormation Linter',
+            usage=usage)
         parser.register('action', 'extend', ExtendAction)
 
         standard = parser.add_argument_group('Standard')


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
Outside of an Sub error for arn:aws:s3:::${transfer:HomeBucket}
*cfn-lint version:  0.21.5*

*Description of issue.*
When defining a scope down policy for AWS SFTP transfer, cfn-lint gives error **E1029 Found an embedded parameter outside of an "Fn::Sub"** on things like `arn:aws:s3:::${transfer:HomeBucket}`. Is there anyway to exclude those?

```yaml
  SftpScopeDownPolicy:
    Type: AWS::IAM::ManagedPolicy
    Properties:
      Description: Policy for restricting SFTP users to have access to only their home directory
      ManagedPolicyName: sftp-scope-down-policy
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          -
            Sid: AllowListingOfUserFolder
            Effect: Allow
            Action:
              - s3:ListBucket
            Resource: arn:aws:s3:::${transfer:HomeBucket}
            Condition:
              StringLike:
                s3:prefix:
                  - '${transfer:HomeFolder}'
                  - '${transfer:HomeFolder}/*'
          -
            Sid: HomeDirectoryObjectAccess
            Effect: Allow
            Action:
              - s3:GetObject
              - s3:GetObjectVersion
              - s3:PutObject
              - s3:DeleteObject
              - s3:DeleteObjectVersion
              - s3:GetObjectACL
            Resource: arn:aws:s3:::${transfer:HomeDirectory}*
```



Discussion:

Resolving patch:
diff --git a/src/cfnlint/rules/functions/SubNeeded.py b/src/cfnlint/rules/functions/SubNeeded.py
--- a/src/cfnlint/rules/functions/SubNeeded.py
+++ b/src/cfnlint/rules/functions/SubNeeded.py
@@ -35,11 +35,13 @@ class SubNeeded(CloudFormationLintRule):
     # https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_variables.html
     # https://docs.aws.amazon.com/iot/latest/developerguide/basic-policy-variables.html
     # https://docs.aws.amazon.com/iot/latest/developerguide/thing-policy-variables.html
+    # https://docs.aws.amazon.com/transfer/latest/userguide/users.html#users-policies-scope-down
     resource_excludes = ['${aws:CurrentTime}', '${aws:EpochTime}', '${aws:TokenIssueTime}', '${aws:principaltype}',
                          '${aws:SecureTransport}', '${aws:SourceIp}', '${aws:UserAgent}', '${aws:userid}',
                          '${aws:username}', '${ec2:SourceInstanceARN}',
                          '${iot:Connection.Thing.ThingName}', '${iot:Connection.Thing.ThingTypeName}',
-                         '${iot:Connection.Thing.IsAttached}', '${iot:ClientId}']
+                         '${iot:Connection.Thing.IsAttached}', '${iot:ClientId}', '${transfer:HomeBucket}',
+                         '${transfer:HomeDirectory}', '${transfer:HomeFolder}', '${transfer:UserName}']
 
     def _match_values(self, searchRegex, cfnelem, path):
         """Recursively search for values matching the searchRegex"""


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
cfn-lint is failing because of `pkg_resources.ContextualVersionConflict: (jsonschema 2.6.0)`. 
*cfn-lint version: (`0.21.6`)*

*Description of issue.*
cfn-lint(python2) requires jsonschema 2.6.0 but aws-sam-translator which got released today requires jsonschema3.0

https://pypi.org/project/aws-sam-translator/#history
pkg_resources.ContextualVersionConflict: (jsonschema 2.6.0 (/usr/lib/python2.7/site-packages), Requirement.parse('jsonschema~=3.0'), set(['aws-sam-translator']))


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -78,7 +78,7 @@ def get_version(filename):
         'requests>=2.15.0',
         'aws-sam-translator>=1.10.0',
         'jsonpatch',
-        'jsonschema~=2.6',
+        'jsonschema~=3.0',
         'pathlib2>=2.3.0;python_version<"3.4"',
         'setuptools',
     ],


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
Follow up to absolute resource paths
https://github.com/aws-cloudformation/cfn-python-lint/issues/691

This missed a location at rules/resources/IAM/Permissions.py line 59. (Not allowed to open a PR at work)

Thanks!


Discussion:

Resolving patch:
diff --git a/src/cfnlint/rules/resources/iam/Permissions.py b/src/cfnlint/rules/resources/iam/Permissions.py
--- a/src/cfnlint/rules/resources/iam/Permissions.py
+++ b/src/cfnlint/rules/resources/iam/Permissions.py
@@ -56,7 +56,7 @@ def load_service_map(self):
         """
         Convert policies.json into a simpler version for more efficient key lookup.
         """
-        service_map = cfnlint.helpers.load_resources('/data/AdditionalSpecs/Policies.json')['serviceMap']
+        service_map = cfnlint.helpers.load_resources('data/AdditionalSpecs/Policies.json')['serviceMap']
 
         policy_service_map = {}
 


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
NoValue support in NaclEntry/icmp
*cfn-lint version: 0.22.3*

*Description of issue.*

Similar to https://github.com/aws-cloudformation/cfn-python-lint/issues/1019
In case of:
```
    "PermitAllInbound": {
      "Type": "AWS::EC2::NetworkAclEntry",
      "Properties": {
        ...
        "Icmp": {
          "Ref": "AWS::NoValue"
        },
```
I get result:
```
E3002 Invalid Property Resources/PermitAllInbound/Properties/Icmp/Ref
```
Icmp property is not required, so this should be accepted.


Discussion:

Resolving patch:
diff --git a/src/cfnlint/rules/resources/properties/Properties.py b/src/cfnlint/rules/resources/properties/Properties.py
--- a/src/cfnlint/rules/resources/properties/Properties.py
+++ b/src/cfnlint/rules/resources/properties/Properties.py
@@ -239,6 +239,8 @@ def propertycheck(self, text, proptype, parenttype, resourcename, path, root):
                             proppath + cond_value['Path'], root))
                 elif text.is_function_returning_object():
                     self.logger.debug('Ran into function "%s".  Skipping remaining checks', prop)
+                elif len(text) == 1 and prop in 'Ref' and text.get(prop) == 'AWS::NoValue':
+                    pass
                 elif not supports_additional_properties:
                     message = 'Invalid Property %s' % ('/'.join(map(str, proppath)))
                     matches.append(RuleMatch(proppath, message))


Test patch:
diff --git a/test/fixtures/templates/bad/generic.yaml b/test/fixtures/templates/bad/generic.yaml
--- a/test/fixtures/templates/bad/generic.yaml
+++ b/test/fixtures/templates/bad/generic.yaml
@@ -200,6 +200,15 @@ Resources:
       GroupDescription: test
       SecurityGroupIngress:
       - Fn::FindInMap: [ runtime, !Ref 'AWS::Region', production ]
+  PermitAllInbound:
+    Type: AWS::EC2::NetworkAclEntry
+    Properties:
+      CidrBlock: 10.0.0.0/16
+      Icmp: !Ref AWS::NoValue
+      NetworkAclId: '1'
+      RuleNumber: 1
+      Protocol: 1
+      RuleAction: allow
 Outputs:
   myOutput:
     Value: !GetAtt ElasticLoadBalancer.CanonicalHostedZoneName
Repository: aws-cloudformation/cfn-lint

Issue:
Don't fail rules when running into a Fn::Transform
*cfn-lint version: (`cfn-lint --version`)*
v0.22.3
*Description of issue.*
Certain rules are failing when running into a Fn::Transform
![image](https://user-images.githubusercontent.com/36457093/61500908-b0eaad80-a992-11e9-9b9d-7577e507af81.png)




Discussion:
@iDVB going to create a separate issue for this.
Thanks @kddejong,
Can you give a bit more detail as what you mean by 
> “Ideally we would be linting the resulting template from Transforms”

This really is more ideal than actuality, no? Transforms happen on AWS in CFN no? Just curious if I’m missing a step where linting could happen after transform.
@iDVB So for linting the difficult part is that transform can change an "invalid" template to "valid" template.  You are correct it happens on the AWS side.

Example: Resources in this case is a list which isn't standard but the transform makes it valid. https://github.com/awslabs/aws-cloudformation-templates/blob/master/aws/services/CloudFormation/MacrosExamples/ShortHand/example.template

You are using a Transform at a level that we shouldn't have errors but having a larger solution to Transforms is going to take some work.  Ideally at the end of any and all transforms the resulting template is valid CloudFormation template.

Resolving patch:
diff --git a/src/cfnlint/rules/resources/properties/Properties.py b/src/cfnlint/rules/resources/properties/Properties.py
--- a/src/cfnlint/rules/resources/properties/Properties.py
+++ b/src/cfnlint/rules/resources/properties/Properties.py
@@ -68,7 +68,7 @@ def primitivetypecheck(self, value, primtype, proppath):
                                     sub_value[2], primtype, proppath + ['Fn::If', 2]))
                     elif sub_key not in ['Fn::Base64', 'Fn::GetAtt', 'Fn::GetAZs', 'Fn::ImportValue',
                                          'Fn::Join', 'Fn::Split', 'Fn::FindInMap', 'Fn::Select', 'Ref',
-                                         'Fn::If', 'Fn::Contains', 'Fn::Sub', 'Fn::Cidr']:
+                                         'Fn::If', 'Fn::Contains', 'Fn::Sub', 'Fn::Cidr', 'Fn::Transform']:
                         message = 'Property %s has an illegal function %s' % ('/'.join(map(str, proppath)), sub_key)
                         matches.append(RuleMatch(proppath, message))
             else:
diff --git a/src/cfnlint/rules/resources/properties/ValuePrimitiveType.py b/src/cfnlint/rules/resources/properties/ValuePrimitiveType.py
--- a/src/cfnlint/rules/resources/properties/ValuePrimitiveType.py
+++ b/src/cfnlint/rules/resources/properties/ValuePrimitiveType.py
@@ -140,7 +140,7 @@ def check_value(self, value, path, **kwargs):
         else:
             # some properties support primitive types and objects
             # skip in the case it could be an object and the value is a object
-            if item_type and isinstance(value, dict):
+            if (item_type or primitive_type) and isinstance(value, dict):
                 return matches
             matches.extend(self.check_primitive_type(value, primitive_type, path))
 


Test patch: