internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: aws-cloudformation/cfn-lint

Issue:
Stdin for templates is broken
As reported here https://github.com/awslabs/aws-cfn-lint-visual-studio-code/issues/47 cfn-lint has broken the ability to use stdin to provide templates. 

The result is the help documentation is provided



Discussion:

Resolving patch:
diff --git a/src/cfnlint/core.py b/src/cfnlint/core.py
--- a/src/cfnlint/core.py
+++ b/src/cfnlint/core.py
@@ -16,6 +16,7 @@
 """
 import logging
 import os
+import sys
 from jsonschema.exceptions import ValidationError
 from cfnlint import RulesCollection
 import cfnlint.config
@@ -139,6 +140,9 @@ def get_args_filenames(cli_args):
         print(rules)
         exit(0)
 
+    if not sys.stdin.isatty():
+        return(config, [None], formatter)
+
     if not config.templates:
         # Not specified, print the help
         config.parser.print_help()
diff --git a/src/cfnlint/rules/templates/LimitSize.py b/src/cfnlint/rules/templates/LimitSize.py
--- a/src/cfnlint/rules/templates/LimitSize.py
+++ b/src/cfnlint/rules/templates/LimitSize.py
@@ -41,10 +41,11 @@ def match(self, cfn):
         filename = cfn.filename
 
         # Only check if the file exists. The template could be passed in using stdIn
-        if Path(filename).is_file():
-            statinfo = os.stat(filename)
-            if statinfo.st_size > LIMITS['template']['body']:
-                message = 'The template file size ({0} bytes) exceeds the limit ({1} bytes)'
-                matches.append(RuleMatch(['Template'], message.format(statinfo.st_size, LIMITS['template']['body'])))
+        if filename:
+            if Path(filename).is_file():
+                statinfo = os.stat(filename)
+                if statinfo.st_size > LIMITS['template']['body']:
+                    message = 'The template file size ({0} bytes) exceeds the limit ({1} bytes)'
+                    matches.append(RuleMatch(['Template'], message.format(statinfo.st_size, LIMITS['template']['body'])))
 
         return matches


Test patch:
diff --git a/test/module/core/test_run_cli.py b/test/module/core/test_run_cli.py
--- a/test/module/core/test_run_cli.py
+++ b/test/module/core/test_run_cli.py
@@ -15,6 +15,7 @@
   SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 """
 import logging
+from six import StringIO
 import cfnlint.core  # pylint: disable=E0401
 from testlib.testcase import BaseTestCase
 from mock import patch, mock_open
@@ -81,6 +82,16 @@ def test_template_invalid_json_ignore(self):
 
         self.assertEqual(len(matches), 1)
 
+    def test_template_via_stdin(self):
+        """Test getting the template from stdin doesn't crash"""
+        filename = 'test/fixtures/templates/good/generic.yaml'
+        with open(filename, 'r') as fp:
+            file_content = fp.read()
+
+        with patch('sys.stdin', StringIO(file_content)):
+            (_, filenames, _) = cfnlint.core.get_args_filenames([])
+            assert filenames == [None]
+
     @patch('cfnlint.config.ConfigFileArgs._read_config', create=True)
     def test_template_config(self, yaml_mock):
         """Test template config"""
Repository: aws-cloudformation/cfn-lint

Issue:
String misinterpreted as an int results in error on E2015
```
cfn-lint --version
cfn-lint 0.19.1
```

*Description of issue.*
The following template
```
Parameters:
  CentralAccountId:
    Default: 112233445566
    MaxLength: 12
    MinLength: 12
    Type: String
```
result in the error:
```
E0002 Unknown exception while processing rule E2015: object of type 'int' has no len()
application-account-initial-setup.yaml:1:1
```

It is solved by putting quotes on the default value. However it is valid to not putting the quotes.


Discussion:

Resolving patch:
diff --git a/src/cfnlint/rules/parameters/Default.py b/src/cfnlint/rules/parameters/Default.py
--- a/src/cfnlint/rules/parameters/Default.py
+++ b/src/cfnlint/rules/parameters/Default.py
@@ -82,8 +82,9 @@ def check_min_length(self, allowed_value, min_length, path):
         """
         message = 'Default should have a length above or equal to MinLength'
 
+        value = allowed_value if isinstance(allowed_value, six.string_types) else str(allowed_value)
         if isinstance(min_length, six.integer_types):
-            if len(allowed_value) < min_length:
+            if len(value) < min_length:
                 return([RuleMatch(path, message)])
 
         return []
@@ -94,8 +95,9 @@ def check_max_length(self, allowed_value, max_length, path):
         """
         message = 'Default should have a length below or equal to MaxLength'
 
+        value = allowed_value if isinstance(allowed_value, six.string_types) else str(allowed_value)
         if isinstance(max_length, six.integer_types):
-            if len(allowed_value) > max_length:
+            if len(value) > max_length:
                 return([RuleMatch(path, message)])
 
         return []


Test patch:
diff --git a/test/fixtures/templates/good/parameters/default.yaml b/test/fixtures/templates/good/parameters/default.yaml
--- a/test/fixtures/templates/good/parameters/default.yaml
+++ b/test/fixtures/templates/good/parameters/default.yaml
@@ -34,4 +34,9 @@ Parameters:
     Description: Remote PostGreSQL database's master user account.
     Type: String
     Default: abcdef
+  CentralAccountId:
+    Default: 112233445566
+    MaxLength: 12
+    MinLength: 12
+    Type: String
 Resources: {}
Repository: aws-cloudformation/cfn-lint

Issue:
IoT Topic Rule fails with E1029
*cfn-lint version: 0.22.4*

*Description of issue.*

When using AWS IoT substitution templates (in my case, for IoT SQL functions) within Cloud Formation, it is necessary to use the dollar sign and curly braces (For example `${topic()}`). This gets misinterpreted as a Fn::Sub Parameter which throws an E1029 error.

Please provide as much information as possible:
* Template linting issues:
  * Please provide a CloudFormation sample that generated the issue.
```yaml
  IotTopicRule: 
    Type: AWS::IoT::TopicRule
    Properties: 
      RuleName: IotTopicRule
      TopicRulePayload:
        RuleDisabled: false
        Sql: !Sub "SELECT * FROM 'some-topic'"
        Actions: 
          - 
            Kinesis: 
              RoleArn: !Sub '${topicRole.Arn}'
              StreamName: !Ref MyKinesisStream
              PartitionKey: "${topic()}" # error happens here
```
  * If present, please add links to the (official) documentation for clarification.

AWS IoT substitution templates are explained here: https://docs.aws.amazon.com/iot/latest/developerguide/iot-substitution-templates.html

How !Sub uses variables (which `cfn-lint` looks for) is found here: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-sub.html#w2ab1c21c24c59b7

  * Validate if the issue still exists with the latest version of `cfn-lint` and/or the latest Spec files

Yes


Discussion:

Resolving patch:
diff --git a/src/cfnlint/rules/functions/SubNeeded.py b/src/cfnlint/rules/functions/SubNeeded.py
--- a/src/cfnlint/rules/functions/SubNeeded.py
+++ b/src/cfnlint/rules/functions/SubNeeded.py
@@ -28,7 +28,7 @@ class SubNeeded(CloudFormationLintRule):
 
     # Free-form text properties to exclude from this rule
     # content is part of AWS::CloudFormation::Init
-    excludes = ['UserData', 'ZipFile', 'Condition', 'AWS::CloudFormation::Init', 'CloudWatchAlarmDefinition']
+    excludes = ['UserData', 'ZipFile', 'Condition', 'AWS::CloudFormation::Init', 'CloudWatchAlarmDefinition', 'TopicRulePayload']
     api_excludes = ['Uri', 'Body']
 
     # IAM Policy has special variables that don't require !Sub, Check for these


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
Error running cfn-lint with pipe (|)
cfn-lint version: *v0.23.0*

Hello we have a problem running cfn-lint with find command. Only this version is affected as far as we know.
We are keeping couple of template is a folder and linting them like that:
```
find ./templates -type f | xargs cfn-lint -f parseable -c I -t
```

It worked flawlessly before but with the new update we are getting this error:

> 2019-08-02 15:37:01,818 - cfnlint.decode - ERROR - Template file not found: None
None:1:1:1:2:E0000:Template file not found: None

Splitting the files in separated lines with `xargs -L 1` doesn't help.

If you run the cfn-lint command on it's own it works as expected.

This example **doesn't** work:
```
find ./templates -type f | xargs -t cfn-lint -f parseable -c I -t
cfn-lint -f parseable -c I -t ./templates/t1.yml ./templates/t2.yml ./templates/t3.yml 
2019-08-02 15:50:20,891 - cfnlint.decode - ERROR - Template file not found: None
None:1:1:1:2:E0000:Template file not found: None
```
This example works:
```
cfn-lint -f parseable -c I -t ./templates/t1.yml ./templates/t2.yml ./templates/t3.yml
 ```

Regards TT


Discussion:
Thanks for reporting this.  Support for stdin feeding a template was fixed in the latest release.  Let me see what our options are to fix when xargs.  

Its worth noting that you can send multiple templates to cfn-lint as well.  We support wild card searching for templates and it will also look in folders.   
I see this issue with v0.23.0, but not with 0.21.6. However, my issue is unrelated to running with pipe or stdin; I see it when I'm running it in a script.

Resolving patch:
diff --git a/src/cfnlint/core.py b/src/cfnlint/core.py
--- a/src/cfnlint/core.py
+++ b/src/cfnlint/core.py
@@ -140,7 +140,7 @@ def get_args_filenames(cli_args):
         print(rules)
         exit(0)
 
-    if not sys.stdin.isatty():
+    if not sys.stdin.isatty() and not config.templates:
         return(config, [None], formatter)
 
     if not config.templates:


Test patch:
diff --git a/test/module/core/test_run_cli.py b/test/module/core/test_run_cli.py
--- a/test/module/core/test_run_cli.py
+++ b/test/module/core/test_run_cli.py
@@ -92,6 +92,10 @@ def test_template_via_stdin(self):
             (_, filenames, _) = cfnlint.core.get_args_filenames([])
             assert filenames == [None]
 
+        with patch('sys.stdin', StringIO(file_content)):
+            (_, filenames, _) = cfnlint.core.get_args_filenames(['--template', filename])
+            assert filenames == [filename]
+
     @patch('cfnlint.config.ConfigFileArgs._read_config', create=True)
     def test_template_config(self, yaml_mock):
         """Test template config"""
Repository: aws-cloudformation/cfn-lint

Issue:
E2532 parameters in step function map
*cfn-lint version: (`cfn-lint --version`)* 0.24.4

*Description of issue.*

Using `parameters` in a `map` state gives this error:

`E2532: Check State Machine Definition for proper syntax`

This is an allowed property for map states: https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-map-state.html?shortFooter=true#map-state-example-params

Credit to @tomaschmelik for the find



Discussion:

Resolving patch:
diff --git a/src/cfnlint/rules/resources/stepfunctions/StateMachine.py b/src/cfnlint/rules/resources/stepfunctions/StateMachine.py
--- a/src/cfnlint/rules/resources/stepfunctions/StateMachine.py
+++ b/src/cfnlint/rules/resources/stepfunctions/StateMachine.py
@@ -55,7 +55,7 @@ def _check_state_json(self, def_json, state_name, path):
             'Task': ['Resource', 'ResultPath', 'Retry', 'Catch',
                      'TimeoutSeconds', 'Parameters', 'HeartbeatSeconds'],
             'Map': ['MaxConcurrency', 'Iterator', 'ItemsPath', 'ResultPath',
-                    'Retry', 'Catch'],
+                    'Retry', 'Catch', 'Parameters'],
             'Choice': ['Choices', 'Default'],
             'Wait': ['Seconds', 'Timestamp', 'SecondsPath', 'TimestampPath'],
             'Succeed': [],


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
Cors in serverless transform trips error 
*cfn-lint version: 0.24.8

*Description of issue.*
Attempting to use CORS in API resource causes error.

E0001 Error transforming template: Resource with id [myserviceAwsUserApi] is invalid. Cors works only with inline Swagger specified in 'DefinitionBody' property
template_clean.yaml:1:1

Commenting out the Cors on lines 141-143 removes the problem:



[myservice_clean.yaml.txt](https://github.com/aws-cloudformation/cfn-python-lint/files/3816455/myservice_clean.yaml.txt)



Discussion:
@mrpackethead we use https://github.com/awslabs/serverless-application-model/blob/master/bin/sam-translate.py to translate a SAM template and then we lint the finished template.  The error we are seeing is actually an error from SAM translator.  

I think this looks correct.
https://github.com/awslabs/serverless-application-model/blob/4fa5d6a4ec60e82681daae6f0ff9d7669642c266/samtranslator/model/api/api_generator.py#L244

I know we are using some logic here to provide some parameter values to help make samtranslator work.  I need to validate that logic.  I'm guessing this template works for you and so this is a false positive find?
Yes, this template works as expected, so yes this is a false positive. 





Resolving patch:
diff --git a/src/cfnlint/transform.py b/src/cfnlint/transform.py
--- a/src/cfnlint/transform.py
+++ b/src/cfnlint/transform.py
@@ -92,7 +92,7 @@ def _replace_local_codeuri(self):
                     Transform._update_to_s3_uri('Location', resource_dict)
             if resource_type == 'AWS::Serverless::Api':
                 if ('DefinitionBody' not in resource_dict and
-                        'Auth' not in resource_dict):
+                        'Auth' not in resource_dict and 'Cors' not in resource_dict):
                     Transform._update_to_s3_uri('DefinitionUri', resource_dict)
                 else:
                     resource_dict['DefinitionBody'] = ''


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
W7001 Mapping used in sar location not found
*cfn-lint version: 0.24.6*

Given this CloudFormation template:

```yaml
AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31

Mappings:
  Alarms:
    SemanticVersion:
      Value: 1.0.0

Resources:
  MyApplication:
    Type: AWS::Serverless::Application
    Properties:
      Location:
        ApplicationId: arn:aws:serverlessrepo:us-west-1:000000000000:applications/myapp
        SemanticVersion: !FindInMap [Alarms, SemanticVersion, Value]
```

`cfn-lint` gives `W7001 Mapping 'Alarms' is defined but not used`

This is valid CloudFormation - `FindInMap` and `Ref` are (the only) allowed intrinsic functions for the Location of a `Serverless::Application`


Discussion:
I'll take a look into this.  We use the sam translator package to lint the resulting template.  It could be that the translator is removing that Mapping.  We have had to make some changes to handle this for some other things as well. 
For tracking purposes this is the translated template.  We will have to check if mappings are used against the pre-transformed template.  I'll get that in.  Thanks for reporting this.
```json
{
  "AWSTemplateFormatVersion": null,
  "Mappings": {
    "Alarms": {
      "SemanticVersion": {
        "Value": "1.0.0"
      }
    }
  },
  "Resources": {
    "MyApplication": {
      "Properties": {
        "Tags": [
          {
            "Key": "lambda:createdBy",
            "Value": "SAM"
          }
        ],
        "TemplateURL": "s3://bucket/value"
      },
      "Type": "AWS::CloudFormation::Stack"
    }
  }
}
```

Resolving patch:
diff --git a/src/cfnlint/__init__.py b/src/cfnlint/__init__.py
--- a/src/cfnlint/__init__.py
+++ b/src/cfnlint/__init__.py
@@ -115,6 +115,7 @@ def __init__(self, filename, template, regions=['us-east-1']):
         self.transform_pre['Globals'] = {}
         self.transform_pre['Ref'] = self.search_deep_keys('Ref')
         self.transform_pre['Fn::Sub'] = self.search_deep_keys('Fn::Sub')
+        self.transform_pre['Fn::FindInMap'] = self.search_deep_keys('Fn::FindInMap')
         self.conditions = cfnlint.conditions.Conditions(self)
 
     def __deepcopy__(self, memo):
diff --git a/src/cfnlint/rules/mappings/Used.py b/src/cfnlint/rules/mappings/Used.py
--- a/src/cfnlint/rules/mappings/Used.py
+++ b/src/cfnlint/rules/mappings/Used.py
@@ -37,7 +37,7 @@ def match(self, cfn):
         if mappings:
 
             # Get all "FindInMaps" that reference a Mapping
-            maptrees = cfn.search_deep_keys('Fn::FindInMap')
+            maptrees = cfn.transform_pre['Fn::FindInMap']
             for maptree in maptrees:
 
                 if isinstance(maptree[-1], list):


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
NoEcho password warning on a Lambda Function named AccountPassword
*cfn-lint version: 0.25.1*

*Description of issue.*

I'm getting this warning:

```
W2501 Inappropriate map found for password on Resources/AccountPassword
```

on this Resource in my SAM template:

```yaml
  AccountPassword:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Sub ${RepositoryName}-account-password-${StageName}
      Runtime: nodejs10.x
      Layers:
        - !Ref LambdaDependencyLayer
      CodeUri: dist/account
      Handler: password.handler
      Description: Reset User Password
      Timeout: 20
      Events:
        GET:
          Type: Api
          Properties:
            RestApiId:
              Ref: RestApi
            Path: /account/password
            Method: get
      Role: !GetAtt LambdaExecutionRole.Arn
    DeploymentPreference:
      Alarms:
        - !Ref AccountPasswordLatestVersionErrorMetricGreaterThanZeroAlarm
```

I'm guessing the resolution would be to make sure this check is only done on Parameters and not Resouces.


Discussion:

Resolving patch:
diff --git a/src/cfnlint/rules/resources/properties/Password.py b/src/cfnlint/rules/resources/properties/Password.py
--- a/src/cfnlint/rules/resources/properties/Password.py
+++ b/src/cfnlint/rules/resources/properties/Password.py
@@ -29,7 +29,11 @@ def match(self, cfn):
         for password_property in password_properties:
             # Build the list of refs
             refs = cfn.search_deep_keys(password_property)
-            trees = [tree for tree in refs if tree[0] == 'Resources']
+            trees = []
+            for tree in refs:
+                if len(tree) > 2:
+                    if tree[0] == 'Resources' and tree[2] == 'Properties':
+                        trees.append(tree)
 
             for tree in trees:
                 obj = tree[-1]


Test patch:
Repository: aws-cloudformation/cfn-lint

Issue:
Warning Check on Unused Parameter hides Error Check about Missing Parameter Type
*cfn-lint version: cfn-lint 0.25.3*

Parameters defined in a template, but not directly used, are not validated for missing attributes like `Type`.

For various reasons, we want to include parameters in our templates that are not used by resources in the templates and therefore disable `W2001` When this happens, the following template will not fail cfn-lint. If I uncomment the `Metadata` section, I will finally see the `E1012` failure. I should not have to resolve a Warning in order to unmask an Error.

```yaml
Parameters:
  Foo:
    Description: "Foo?"
Conditions:
  AlwaysFalse: !Equals [ true, false ]
Resources:
  # Metadata:
  #   Foo: !Ref Foo
  NullResource:
    Type: Custom::NullResource
    Condition: AlwaysFalse
```



Discussion:

Resolving patch:
diff --git a/src/cfnlint/rules/parameters/Configuration.py b/src/cfnlint/rules/parameters/Configuration.py
--- a/src/cfnlint/rules/parameters/Configuration.py
+++ b/src/cfnlint/rules/parameters/Configuration.py
@@ -28,6 +28,10 @@ class Configuration(CloudFormationLintRule):
         'Type',
     ]
 
+    required_keys = [
+        'Type'
+    ]
+
     def match(self, cfn):
         """Check CloudFormation Parameters"""
 
@@ -41,5 +45,12 @@ def match(self, cfn):
                         ['Parameters', paramname, propname],
                         message.format(paramname, propname)
                     ))
+            for reqname in self.required_keys:
+                if reqname not in paramvalue.keys():
+                    message = 'Parameter {0} is missing required property {1}'
+                    matches.append(RuleMatch(
+                        ['Parameters', paramname],
+                        message.format(paramname, reqname)
+                    ))
 
         return matches


Test patch:
diff --git a/test/unit/rules/parameters/test_configuration.py b/test/unit/rules/parameters/test_configuration.py
--- a/test/unit/rules/parameters/test_configuration.py
+++ b/test/unit/rules/parameters/test_configuration.py
@@ -20,4 +20,4 @@ def test_file_positive(self):
 
     def test_file_negative(self):
         """Test failure"""
-        self.helper_file_negative('test/fixtures/templates/bad/parameters.yaml', 1)
+        self.helper_file_negative('test/fixtures/templates/bad/parameters.yaml', 2)
Repository: aws-cloudformation/cfn-lint

Issue:
False positive: Sub is required if a variable is used in a string in parameter descriptions
*cfn-lint version: 0.26.0*

*Description of issue.*
Parameter descriptions fail E1029 if they contain text which looks like variable substitution:

e.g.

```yaml
  MyContentBucket:
    Description: "Bucket name for content (usually ${VPCName}-my-content), use 'none' to disable creation"
    Type: String
```

Gives an error:

    [E1029: Sub is required if a variable is used in a string] (Found an embedded parameter outside of an "Fn::Sub" at Parameters/MyContentBucket/Description)



Discussion:

Resolving patch:
diff --git a/src/cfnlint/rules/functions/SubNeeded.py b/src/cfnlint/rules/functions/SubNeeded.py
--- a/src/cfnlint/rules/functions/SubNeeded.py
+++ b/src/cfnlint/rules/functions/SubNeeded.py
@@ -98,6 +98,8 @@ def match(self, cfn):
 
         # We want to search all of the paths to check if each one contains an 'Fn::Sub'
         for parameter_string_path in parameter_string_paths:
+            if parameter_string_path[0] in ['Parameters']:
+                continue
             # Exxclude the special IAM variables
             variable = parameter_string_path[-1]
 


Test patch:
diff --git a/test/fixtures/templates/good/functions/sub_needed.yaml b/test/fixtures/templates/good/functions/sub_needed.yaml
--- a/test/fixtures/templates/good/functions/sub_needed.yaml
+++ b/test/fixtures/templates/good/functions/sub_needed.yaml
@@ -1,6 +1,9 @@
 ---
 AWSTemplateFormatVersion: "2010-09-09"
 Parameters:
+  MyContentBucket:
+    Description: "Bucket name for content (usually ${VPCName}-my-content), use 'none' to disable creation"
+    Type: String
   AMIId:
     Type: 'String'
     Description: 'The AMI ID for the image to use.'