Repository: aws-cloudformation/cfn-lint

Issue:
RDS instance types
Apologies for another enhancement ticket, but just ran into this problem with a development team.

*Description of issue.*

The team was using:

```
  DatabaseDBClusterParameterGroup:
    Type: "AWS::RDS::DBClusterParameterGroup"
    Properties:
      Family: "aurora5.6"

  DBInstance1a:
    Type: "AWS::RDS::DBInstance"
    Properties:
      DBInstanceClass: "db.m4.xlarge"
```

Which in CloudFormation will lead to the following eventual error:

> RDS does not support creating a DB instance with the following combination: DBInstanceClass=db.m4.xlarge, Engine=aurora, EngineVersion=5.6.10a, LicenseModel=general-public-license. For supported combinations of instance class and database engine version, see the documentation.

Developer points to https://aws.amazon.com/rds/instance-types/ which has the db.m4.xlarge on it. It's only when you scroll to the absolute bottom that you see, in a smaller font mind you, the following:

> Not every instance type is supported for every database engine, version, edition or region. Please see the DB engine-specific pricing pages for details.

Which leads to https://aws.amazon.com/rds/aurora/pricing/ and that shows only r3/r4 is available for Aurora MySQL.

This seems like something that is easily lintable, provided there is some auxiliary files like the CloudFormation spec files. So is there something similar available for instance types available for these aspects?


Discussion:
Was looking into this one tonight.  I think we could get those out as a MVP that only does this evaluation if the values of those two are strings.  (Not a function)
- We don't have a good conditions solution yet to handle cross resource cross property values when !If is being used.  
- REFs to parameters would again cause too much confusion for rule writing

This would be a great check for most people and we can handle improvement requests once they arrive in the issues list.
This is going to be a little tedious.   Working on the list of what is acceptable and its based on region, db engine, and license model and sadly there isn't a better way to get the list than via the pricing page (that I've found)

Resolving patch:
diff --git a/src/cfnlint/rules/resources/rds/InstanceSize.py b/src/cfnlint/rules/resources/rds/InstanceSize.py
new file mode 100644
--- /dev/null
+++ b/src/cfnlint/rules/resources/rds/InstanceSize.py
@@ -0,0 +1,94 @@
+"""
+  Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+  Permission is hereby granted, free of charge, to any person obtaining a copy of this
+  software and associated documentation files (the "Software"), to deal in the Software
+  without restriction, including without limitation the rights to use, copy, modify,
+  merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
+  permit persons to whom the Software is furnished to do so.
+
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
+  INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
+  PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+  HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+  OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+  SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+"""
+import six
+import cfnlint.helpers
+from cfnlint import CloudFormationLintRule
+from cfnlint import RuleMatch
+
+
+class InstanceSize(CloudFormationLintRule):
+    """Check if Resources RDS Instance Size is compatible with the RDS type"""
+    id = 'E3025'
+    shortdesc = 'RDS instance type is compatible with the RDS type'
+    description = 'Check the RDS instance types are supported by the type of RDS engine. ' \
+                  'Only if the values are strings will this be checked.'
+    source_url = 'https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.DBInstanceClass.html'
+    tags = ['resources', 'rds']
+
+    valid_instance_types = cfnlint.helpers.load_resources('/data/AdditionalSpecs/RdsProperties.json')
+
+    def get_resources(self, cfn):
+        """ Get resources that can be checked """
+        results = []
+        for resource_name, resource_values in cfn.get_resources('AWS::RDS::DBInstance').items():
+            path = ['Resources', resource_name, 'Properties']
+            properties = resource_values.get('Properties')
+            for prop_safe, prop_path_safe in properties.items_safe(path):
+                engine = prop_safe.get('Engine')
+                inst_class = prop_safe.get('DBInstanceClass')
+                license_model = prop_safe.get('LicenseModel')
+                if isinstance(engine, six.string_types) and isinstance(inst_class, six.string_types):
+                    results.append(
+                        {
+                            'Engine': engine,
+                            'DBInstanceClass': inst_class,
+                            'Path': prop_path_safe,
+                            'LicenseModel': license_model
+                        })
+
+        return results
+
+    def check_db_config(self, properties, region):
+        """ Check db properties """
+        matches = []
+
+        for valid_instance_type in self.valid_instance_types:
+            engine = properties.get('Engine')
+            if engine in valid_instance_type.get('engines'):
+                if region in valid_instance_type.get('regions'):
+                    db_license = properties.get('LicenseModel')
+                    valid_licenses = valid_instance_type.get('license')
+                    db_instance_class = properties.get('DBInstanceClass')
+                    valid_instance_types = valid_instance_type.get('instance_types')
+                    if db_license and valid_licenses:
+                        if db_license not in valid_licenses:
+                            self.logger.debug('Skip evaluation based on license not matching.')
+                            continue
+                    if db_instance_class not in valid_instance_types:
+                        if db_license is None:
+                            message = 'DBInstanceClass "{0}" is not compatible with engine type "{1}" in region "{2}". Use instance types [{3}]'
+                            matches.append(
+                                RuleMatch(
+                                    properties.get('Path') + ['DBInstanceClass'], message.format(
+                                        db_instance_class, engine, region, ', '.join(map(str, valid_instance_types)))))
+                        else:
+                            message = 'DBInstanceClass "{0}" is not compatible with engine type "{1}" and LicenseModel "{2}" in region "{3}". Use instance types [{4}]'
+                            matches.append(
+                                RuleMatch(
+                                    properties.get('Path') + ['DBInstanceClass'], message.format(
+                                        db_instance_class, engine, db_license, region, ', '.join(map(str, valid_instance_types)))))
+        return matches
+
+    def match(self, cfn):
+        """Check RDS Resource Instance Sizes"""
+
+        matches = []
+        resources = self.get_resources(cfn)
+        for resource in resources:
+            for region in cfn.regions:
+                matches.extend(self.check_db_config(resource, region))
+        return matches
diff --git a/src/cfnlint/rules/resources/rds/__init__.py b/src/cfnlint/rules/resources/rds/__init__.py
new file mode 100644
--- /dev/null
+++ b/src/cfnlint/rules/resources/rds/__init__.py
@@ -0,0 +1,16 @@
+"""
+  Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+  Permission is hereby granted, free of charge, to any person obtaining a copy of this
+  software and associated documentation files (the "Software"), to deal in the Software
+  without restriction, including without limitation the rights to use, copy, modify,
+  merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
+  permit persons to whom the Software is furnished to do so.
+
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
+  INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
+  PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+  HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+  OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+  SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+"""


Test patch:
diff --git a/test/fixtures/templates/bad/resources/rds/instance_sizes.yaml b/test/fixtures/templates/bad/resources/rds/instance_sizes.yaml
new file mode 100644
--- /dev/null
+++ b/test/fixtures/templates/bad/resources/rds/instance_sizes.yaml
@@ -0,0 +1,29 @@
+---
+AWSTemplateFormatVersion: '2010-09-09'
+Resources:
+  # Fails on invalid instance type
+  DBInstance1:
+    Type: AWS::RDS::DBInstance
+    Properties:
+      DBInstanceClass: db.m4.xlarge
+      Engine: aurora-mysql
+  # invalid Instance Type
+  DBInstance2:
+    Type: AWS::RDS::DBInstance
+    Properties:
+      DBInstanceClass: db.x2e.xlarge
+      Engine: oracle-se2
+      LicenseModel: bring-your-own-license
+  # db.x1e.xlarge isn't for license include
+  DBInstance3:
+    Type: AWS::RDS::DBInstance
+    Properties:
+      DBInstanceClass: db.x1e.xlarge
+      Engine: oracle-se2
+      LicenseModel: license-included
+  # Fails in another region
+  DBInstance4:
+    Type: AWS::RDS::DBInstance
+    Properties:
+      DBInstanceClass: db.m4.xlarge
+      Engine: mysql
diff --git a/test/fixtures/templates/good/resources/rds/instance_sizes.yaml b/test/fixtures/templates/good/resources/rds/instance_sizes.yaml
new file mode 100644
--- /dev/null
+++ b/test/fixtures/templates/good/resources/rds/instance_sizes.yaml
@@ -0,0 +1,34 @@
+---
+AWSTemplateFormatVersion: '2010-09-09'
+Parameters:
+  snapshotid:
+    Type: String
+Conditions:
+  snapshot: !Not [!Equals [!Ref snapshotid, ""]]
+Resources:
+  # Doesnt check if conditions are used
+  DBInstance1:
+    Type: AWS::RDS::DBInstance
+    Properties:
+      DBInstanceClass: db.m4.xlarge
+      Engine: !If [snapshot, !Ref snapshotid, aurora-mysql ]
+  # Doesnt fail on valid instance type
+  DBInstance2:
+    Type: AWS::RDS::DBInstance
+    Properties:
+      DBInstanceClass: db.r4.xlarge
+      Engine: aurora-mysql
+  # Doesnt fail on valid instance type with license
+  DBInstance3:
+    Type: AWS::RDS::DBInstance
+    Properties:
+      DBInstanceClass: db.x1e.xlarge
+      Engine: oracle-se2
+      LicenseModel: bring-your-own-license
+  # Doesnt fail on valid instance type with license
+  DBInstance4:
+    Type: AWS::RDS::DBInstance
+    Properties:
+      DBInstanceClass: db.r4.large
+      Engine: oracle-se2
+      LicenseModel: license-included
diff --git a/test/rules/__init__.py b/test/rules/__init__.py
--- a/test/rules/__init__.py
+++ b/test/rules/__init__.py
@@ -43,10 +43,11 @@ def helper_file_positive_template(self, filename):
         good_runner.transform()
         self.assertEqual([], good_runner.run())
 
-    def helper_file_negative(self, filename, err_count):
+    def helper_file_negative(self, filename, err_count, regions=None):
         """Failure test"""
+        regions = regions or ['us-east-1']
         template = self.load_template(filename)
-        bad_runner = Runner(self.collection, filename, template, ['us-east-1'], [])
+        bad_runner = Runner(self.collection, filename, template, regions, [])
         bad_runner.transform()
         errs = bad_runner.run()
         self.assertEqual(err_count, len(errs))
diff --git a/test/rules/resources/rds/__init__.py b/test/rules/resources/rds/__init__.py
new file mode 100644
--- /dev/null
+++ b/test/rules/resources/rds/__init__.py
@@ -0,0 +1,16 @@
+"""
+  Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+  Permission is hereby granted, free of charge, to any person obtaining a copy of this
+  software and associated documentation files (the "Software"), to deal in the Software
+  without restriction, including without limitation the rights to use, copy, modify,
+  merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
+  permit persons to whom the Software is furnished to do so.
+
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
+  INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
+  PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+  HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+  OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+  SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+"""
diff --git a/test/rules/resources/rds/test_instance_sizes.py b/test/rules/resources/rds/test_instance_sizes.py
new file mode 100644
--- /dev/null
+++ b/test/rules/resources/rds/test_instance_sizes.py
@@ -0,0 +1,37 @@
+"""
+  Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+  Permission is hereby granted, free of charge, to any person obtaining a copy of this
+  software and associated documentation files (the "Software"), to deal in the Software
+  without restriction, including without limitation the rights to use, copy, modify,
+  merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
+  permit persons to whom the Software is furnished to do so.
+
+  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
+  INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
+  PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+  HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+  OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+  SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+"""
+from cfnlint.rules.resources.rds.InstanceSize import InstanceSize  # pylint: disable=E0401
+from ... import BaseRuleTestCase
+
+
+class TestInstanceSize(BaseRuleTestCase):
+    """Test RDS Instance Size Configuration"""
+    def setUp(self):
+        """Setup"""
+        super(TestInstanceSize, self).setUp()
+        self.collection.register(InstanceSize())
+        self.success_templates = [
+            'fixtures/templates/good/resources/rds/instance_sizes.yaml'
+        ]
+
+    def test_file_positive(self):
+        """Test Positive"""
+        self.helper_file_positive()
+
+    def test_file_negative_alias(self):
+        """Test failure"""
+        self.helper_file_negative('fixtures/templates/bad/resources/rds/instance_sizes.yaml', 7, ['us-east-1', 'eu-west-3'])