Repository: aws-cloudformation/cfn-lint

Issue:
Use of -i IGNORE_CHECKS conflicts with Metadata ignore_checks
*cfn-lint version: 0.25.0

*Description of issue.*
When specifying -i <CHECK> on the command line cfn-lint no longer honours ignore_checks metadata in the template (at last for template level metadata).

*Background*
The recently introduced I1022 rule has trigger a large number of hits across our templates, some of which are questionably false positives (I intend to open an issue for this once I have collected the details / example), so for now I tried to disable it globally by adding '-i I1022' in the pipeline that runs cfn-lint checks for our templates.  After doing this the other checks that have been suppressed using ignore_checks metadata in individual template files all started being reported again instead of being ignored.


Discussion:
The way it works today is that things are overrode and not merged.  So adding `-i I1022` will override ignore checks in .cfnlintrc and any template level Metadata ignore checks.   We could work on a way to merge those settings instead of overriding.  At least for the list based settings.

Were you able to investigate the false positives?
@kddejong I've opened #1195 with some examples of what I feel are inappropriate / false-positive triggering of I1022.

Regarding merging if I moved the ignore to a .cfnlinrc will that merge with template level metadata? that would be a reasonable workaround.

Ultimately I would suggest that -i should stack with other config, but if it's not possible / practical / desired then it would be good to note that it doesn't in the --help output and readme, since it took me a while to figure out why I suddenly saw checks I thought I'd already addressed.
I think this could be possible.  @PatMyron what do you think?  We have also been asked to add a required rule parameter which I think would allow us to switch the -i to append.
I just ran into a similar issue that I think should be considered if anyone goes to implement this. I was surprised when I added a top-level `Metadata` to ignore a check, my `ignore_checks` list declared in `.cfnlintrc.yml` were suddenly ignored.

I think a valid use case is a top level `.cfnlintrc.yml` to ignore checks globally for a group of templates in a collective project, and any `metadata` config within templates should be merged over that global config. currently if any ignore_checks are declared in a template they have to also redeclare ignore_checks from the `.cfnlintrc.yml`.

I'd propose add a flag to merge configurations, and make that flag the default functionality in a future version.

Resolving patch:
diff --git a/src/cfnlint/config.py b/src/cfnlint/config.py
--- a/src/cfnlint/config.py
+++ b/src/cfnlint/config.py
@@ -425,6 +425,10 @@ def __call__(self, parser, namespace, values, option_string=None):
             '--output-file', type=str, default=None,
             help='Writes the output to the specified file, ideal for producing reports'
         )
+        standard.add_argument(
+            '--merge-configs', default=False, action='store_true',
+            help='Merges lists between configuration layers'
+        )
 
         return parser
 
@@ -472,6 +476,7 @@ def set_template_args(self, template):
     template_args = property(get_template_args, set_template_args)
 
 
+# pylint: disable=too-many-public-methods
 class ConfigMixIn(TemplateArgs, CliArgs, ConfigFileArgs, object):
     """ Mixin for the Configs """
 
@@ -487,6 +492,22 @@ def _get_argument_value(self, arg_name, is_template, is_config_file):
         cli_value = getattr(self.cli_args, arg_name)
         template_value = self.template_args.get(arg_name)
         file_value = self.file_args.get(arg_name)
+
+        # merge list configurations
+        # make sure we don't do an infinite loop so skip this check for merge_configs
+        if arg_name != 'merge_configs':
+            if self.merge_configs:
+                # the CLI will always have an empty list when the item is a list
+                # we will use that to evaluate if we need to merge the lists
+                if isinstance(cli_value, list):
+                    result = cli_value
+                    if isinstance(template_value, list):
+                        result.extend(template_value)
+                    if isinstance(file_value, list):
+                        result.extend(file_value)
+                    return result
+
+        # return individual items
         if cli_value:
             return cli_value
         if template_value and is_template:
@@ -642,3 +663,7 @@ def output_file(self):
     @property
     def registry_schemas(self):
         return self._get_argument_value('registry_schemas', False, True)
+
+    @property
+    def merge_configs(self):
+        return self._get_argument_value('merge_configs', True, True)


Test patch:
diff --git a/test/unit/module/config/test_config_mixin.py b/test/unit/module/config/test_config_mixin.py
--- a/test/unit/module/config/test_config_mixin.py
+++ b/test/unit/module/config/test_config_mixin.py
@@ -159,3 +159,29 @@ def test_config_expand_ignore_templates(self, yaml_mock):
         self.assertNotIn(
             'test/fixtures/templates/bad/resources/iam/resource_policy.yaml', config.templates)
         self.assertEqual(len(config.templates), 4)
+
+    @patch('cfnlint.config.ConfigFileArgs._read_config', create=True)
+    def test_config_merge(self, yaml_mock):
+        """ Test merging lists  """
+
+        yaml_mock.side_effect = [
+            {"include_checks": ["I"], "ignore_checks": ["E3001"], "regions": ["us-west-2"]},
+            {}
+        ]
+        config = cfnlint.config.ConfigMixIn(['--include-checks', 'I1234', 'I4321', '--merge-configs'])
+        config.template_args = {
+            'Metadata': {
+                'cfn-lint': {
+                    'config': {
+                        'include_checks': ['I9876'],
+                        'ignore_checks': ['W3001']
+                    }
+                }
+            }
+        }
+        # config files wins
+        self.assertEqual(config.regions, ['us-west-2'])
+        # CLI should win
+        self.assertEqual(config.include_checks, ['W', 'E', 'I1234', 'I4321', 'I9876', 'I'])
+        # template file wins over config file
+        self.assertEqual(config.ignore_checks, ['W3001', 'E3001'])