Repository: bridgecrewio/checkov

Issue:
checkov -f plan.json does not download external terraform modules for enrichment
with a sample config when file is used for tfplan.json
i can see it does not download the external terraform modules ( my scenario is a git source )
i can see when --directory is used it does download the external terraform module

I thought the enrichment should download the external module so the enrichment can work and suppression and so forth
I only use the #comment suppression.


```
download-external-modules: True
evaluate-variables: True
external-modules-download-path: '.external_modules'
file:
  - 'tfplan.json'
repo-root-for-plan-enrichment: '.'
framework: 'all'
no-guide: True
output: 'cli'
skip-fixes: True
skip-suppressions: True
```



Discussion:
I have the same behaviour when running the latest version from my Windows OS. when we run checkov via GH actions it does download the external modules to the ./external_modules folder and scans the files. Doing it locally that process does not happen. 
Same here, I assumed having the plan file and the stuff in the `.terraform` directory is already enough for Checkov
Maybe to enrich this issue with a bit of debug output, since we are also running into this issue and want to use skips on specific Terraform definition blocks.
Calling `checkov -d .  --download-external-modules True  --framework terraform -o junitxml`  will work fine and result in the following 
```
2022-03-09 11:16:19,804 [MainThread  ] [INFO ]  Downloading module git::https://<GITREPOURI>/module-tagging.git?ref=tags/v1.1.0:None
2022-03-09 11:16:19,804 [MainThread  ] [DEBUG]  Cache miss for git::https://<GITREPOURI>/module-tagging.git?ref=tags/v1.1.0:latest
2022-03-09 11:16:19,805 [MainThread  ] [DEBUG]  getting module git::https://<GITREPOURI>/module-tagging.git?ref=tags/v1.1.0 version: latest
2022-03-09 11:16:19,805 [MainThread  ] [DEBUG]  cloning https://<GITREPOURI>/module-tagging.git to /mnt/runner-workdir/builds/P4teuDUM/0/<SOMEWORKDIR>/.external_modules/<GITREPOURI>/module-tagging/tags/v1.1.0
2022-03-09 11:16:19,805 [MainThread  ] [DEBUG]  Popen(['git', 'clone', '-v', '--depth=1', '-b', 'v1.1.0', 'https://<GITREPOURI>/module-tagging.git', '/mnt/runner-workdir/builds/P4teuDUM/0/<SOMEWORKDIR>/.external_modules/<GITREPOURI>/module-tagging/tags/v1.1.0'], cwd=/mnt/runner-workdir/builds/P4teuDUM/0/<SOMEWORKDIR> , universal_newlines=True, shell=None, istream=None)
```

Calling `checkov   -f plan.cache.json --framework terraform_plan --download-external-modules True --repo-root-for-plan-enrichment . -o junitxml` will fail with the following Debug: 

```
2022-03-09 12:39:31,992 [MainThread  ] [INFO ]  Downloading module git::https://<GITREPOURI>/module-tagging.git?ref=tags/v1.1.0:None
2022-03-09 12:39:31,992 [MainThread  ] [DEBUG]  Cache miss for git::https://<GITREPOURI>/module-tagging.git?ref=tags/v1.1.0:latest
2022-03-09 12:39:31,992 [MainThread  ] [WARNI]  Failed to download module git::https://<GITREPOURI>/module-tagging.git?ref=tags/v1.1.0:None (for external modules, the --download-external-modules flag is required)
```
So this looks like it is because of the plan enrichment. Specifically, this call to parse_directory which doesn't pass along the configuration for download-external-modules (or external-modules-download-path):

https://github.com/bridgecrewio/checkov/blob/d73fd4bd7096d48ab3434a92a177bcc55605460a/checkov/common/runners/runner_registry.py#L250

```
Parser().parse_directory(
                directory=repo_root,  # assume plan file is in the repo-root
                out_definitions=tf_definitions,
                out_parsing_errors=parsing_errors,
            )
```

However, I'm not sure modifying that would help with suppression, since as best I can tell, this stage only happens after checkov has determined any failures already, and is only used for output formatting.

In my particular use case, I have a terraform module which creates an aws_acm_certificate, with the create_before_destroy lifecycle rule, however CKV_AWS_233 "Ensure Create before destroy for ACM certificates" still fires when checking against the plan since that information isn't contained within the plan file. 

Enriching the plan (and forcing it to download modules by modifying the code I linked above) changes the output from a JSON map of the acm certificate resource to the full terraform code of the resource, but does nothing to correct the false positive.





So at present I see that the plan enrichment works... but only for the root config and not the modules being called.
As a workaround we included the skips to the actual module calls in the root config and now it will at least not report the issues on the plan step anymore. However this effect might not be wanted, example being now the Check CKV_AZURE_118 ignores all 4 Interfaces of a particular NVA even though it only needs to be ignored on two of those interfaces. This is just a simple example, there might be more critical examples.
Thank you @melbit-michaelw for your investigation. This was a blocker for me so I ended up patching it after install:

```patch
--- runner_registry.py	2022-05-12 12:13:47.000000000 +0100
+++ runner_registry_fixed.py	2022-05-18 16:44:19.000000000 +0100
@@ -277,6 +277,7 @@
                 directory=repo_root,  # assume plan file is in the repo-root
                 out_definitions=tf_definitions,
                 out_parsing_errors=parsing_errors,
+                download_external_modules=True,
             )
             repo_definitions[repo_root] = {'tf_definitions': tf_definitions, 'parsing_errors': parsing_errors}
```

```bash
patch runner_registry.py < /tmp/checkov-runner-registry.patch
```

Resolving patch:
diff --git a/checkov/common/runners/runner_registry.py b/checkov/common/runners/runner_registry.py
--- a/checkov/common/runners/runner_registry.py
+++ b/checkov/common/runners/runner_registry.py
@@ -118,7 +118,10 @@ def _handle_report(self, scan_report: Report, repo_root_for_plan_enrichment: lis
         if metadata_integration.check_metadata:
             RunnerRegistry.enrich_report_with_guidelines(scan_report)
         if repo_root_for_plan_enrichment:
-            enriched_resources = RunnerRegistry.get_enriched_resources(repo_root_for_plan_enrichment)
+            enriched_resources = RunnerRegistry.get_enriched_resources(
+                repo_roots=repo_root_for_plan_enrichment,
+                download_external_modules=self.runner_filter.download_external_modules,
+            )
             scan_report = Report("terraform_plan").enrich_plan_report(scan_report, enriched_resources)
             scan_report = Report("terraform_plan").handle_skipped_checks(scan_report, enriched_resources)
         self.scan_reports.append(scan_report)
@@ -404,7 +407,9 @@ def enrich_report_with_guidelines(scan_report: Report) -> None:
                 record.set_guideline(guideline)
 
     @staticmethod
-    def get_enriched_resources(repo_roots: list[str | Path]) -> dict[str, dict[str, Any]]:
+    def get_enriched_resources(
+        repo_roots: list[str | Path], download_external_modules: bool
+    ) -> dict[str, dict[str, Any]]:
         repo_definitions = {}
         for repo_root in repo_roots:
             tf_definitions: dict[str, Any] = {}
@@ -413,6 +418,7 @@ def get_enriched_resources(repo_roots: list[str | Path]) -> dict[str, dict[str,
                 directory=repo_root,  # assume plan file is in the repo-root
                 out_definitions=tf_definitions,
                 out_parsing_errors=parsing_errors,
+                download_external_modules=download_external_modules,
             )
             repo_definitions[repo_root] = {'tf_definitions': tf_definitions, 'parsing_errors': parsing_errors}
 


Test patch:
diff --git a/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/log_group_external/main.tf b/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/log_group_external/main.tf
new file mode 100644
--- /dev/null
+++ b/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/log_group_external/main.tf
@@ -0,0 +1,12 @@
+resource "aws_cloudwatch_log_group" "this" {
+  # checkov:skip=CKV_AWS_158: skip it
+
+  count = var.create ? 1 : 0
+
+  name              = var.name
+  name_prefix       = var.name_prefix
+  retention_in_days = var.retention_in_days
+  kms_key_id        = var.kms_key_id
+
+  tags = var.tags
+}
diff --git a/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/log_group_external/outputs.tf b/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/log_group_external/outputs.tf
new file mode 100644
--- /dev/null
+++ b/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/log_group_external/outputs.tf
@@ -0,0 +1,9 @@
+output "cloudwatch_log_group_name" {
+  description = "Name of Cloudwatch log group"
+  value       = element(concat(aws_cloudwatch_log_group.this.*.name, [""]), 0)
+}
+
+output "cloudwatch_log_group_arn" {
+  description = "ARN of Cloudwatch log group"
+  value       = element(concat(aws_cloudwatch_log_group.this.*.arn, [""]), 0)
+}
diff --git a/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/log_group_external/variables.tf b/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/log_group_external/variables.tf
new file mode 100644
--- /dev/null
+++ b/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/log_group_external/variables.tf
@@ -0,0 +1,35 @@
+variable "create" {
+  description = "Whether to create the Cloudwatch log group"
+  type        = bool
+  default     = true
+}
+
+variable "name" {
+  description = "A name for the log group"
+  type        = string
+  default     = null
+}
+
+variable "name_prefix" {
+  description = "A name prefix for the log group"
+  type        = string
+  default     = null
+}
+
+variable "retention_in_days" {
+  description = "Specifies the number of days you want to retain log events in the specified log group"
+  type        = number
+  default     = null
+}
+
+variable "kms_key_id" {
+  description = "The ARN of the KMS Key to use when encrypting logs"
+  type        = string
+  default     = null
+}
+
+variable "tags" {
+  description = "A map of tags to add to Cloudwatch log group"
+  type        = map(string)
+  default     = {}
+}
diff --git a/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/log_group_external/versions.tf b/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/log_group_external/versions.tf
new file mode 100644
--- /dev/null
+++ b/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/log_group_external/versions.tf
@@ -0,0 +1,10 @@
+terraform {
+  required_version = ">= 0.12.26"
+
+  required_providers {
+    aws = {
+      source  = "hashicorp/aws"
+      version = ">= 2.55"
+    }
+  }
+}
diff --git a/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/main.tf b/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/main.tf
new file mode 100644
--- /dev/null
+++ b/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/main.tf
@@ -0,0 +1,4 @@
+module "log_group_external" {
+  source  = "terraform-aws-modules/cloudwatch/aws//modules/log-group"
+  version = "2.1.0"
+}
diff --git a/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/providers.tf b/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/providers.tf
new file mode 100644
--- /dev/null
+++ b/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/providers.tf
@@ -0,0 +1,3 @@
+provider "aws" {
+  region  = "eu-central-1"
+}
diff --git a/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/tfplan.json b/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/tfplan.json
new file mode 100644
--- /dev/null
+++ b/tests/common/runner_registry/plan_with_external_tf_modules_for_enrichment/tfplan.json
@@ -0,0 +1,106 @@
+{
+    "format_version": "0.1",
+    "terraform_version": "0.15.1",
+    "planned_values": {
+        "root_module": {
+            "child_modules": [
+                {
+                    "resources": [
+                        {
+                            "address": "module.log_group_external.aws_cloudwatch_log_group.this[0]",
+                            "mode": "managed",
+                            "type": "aws_cloudwatch_log_group",
+                            "name": "this",
+                            "index": 0,
+                            "provider_name": "registry.terraform.io/hashicorp/aws",
+                            "schema_version": 0,
+                            "values": {"kms_key_id": null, "name_prefix": null, "retention_in_days": 0, "tags": null}
+                        }
+                    ],
+                    "address": "module.log_group_external"
+                }
+            ]
+        }
+    },
+    "resource_changes": [
+        {
+            "address": "module.log_group_external.aws_cloudwatch_log_group.this[0]",
+            "module_address": "module.log_group_external",
+            "mode": "managed",
+            "type": "aws_cloudwatch_log_group",
+            "name": "this",
+            "index": 0,
+            "provider_name": "registry.terraform.io/hashicorp/aws",
+            "change": {
+                "actions": ["create"],
+                "before": null,
+                "after": {"kms_key_id": null, "name_prefix": null, "retention_in_days": 0, "tags": null},
+                "after_unknown": {"arn": true, "id": true, "name": true, "tags_all": true},
+                "before_sensitive": false,
+                "after_sensitive": {"tags_all": {}}
+            }
+        }
+    ],
+    "configuration": {
+        "provider_config": {
+            "aws": {"name": "aws", "expressions": {"region": {"constant_value": "eu-central-1"}}},
+            "module.log_group_external:aws": {
+                "name": "aws",
+                "version_constraint": "\u003e= 2.55.0",
+                "module_address": "module.log_group_external"
+            }
+        },
+        "root_module": {
+            "module_calls": {
+                "log_group_external": {
+                    "source": "terraform-aws-modules/cloudwatch/aws//modules/log-group",
+                    "module": {
+                        "outputs": {
+                            "cloudwatch_log_group_arn": {
+                                "expression": {"references": ["aws_cloudwatch_log_group.this"]},
+                                "description": "ARN of Cloudwatch log group"
+                            },
+                            "cloudwatch_log_group_name": {
+                                "expression": {"references": ["aws_cloudwatch_log_group.this"]},
+                                "description": "Name of Cloudwatch log group"
+                            }
+                        },
+                        "resources": [
+                            {
+                                "address": "aws_cloudwatch_log_group.this",
+                                "mode": "managed",
+                                "type": "aws_cloudwatch_log_group",
+                                "name": "this",
+                                "provider_config_key": "log_group_external:aws",
+                                "expressions": {
+                                    "kms_key_id": {"references": ["var.kms_key_id"]},
+                                    "name": {"references": ["var.name"]},
+                                    "name_prefix": {"references": ["var.name_prefix"]},
+                                    "retention_in_days": {"references": ["var.retention_in_days"]},
+                                    "tags": {"references": ["var.tags"]}
+                                },
+                                "schema_version": 0,
+                                "count_expression": {"references": ["var.create"]}
+                            }
+                        ],
+                        "variables": {
+                            "create": {"default": true, "description": "Whether to create the Cloudwatch log group"},
+                            "kms_key_id": {
+                                "default": null,
+                                "description": "The ARN of the KMS Key to use when encrypting logs"
+                            },
+                            "name": {"default": null, "description": "A name for the log group"},
+                            "name_prefix": {"default": null, "description": "A name prefix for the log group"},
+                            "retention_in_days": {
+                                "default": null,
+                                "description": "Specifies the number of days you want to retain log events in the specified log group"
+                            },
+                            "tags": {"default": {}, "description": "A map of tags to add to Cloudwatch log group"}
+                        }
+                    },
+                    "version_constraint": "~\u003e 2.0"
+                }
+            }
+        }
+    }
+}
diff --git a/tests/common/runner_registry/test_runner_registry_plan_enrichment.py b/tests/common/runner_registry/test_runner_registry_plan_enrichment.py
--- a/tests/common/runner_registry/test_runner_registry_plan_enrichment.py
+++ b/tests/common/runner_registry/test_runner_registry_plan_enrichment.py
@@ -2,14 +2,17 @@
 
 from pathlib import Path
 
+from pytest_mock import MockerFixture
+
 from checkov.common.runners.runner_registry import RunnerRegistry
 from checkov.common.util.banner import banner
 from checkov.runner_filter import RunnerFilter
+from checkov.terraform.module_loading.content import ModuleContent
+from checkov.terraform.module_loading.registry import module_loader_registry
 from checkov.terraform.plan_runner import Runner as tf_plan_runner
 
 
 class TestRunnerRegistryEnrichment(unittest.TestCase):
-
     def test_enrichment_of_plan_report(self):
         allowed_checks = ["CKV_AWS_19", "CKV_AWS_20", "CKV_AWS_28", "CKV_AWS_63", "CKV_AWS_119"]
         runner_registry = RunnerRegistry(
@@ -105,13 +108,13 @@ def test_enrichment_of_plan_report(self):
     def test_enrichment_of_plan_report_with_modules(self):
         allowed_checks = ["CKV_AWS_66", "CKV_AWS_158"]
         runner_registry = RunnerRegistry(
-            banner, RunnerFilter(checks=allowed_checks, framework="terraform_plan"), tf_plan_runner()
+            banner, RunnerFilter(checks=allowed_checks, framework=["terraform_plan"]), tf_plan_runner()
         )
 
         repo_root = Path(__file__).parent / "plan_with_tf_modules_for_enrichment"
         valid_plan_path = repo_root / "tfplan.json"
 
-        report = runner_registry.run(repo_root_for_plan_enrichment=[repo_root], files=[valid_plan_path])[0]
+        report = runner_registry.run(repo_root_for_plan_enrichment=[repo_root], files=[str(valid_plan_path)])[0]
 
         failed_check_ids = [c.check_id for c in report.failed_checks]
         passed_check_ids = [c.check_id for c in report.passed_checks]
@@ -143,13 +146,13 @@ def test_enrichment_of_plan_report_with_modules(self):
     def test_skip_check(self):
         allowed_checks = ["CKV_AWS_20", "CKV_AWS_28"]
         runner_registry = RunnerRegistry(
-            banner, RunnerFilter(checks=allowed_checks, framework="terraform_plan"), tf_plan_runner()
+            banner, RunnerFilter(checks=allowed_checks, framework=["terraform_plan"]), tf_plan_runner()
         )
 
         repo_root = Path(__file__).parent / "plan_with_hcl_for_enrichment"
         valid_plan_path = repo_root / "tfplan.json"
 
-        report = runner_registry.run(repo_root_for_plan_enrichment=[repo_root], files=[valid_plan_path])[0]
+        report = runner_registry.run(repo_root_for_plan_enrichment=[repo_root], files=[str(valid_plan_path)])[0]
 
         failed_check_ids = {c.check_id for c in report.failed_checks}
         skipped_check_ids = {c.check_id for c in report.skipped_checks}
@@ -168,7 +171,7 @@ def test_skip_check_in_module(self):
         repo_root = Path(__file__).parent / "plan_module_skip_for_enrichment" / "tf"
         valid_plan_path = repo_root / "tfplan.json"
 
-        report = runner_registry.run(repo_root_for_plan_enrichment=[repo_root], files=[valid_plan_path])[0]
+        report = runner_registry.run(repo_root_for_plan_enrichment=[repo_root], files=[str(valid_plan_path)])[0]
 
         failed_check_ids = {c.check_id for c in report.failed_checks}
         skipped_check_ids = {c.check_id for c in report.skipped_checks}
@@ -179,5 +182,46 @@ def test_skip_check_in_module(self):
         self.assertEqual(skipped_check_ids, expected_skipped_check_ids)
 
 
+def test_enrichment_of_plan_report_with_external_modules(mocker: MockerFixture):
+    # given
+    allowed_checks = ["CKV_AWS_66", "CKV_AWS_158"]
+    runner_filter = RunnerFilter(
+        checks=allowed_checks,
+        framework=["terraform_plan"],
+        download_external_modules=True,
+    )
+    runner_registry = RunnerRegistry(banner, runner_filter, tf_plan_runner())
+
+    repo_root = Path(__file__).parent / "plan_with_external_tf_modules_for_enrichment"
+    valid_plan_path = repo_root / "tfplan.json"
+
+    def _load_tf_modules(*args, **kwargs):
+        # set module cache to be the local folder instead of downloading the external module
+        module_loader_registry.module_content_cache = {
+            'terraform-aws-modules/cloudwatch/aws//modules/log-group:2.1.0': ModuleContent(
+                dir=str(repo_root / "log_group_external")
+            )
+        }
+
+    mocker.patch("checkov.terraform.parser.load_tf_modules", side_effect=_load_tf_modules)
+
+    # when
+    report = runner_registry.run(repo_root_for_plan_enrichment=[repo_root], files=[str(valid_plan_path)])[0]
+
+    # reset module cache
+    module_loader_registry.reset_module_content_cache()
+
+    # then
+    summary = report.get_summary()
+
+    assert summary["passed"] == 1
+    assert summary["failed"] == 0
+    assert summary["skipped"] == 1
+    assert summary["parsing_errors"] == 0
+
+    assert {c.check_id for c in report.passed_checks} == {"CKV_AWS_66"}
+    assert {c.check_id for c in report.skipped_checks} == {"CKV_AWS_158"}
+
+
 if __name__ == "__main__":
     unittest.main()