Repository: bridgecrewio/checkov

Issue:
Environment variable  IGNORE_HIDDEN_DIRECTORY_ENV  not working
**Describe the issue**
After setting the environment variable IGNORE_HIDDEN_DIRECTORY_ENV  to false, checkov is not scanning any folder beginning with ".". I'm trying to check also terraform modules in directory .terraform.

**Examples**
chris@ /mnt/c/workspace/bxs/backupplaygroundbxs (main) $ export IGNORE_HIDDEN_DIRECTORY_ENV=false
chris@ /mnt/c/workspace/bxs/backupplaygroundbxs (main) $ echo $IGNORE_HIDDEN_DIRECTORY_ENV
false
chris@ /mnt/c/workspace/bxs/backupplaygroundbxs (main) $ checkov -d .

       _               _
   ___| |__   ___  ___| | _______   __
  / __| '_ \ / _ \/ __| |/ / _ \ \ / /
 | (__| | | |  __/ (__|   < (_) \ V /
  \___|_| |_|\___|\___|_|\_\___/ \_/

By bridgecrew.io | version: 2.0.906
Update available 2.0.906 -> 2.0.907
Run pip3 install -U checkov to update


terraform scan results:

Passed checks: 3, Failed checks: 0, Skipped checks: 0

Check: CKV_AZURE_77: "Ensure that UDP Services are restricted from the Internet "
        PASSED for resource: azurerm_network_security_group.devops
        File: /virtual_machine.tf:57-62
        Guide: https://docs.bridgecrew.io/docs/ensure-that-udp-services-are-restricted-from-the-internet
Check: CKV_AZURE_9: "Ensure that RDP access is restricted from the internet"
        PASSED for resource: azurerm_network_security_group.devops
        File: /virtual_machine.tf:57-62
        Guide: https://docs.bridgecrew.io/docs/bc_azr_networking_2
Check: CKV_AZURE_10: "Ensure that SSH access is restricted from the internet"
        PASSED for resource: azurerm_network_security_group.devops
        File: /virtual_machine.tf:57-62
        Guide: https://docs.bridgecrew.io/docs/bc_azr_networking_3

chris@ /mnt/c/workspace/bxs/backupplaygroundbxs (main) $
chris@ /mnt/c/workspace/bxs/backupplaygroundbxs (main) $
chris@ /mnt/c/workspace/bxs/backupplaygroundbxs (main) $ checkov -d .terraform


       _               _
   ___| |__   ___  ___| | _______   __
  / __| '_ \ / _ \/ __| |/ / _ \ \ / /
 | (__| | | |  __/ (__|   < (_) \ V /
  \___|_| |_|\___|\___|_|\_\___/ \_/

By bridgecrew.io | version: 2.0.906
Update available 2.0.906 -> 2.0.907
Run pip3 install -U checkov to update


terraform scan results:

Passed checks: 4, Failed checks: 9, Skipped checks: 0

Check: CKV_AZURE_118: "Ensure that Network Interfaces disable IP forwarding"
        PASSED for resource: azurerm_network_interface.vm-nic
        File: /modules/vm01/main.tf:50-64
        Guide: https://docs.bridgecrew.io/docs/ensure-that-network-interfaces-disable-ip-forwarding
Check: CKV_AZURE_92: "Ensure that Virtual Machines use managed disks"
        PASSED for resource: azurerm_linux_virtual_machine.vm
        File: /modules/vm01/main.tf:79-126
        Guide: https://docs.bridgecrew.io/docs/ensure-that-virtual-machines-use-managed-disks
Check: CKV_AZURE_118: "Ensure that Network Interfaces disable IP forwarding"
        PASSED for resource: azurerm_network_interface.vm-nic
        File: /modules/vm02/main.tf:50-64
        Guide: https://docs.bridgecrew.io/docs/ensure-that-network-interfaces-disable-ip-forwarding
Check: CKV_AZURE_92: "Ensure that Virtual Machines use managed disks"
        PASSED for resource: azurerm_linux_virtual_machine.vm
        File: /modules/vm02/main.tf:79-126
        Guide: https://docs.bridgecrew.io/docs/ensure-that-virtual-machines-use-managed-disks
Check: CKV_AZURE_1: "Ensure Azure Instance does not use basic authentication(Use SSH Key Instead)"
        FAILED for resource: azurerm_linux_virtual_machine.vm
        File: /modules/vm01/main.tf:79-126
        Guide: https://docs.bridgecrew.io/docs/bc_azr_networking_1

                79  | resource "azurerm_linux_virtual_machine" "vm" {
                80  |   name                = var.vm_name
                81  |   resource_group_name = var.resource_group.name
                82  |   location            = var.resource_group.location


**Version (please complete the following information):**
 - Checkov Version 2.0.906


Discussion:
Hey @chrisfromzurigo , Can you try overriding `CKV_IGNORED_DIRECTORIES` with `""` and run this again?
Hi 
I've tryed but unfortunate there is no change in behaviour.

 
chris@ /mnt/c/workspace/bxs/backupplaygroundbxs (main) $ export IGNORE_HIDDEN_DIRECTORY_ENV=false
chris@ /mnt/c/workspace/bxs/backupplaygroundbxs (main) $ export CKV_IGNORED_DIRECTORIES=""
chris@ /mnt/c/workspace/bxs/backupplaygroundbxs (main) $ checkov -d .
2022-03-15 14:17:12,032 [MainThread  ] [WARNI]  Failed to download module git@ssh.dev.azure.com:v3/six-group/SharedModules/shared_module_look_ups?ref=v09:None (for external modules, the --download-external-modules flag is required)
2022-03-15 14:17:12,033 [MainThread  ] [WARNI]  Failed to download module git@ssh.dev.azure.com:v3/six-group/SharedModules/shared_module_checkov?ref=v1.0.0:None (for external modules, the --download-external-modules flag is required)
2022-03-15 14:17:12,033 [MainThread  ] [WARNI]  Failed to download module git@ssh.dev.azure.com:v3/six-group/SharedModules/shared_module_data_sources?ref=v02:None (for external modules, the --download-external-modules flag is required)
2022-03-15 14:17:12,033 [MainThread  ] [WARNI]  Failed to download module git@ssh.dev.azure.com:v3/six-group/SharedModules/shared_module_app_infrastructure?ref=v24:None (for external modules, the --download-external-modules flag is required)
2022-03-15 14:17:12,033 [MainThread  ] [WARNI]  Failed to download module git@ssh.dev.azure.com:v3/six-group/SharedModules/shared_module_linux_vm?ref=v1.0.2:None (for external modules, the --download-external-modules flag is required)


       _               _
   ___| |__   ___  ___| | _______   __
  / __| '_ \ / _ \/ __| |/ / _ \ \ / /
 | (__| | | |  __/ (__|   < (_) \ V /
  \___|_| |_|\___|\___|_|\_\___/ \_/

By bridgecrew.io | version: 2.0.917
Update available 2.0.917 -> 2.0.959
Run pip3 install -U checkov to update


terraform scan results:

Passed checks: 3, Failed checks: 0, Skipped checks: 0

Check: CKV_AZURE_77: "Ensure that UDP Services are restricted from the Internet "
        PASSED for resource: azurerm_network_security_group.devops
        File: /virtual_machine.tf:57-62
        Guide: https://docs.bridgecrew.io/docs/ensure-that-udp-services-are-restricted-from-the-internet
Check: CKV_AZURE_9: "Ensure that RDP access is restricted from the internet"
        PASSED for resource: azurerm_network_security_group.devops
        File: /virtual_machine.tf:57-62
        Guide: https://docs.bridgecrew.io/docs/bc_azr_networking_2
Check: CKV_AZURE_10: "Ensure that SSH access is restricted from the internet"
        PASSED for resource: azurerm_network_security_group.devops
        File: /virtual_machine.tf:57-62
        Guide: https://docs.bridgecrew.io/docs/bc_azr_networking_3

chris@ /mnt/c/workspace/bxs/backupplaygroundbxs (main) $ checkov -d .terraform


       _               _
   ___| |__   ___  ___| | _______   __
  / __| '_ \ / _ \/ __| |/ / _ \ \ / /
 | (__| | | |  __/ (__|   < (_) \ V /
  \___|_| |_|\___|\___|_|\_\___/ \_/

By bridgecrew.io | version: 2.0.917
Update available 2.0.917 -> 2.0.959
Run pip3 install -U checkov to update


terraform scan results:

Passed checks: 4, Failed checks: 9, Skipped checks: 0

Check: CKV_AZURE_118: "Ensure that Network Interfaces disable IP forwarding"
        PASSED for resource: azurerm_network_interface.vm-nic
        File: /modules/vm01/main.tf:50-64
        Guide: https://docs.bridgecrew.io/docs/ensure-that-network-interfaces-disable-ip-forwarding
Check: CKV_AZURE_92: "Ensure that Virtual Machines use managed disks"
        PASSED for resource: azurerm_linux_virtual_machine.vm
        File: /modules/vm01/main.tf:79-126
        Guide: https://docs.bridgecrew.io/docs/ensure-that-virtual-machines-use-managed-disks
Check: CKV_AZURE_118: "Ensure that Network Interfaces disable IP forwarding"
        PASSED for resource: azurerm_network_interface.vm-nic
        File: /modules/vm02/main.tf:50-64
        Guide: https://docs.bridgecrew.io/docs/ensure-that-network-interfaces-disable-ip-forwarding
Check: CKV_AZURE_92: "Ensure that Virtual Machines use managed disks"
        PASSED for resource: azurerm_linux_virtual_machine.vm
        File: /modules/vm02/main.tf:79-126
        Guide: https://docs.bridgecrew.io/docs/ensure-that-virtual-machines-use-managed-disks
Check: CKV_AZURE_1: "Ensure Azure Instance does not use basic authentication(Use SSH Key Instead)"
        FAILED for resource: azurerm_linux_virtual_machine.vm
        File: /modules/vm01/main.tf:79-126
        Guide: https://docs.bridgecrew.io/docs/bc_azr_networking_1

                79  | resource "azurerm_linux_virtual_machine" "vm" {
                80  |   name                = var.vm_name
                81  |   resource_group_name = var.resource_group.name
                82  |   location            = var.resource_group.location
                83  |   size                = var.vm_size
                84  |

@nimrodkor I think the problem is in `terraform/parser.py`, it explicitly strips out any hidden files or directories - https://github.com/bridgecrewio/checkov/blob/master/checkov/terraform/parser.py#L202

Resolving patch:
diff --git a/checkov/terraform/parser.py b/checkov/terraform/parser.py
--- a/checkov/terraform/parser.py
+++ b/checkov/terraform/parser.py
@@ -11,7 +11,7 @@
 import hcl2
 from lark import Tree
 
-from checkov.common.runners.base_runner import filter_ignored_paths
+from checkov.common.runners.base_runner import filter_ignored_paths, IGNORE_HIDDEN_DIRECTORY_ENV
 from checkov.common.util.config_utils import should_scan_hcl_files
 from checkov.common.util.consts import DEFAULT_EXTERNAL_MODULES_DIR, RESOLVED_MODULE_ENTRY_NAME
 from checkov.common.util.json_utils import CustomJSONEncoder
@@ -192,14 +192,14 @@ def _internal_dir_load(self, directory: str,
         explicit_var_files: List[os.DirEntry] = []  # files passed with --var-file; only process the ones that are in this directory
 
         dir_contents = list(os.scandir(directory))
-        if excluded_paths:
+        if excluded_paths or IGNORE_HIDDEN_DIRECTORY_ENV:
             filter_ignored_paths(root_dir, dir_contents, excluded_paths)
 
         tf_files_to_load = []
         for file in dir_contents:
             # Ignore directories and hidden files
             try:
-                if not file.is_file() or file.name.startswith("."):
+                if not file.is_file():
                     continue
             except OSError:
                 # Skip files that can't be accessed
diff --git a/checkov/terraform/runner.py b/checkov/terraform/runner.py
--- a/checkov/terraform/runner.py
+++ b/checkov/terraform/runner.py
@@ -20,6 +20,7 @@
 from checkov.common.util import data_structures_utils
 from checkov.common.util.config_utils import should_scan_hcl_files
 from checkov.common.variables.context import EvaluationContext
+from checkov.common.runners.base_runner import ignored_directories
 from checkov.runner_filter import RunnerFilter
 from checkov.terraform.checks.data.registry import data_registry
 from checkov.terraform.checks.module.registry import module_registry
@@ -36,6 +37,7 @@
 from checkov.terraform.parser import Parser
 from checkov.terraform.tag_providers import get_resource_tags
 
+
 dpath.options.ALLOW_EMPTY_STRING_KEYS = True
 
 CHECK_BLOCK_TYPES = frozenset(['resource', 'data', 'provider', 'module'])
@@ -98,7 +100,7 @@ def run(
                     download_external_modules=runner_filter.download_external_modules,
                     external_modules_download_path=runner_filter.external_modules_download_path,
                     parsing_errors=parsing_errors,
-                    excluded_paths=runner_filter.excluded_paths,
+                    excluded_paths=runner_filter.excluded_paths ,
                     vars_files=runner_filter.var_files
                 )
             elif files:


Test patch:
diff --git a/tests/terraform/runner/resources/hidden_dir/.dir/.example1.tf b/tests/terraform/runner/resources/hidden_dir/.dir/.example1.tf
new file mode 100644
--- /dev/null
+++ b/tests/terraform/runner/resources/hidden_dir/.dir/.example1.tf
@@ -0,0 +1,10 @@
+resource "aws_s3_bucket" "foo-bucket" {
+  region        = var.region
+  bucket        = local.bucket_name
+  force_destroy = true
+
+  tags = {
+    Name = "foo-${data.aws_caller_identity.current.account_id}"
+  }
+}
+data "aws_caller_identity" "current" {}
diff --git a/tests/terraform/runner/resources/hidden_dir/.example2.tf b/tests/terraform/runner/resources/hidden_dir/.example2.tf
new file mode 100644
--- /dev/null
+++ b/tests/terraform/runner/resources/hidden_dir/.example2.tf
@@ -0,0 +1,10 @@
+resource "aws_s3_bucket" "foo-bucket" {
+  region        = var.region
+  bucket        = local.bucket_name
+  force_destroy = true
+
+  tags = {
+    Name = "foo-${data.aws_caller_identity.current.account_id}"
+  }
+}
+data "aws_caller_identity" "current" {}
diff --git a/tests/terraform/runner/resources/hidden_dir/dir1/.example1.tf b/tests/terraform/runner/resources/hidden_dir/dir1/.example1.tf
new file mode 100644
--- /dev/null
+++ b/tests/terraform/runner/resources/hidden_dir/dir1/.example1.tf
@@ -0,0 +1,10 @@
+resource "aws_s3_bucket" "foo-bucket" {
+  region        = var.region
+  bucket        = local.bucket_name
+  force_destroy = true
+
+  tags = {
+    Name = "foo-${data.aws_caller_identity.current.account_id}"
+  }
+}
+data "aws_caller_identity" "current" {}
diff --git a/tests/terraform/runner/resources/hidden_dir/dir1/example.tf b/tests/terraform/runner/resources/hidden_dir/dir1/example.tf
new file mode 100644
--- /dev/null
+++ b/tests/terraform/runner/resources/hidden_dir/dir1/example.tf
@@ -0,0 +1,10 @@
+resource "aws_s3_bucket" "foo-bucket" {
+  region        = var.region
+  bucket        = local.bucket_name
+  force_destroy = true
+
+  tags = {
+    Name = "foo-${data.aws_caller_identity.current.account_id}"
+  }
+}
+data "aws_caller_identity" "current" {}
diff --git a/tests/terraform/runner/resources/hidden_dir/example.tf b/tests/terraform/runner/resources/hidden_dir/example.tf
new file mode 100644
--- /dev/null
+++ b/tests/terraform/runner/resources/hidden_dir/example.tf
@@ -0,0 +1,10 @@
+resource "aws_s3_bucket" "foo-bucket" {
+  region        = var.region
+  bucket        = local.bucket_name
+  force_destroy = true
+
+  tags = {
+    Name = "foo-${data.aws_caller_identity.current.account_id}"
+  }
+}
+data "aws_caller_identity" "current" {}
diff --git a/tests/terraform/runner/test_runner.py b/tests/terraform/runner/test_runner.py
--- a/tests/terraform/runner/test_runner.py
+++ b/tests/terraform/runner/test_runner.py
@@ -8,6 +8,7 @@
 
 # do not remove; prevents circular import error
 from typing import Dict, Any
+from unittest import mock
 
 from checkov.common.bridgecrew.integration_features.features.policy_metadata_integration import integration as metadata_integration
 from checkov.common.bridgecrew.severities import Severities, BcSeverities
@@ -85,6 +86,7 @@ def test_runner_valid_tf(self):
 
     def test_runner_passing_valid_tf(self):
         current_dir = os.path.dirname(os.path.realpath(__file__))
+
         passing_tf_dir_path = current_dir + "/resources/valid_tf_only_passed_checks"
 
         print("testing dir" + passing_tf_dir_path)
@@ -239,7 +241,7 @@ def test_no_missing_ids(self):
                 # These checks were removed because they were duplicates
                 continue
             if f'CKV_AWS_{i}' in 'CKV_AWS_95':
-                # CKV_AWS_95 is currently implemented just on cfn - actually is CKV_AWS_76 
+                # CKV_AWS_95 is currently implemented just on cfn - actually is CKV_AWS_76
                 continue
             if f'CKV_AWS_{i}' == 'CKV_AWS_52':
                 # CKV_AWS_52 was deleted since it cannot be toggled in terraform.
@@ -1342,6 +1344,39 @@ def scan_resource_conf(self, conf: Dict[str, Any]) -> CheckResult:
         all_checks = report.failed_checks + report.passed_checks
         self.assertFalse(any(c.check_id == custom_check_id for c in all_checks))
 
+    @mock.patch("checkov.common.runners.base_runner.ignored_directories", ['dir1'])
+    def test_runner_ignore_dirs(self):
+        """CKV_IGNORED_DIRECTORIES='dir1' and CKV_IGNORE_HIDDEN_DIRECTORIES=True (default)"""
+        report = self.scan_hidden_dir()
+        self.assertEqual(len(report.resources), 1)
+
+    @mock.patch("checkov.common.runners.base_runner.ignored_directories", ['dir1'])
+    @mock.patch("checkov.common.runners.base_runner.IGNORE_HIDDEN_DIRECTORY_ENV", 0)
+    def test_runner_scan_hidden_dirs_and_ignore_dirs(self):
+        """CKV_IGNORED_DIRECTORIES='dir1' and CKV_IGNORE_HIDDEN_DIRECTORIES=False"""
+        report = self.scan_hidden_dir()
+        self.assertEqual(len(report.resources), 3)
+
+    def test_runner_scan_default_env_vars(self):
+        """CKV_IGNORED_DIRECTORIES and CKV_IGNORE_HIDDEN_DIRECTORIES are equal to default"""
+        report = self.scan_hidden_dir()
+        self.assertEqual(len(report.resources), 2)
+
+    @mock.patch("checkov.common.runners.base_runner.IGNORE_HIDDEN_DIRECTORY_ENV", 0)
+    def test_runner_scan_hidden_dirs(self):
+        """CKV_IGNORE_HIDDEN_DIRECTORIES=False and CKV_IGNORED_DIRECTORIES equals to default value"""
+        report = self.scan_hidden_dir()
+        self.assertEqual(len(report.resources), 5)
+
+    def scan_hidden_dir(self):
+        """ scan resources/hidden_dir directory."""
+        current_dir = os.path.dirname(os.path.realpath(__file__))
+        path_to_scan = os.path.join(current_dir, 'resources', 'hidden_dir')
+        runner = Runner()
+        report = runner.run(root_folder=path_to_scan, external_checks_dir=None,
+                            runner_filter=RunnerFilter(framework=["terraform"]))
+        return report
+
     def test_severity_check_filter(self):
         custom_check_id = "MY_CUSTOM_CHECK"