Repository: bridgecrewio/checkov
Pull request: output baseline skipped in results
**By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.**

## Description

added boolean flag `--output-baseline-as-skipped` to output skipped checks due to baseline in the results report.

Fixes [#1502](https://github.com/bridgecrewio/checkov/issues/1502)


Problem statement:
Add option to output checks skipped from baseline
**Is your feature request related to a problem? Please describe.**
Yes.  We've inherited a terraform repo with a lot of legacy problems.  We've introduced checkov to ensure new config going 
forward meets our standards.  We've also generated baseline files for all of the existing legacy terraform config, so that we only fail checks for new problems.  However, we'd like to keep visibility on all of the baseline/legacy config that we're skipping.  In particular, with junitxml output, we'd like to see the checks that are being skipped due to the baseline.  So we'd effectively be treating all of the legacy config as skipped tests with visibility on those as we update/fix them.

**Describe the solution you'd like**
Add the ability to explicitly output baseline-skipped checks as baseline-skipped.  For junitxml output formatting, render these as skipped tests.

**Describe alternatives you've considered**
We've considered running checkov twice.  Once with the `--baseline` flag and once without, and then comparing the output junitxml files.  Failures unique to the run without the `--baseline` flag would basically be those skipped checks, so we could post-process the result XML and display those as skipped tests.  However, this is pretty ugly.

Hints from issue discussion:
Sounds very interesting. We're open for contributions here, should be implemented somewhere near this:
https://github.com/bridgecrewio/checkov/blob/ca2673633080910d69ae5a6c8133eefdaeb36787/checkov/common/output/baseline.py#L13

I'm interested in giving this a try. Has anyone worked on this issue yet?
Our team at my company had intended to, but haven't been able to prioritize it yet.  Would still love to see this, though.
@nimrodkor I could give this a shot, if @oliver-pham isn't already working on this
@chtnnh you can go ahead 🚀

Commit message: output baseline skipped in results

Patch to review:
diff --git a/checkov/common/output/baseline.py b/checkov/common/output/baseline.py
index 7a72cc0496..ca16ecdd4e 100644
--- a/checkov/common/output/baseline.py
+++ b/checkov/common/output/baseline.py
@@ -2,6 +2,7 @@
 
 import json
 from collections import defaultdict
+from checkov.common.models.enums import CheckResult
 from typing import Any, TYPE_CHECKING
 
 if TYPE_CHECKING:
@@ -11,10 +12,11 @@
 
 
 class Baseline:
-    def __init__(self) -> None:
+    def __init__(self, output_skipped: bool = False) -> None:
         self.path = ""
         self.path_failed_checks_map: dict[str, list[_BaselineFinding]] = defaultdict(list)
         self.failed_checks: list[_BaselineFailedChecks] = []
+        self.output_skipped = output_skipped
 
     def add_findings_from_report(self, report: Report) -> None:
         for check in report.failed_checks:
@@ -67,9 +69,15 @@ def compare_and_reduce_reports(self, scan_reports: list[Report]) -> None:
             scan_report.skipped_checks = [
                 check for check in scan_report.skipped_checks if self._is_check_in_baseline(check)
             ]
-            scan_report.failed_checks = [
-                check for check in scan_report.failed_checks if not self._is_check_in_baseline(check)
-            ]
+            failed_checks = []
+            for check in scan_report.failed_checks:
+                if not self._is_check_in_baseline(check):
+                    failed_checks.append(check)
+                elif self.output_skipped:
+                    check.check_result["suppress_comment"] = "baseline-skipped"
+                    check.check_result["result"] = CheckResult.SKIPPED
+                    scan_report.skipped_checks.append(check)
+            scan_report.failed_checks = failed_checks
 
     def _is_check_in_baseline(self, check: Record) -> bool:
         failed_check_id = check.check_id
diff --git a/checkov/main.py b/checkov/main.py
index aa48cebb23..be470ea396 100644
--- a/checkov/main.py
+++ b/checkov/main.py
@@ -233,7 +233,7 @@ def run(banner: str = checkov_banner, argv: List[str] = sys.argv[1:]) -> Optiona
 
     baseline = None
     if config.baseline:
-        baseline = Baseline()
+        baseline = Baseline(config.output_baseline_as_skipped)
         baseline.from_json(config.baseline)
 
     external_checks_dir = get_external_checks_dir(config)
@@ -497,6 +497,9 @@ def add_parser_args(parser: ArgumentParser) -> None:
     parser.add('--skip-cve-package',
                help='filter scan to run on all packages but a specific package identifier (denylist), You can '
                     'specify this argument multiple times to skip multiple packages', action='append', default=None)
+    parser.add('--output-baseline-as-skipped',
+            help="output checks that are skipped due to baseline file presence",
+            action='store_true', default=False)
 
 
 def get_external_checks_dir(config: Any) -> Any:


Code review comments:
diff_hunk: @@ -497,6 +497,9 @@ def add_parser_args(parser: ArgumentParser) -> None:
     parser.add('--skip-cve-package',
                help='filter scan to run on all packages but a specific package identifier (denylist), You can '
                     'specify this argument multiple times to skip multiple packages', action='append', default=None)
+    parser.add('--output-baseline-as-skipped',
path: checkov/main.py
text: @user1:
I think it would be better to put it directly after the baseline flag, so three lines higher 🙂 

@user1:
and please add it to the docs 💪 https://github.com/bridgecrewio/checkov/blob/master/docs/2.Basics/CLI%20Command%20Reference.md

diff_hunk: @@ -67,9 +69,15 @@ def compare_and_reduce_reports(self, scan_reports: list[Report]) -> None:
             scan_report.skipped_checks = [
                 check for check in scan_report.skipped_checks if self._is_check_in_baseline(check)
             ]
-            scan_report.failed_checks = [
-                check for check in scan_report.failed_checks if not self._is_check_in_baseline(check)
-            ]
+            failed_checks = []
+            for check in scan_report.failed_checks:
+                if not self._is_check_in_baseline(check):
+                    failed_checks.append(check)
+                elif self.output_skipped:
+                    check.check_result["suppress_comment"] = "baseline-skipped"
+                    check.check_result["result"] = CheckResult.SKIPPED
+                    scan_report.skipped_checks.append(check)
+            scan_report.failed_checks = failed_checks
path: checkov/common/output/baseline.py
text: @user1:
since this is a very special case, I don't think it is worth to remove the list comprehension and use the slower iteration, just run through the failed checks, if someone wants the output
```suggestion
            if self.output_skipped:
                for check in scan_report.failed_checks:
                    check.check_result["suppress_comment"] = "baseline-skipped"
                    check.check_result["result"] = CheckResult.SKIPPED
                    scan_report.skipped_checks.append(check)
```