Repository: redhat-openshift-ecosystem/openshift-preflight
Pull request #1086 (2023-10-19 20:12:45)

adding required annotations check and necessary tests
Adding a new check that checks for newly required annotations. At first we will only warn partners about this, then after sometime we will move to enforce this check.

- Relates: JIRA: EET-3540

**Note: There are backend changes being made to support `Warning` being in the results**

openshift-ci[bot]: Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with `/test all`
openshift-ci[bot]: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086#" title="Author self-approved">acornett21</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=redhat-openshift-ecosystem%2Fopenshift-preflight).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/redhat-openshift-ecosystem/openshift-preflight/blob/main/OWNERS)~~ [acornett21]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
dcibot: - FAILURE https://www.distributed-ci.io/jobs/04d95d8e-1033-4ed4-bc74-a7e360657d1c/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086
tkrishtop: Hi @acornett21 the DCI job failed because of a new test you've added
```
TASK [redhatci.ocp.verify_tests : Fail if not all test results are as expected for preflight_operator_simple-demo-operator_results-junit.xml]
0s
task path: /usr/share/ansible/collections/ansible_collections/redhatci/ocp/roles/verify_tests/tasks/parse_junit_file.yml:35
fatal: [jumphost]: FAILED! => {"changed": false, "msg": "The following expectations failed: [{'failed_expectation': {'testcase': '[a-zA-Z]+', 'passed': True}, 'actual_result': {'testcase': 'RequiredAnnotations', 'passed': False}}]"}
```
I'll change the expectations for simple-demo-operator tomorrow - for the moment we simply expect all tests to be green.
acornett21: > I'll change the expectations for simple-demo-operator tomorrow - for the moment we simply expect all tests to be green.

@tkrishtop Are you saying the new check showed up as failed? If so that's incorrect. This is still WIP, and I'll be working to update `simple-demo-operator` and `simple-disconnected-operator` accordingly, once this feature is fully hashed out.
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/6fa21747-3eb5-4cbe-acbd-177e128e07c5/jobStates
 from https://github.com/dci-labs/dallas-pipelines/pull/911
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/fc238d3b-40e4-4fc0-9b17-7e4448bec34a/jobStates
 from https://github.com/dci-labs/dallas-pipelines/pull/911
dcibot: - FAILURE https://www.distributed-ci.io/jobs/e33a353a-c880-4c82-b0b0-8307748d0771/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086
dcibot: - FAILURE https://www.distributed-ci.io/jobs/b5469f1b-c770-4c81-ab7d-28c16a164ef1/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086
tkrishtop: I adjusted the pinned tests on our side, to ignore the new test failing for simple-demo. For the future, it would be nice to fix the simple-demo as well.
tkrishtop: check workload preflight-green
dcibot: - FAILURE https://www.distributed-ci.io/jobs/fbc562fe-75b1-4782-8f58-d65983c4bf91/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086
acornett21: @tkrishtop I've updated the the `junit` code to add another level/tag (not sure what to call this for junit). But the output will look like this for checks that are at a `warn` level.

```
<testsuites>
	<testsuite tests="5" failures="0" warnings="1" time="35.215827" name="Red Hat Certification">
		<properties></properties>
		<testcase classname="quay.io/opdev/simple-demo-operator-bundle:v0.0.6" name="ScorecardBasicSpecCheck" time="3.516921">Check to make sure that all CRs have a spec block.</testcase>
		<testcase classname="quay.io/opdev/simple-demo-operator-bundle:v0.0.6" name="ScorecardOlmSuiteCheck" time="3.530103">Operator-sdk scorecard OLM Test Suite Check</testcase>
		<testcase classname="quay.io/opdev/simple-demo-operator-bundle:v0.0.6" name="DeployableByOLM" time="28.148170">Checking if the operator could be deployed by OLM</testcase>
		<testcase classname="quay.io/opdev/simple-demo-operator-bundle:v0.0.6" name="ValidateOperatorBundle" time="0.010793">Validating Bundle image that checks if it can validate the content and format of the operator bundle</testcase>
		<testcase classname="quay.io/opdev/simple-demo-operator-bundle:v0.0.6" name="RequiredAnnotations" time="9.839866ms">
			<warning message="Warn" type="">Check that the CSV has all of the required feature annotations.: Suggested Fix: Add all of the required annotations, and make sure the value is set to either &#39;true&#39; or &#39;false&#39;</warning>
		</testcase>
	</testsuite>
</testsuites>
```

The idea of introducing a new check at level `warn` is so that partners can see the check in their results, but not be penalized if it fails. Then at a later date the check will be moved to level `best` where the check is enforced. I hope this clears up what we are trying to accomplish. Let me know if we need to adjust the XML, or if you see any issue in how the XML works, since I think DCI is the only consumer of this. 
dcibot: - FAILURE https://www.distributed-ci.io/jobs/084ea93c-3924-40dc-bb99-47becbdc8ec9/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086
dcibot: - FAILURE https://www.distributed-ci.io/jobs/c5be0b64-9302-4aae-a700-b30eeb40afb2/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/de9ade0f-5c61-46df-937c-e46f4f2edb6c/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086
acornett21: /ok-to-test
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/32501c48-f325-41ff-a794-0868f5d90ed6/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086
acornett21: /ok-to-test
acornett21: Tested locally as well now that the operators are updated...
```
{
    "image": "quay.io/opdev/simple-demo-operator-bundle:v0.0.7",
    "passed": true,
    "certification_hash": "d41d8cd98f00b204e9800998ecf8427e",
    "test_library": {
        "name": "github.com/redhat-openshift-ecosystem/openshift-preflight",
        "version": "unknown",
        "commit": "unknown"
    },
    "results": {
        "passed": [
            {
                "name": "ScorecardBasicSpecCheck",
                "elapsed_time": 12673,
                "description": "Check to make sure that all CRs have a spec block."
            },
            {
                "name": "ScorecardOlmSuiteCheck",
                "elapsed_time": 3507,
                "description": "Operator-sdk scorecard OLM Test Suite Check"
            },
            {
                "name": "DeployableByOLM",
                "elapsed_time": 66839,
                "description": "Checking if the operator could be deployed by OLM"
            },
            {
                "name": "ValidateOperatorBundle",
                "elapsed_time": 12,
                "description": "Validating Bundle image that checks if it can validate the content and format of the operator bundle"
            },
            {
                "name": "RequiredAnnotations",
                "elapsed_time": 11,
                "description": "Checks that the CSV has all of the required feature annotations."
            }
        ],
        "failed": [],
        "errors": []
    }
}
```

dcibot: - SUCCESS https://www.distributed-ci.io/jobs/bd700a34-3fe9-46db-a4ab-6494fcfc838c/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/dbba234e-f7f5-4bf1-93ac-3c388d3d22d8/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/2878be73-a4ae-4f29-841e-51c36f03f405/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086
openshift-ci[bot]: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086#" title="Author self-approved">acornett21</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086#pullrequestreview-1778151725" title="LGTM">komish</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=redhat-openshift-ecosystem%2Fopenshift-preflight).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/redhat-openshift-ecosystem/openshift-preflight/blob/main/OWNERS)~~ [acornett21,komish]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
openshift-ci[bot]: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086#" title="Author self-approved">acornett21</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086#pullrequestreview-1778155480" title="LGTM">bcrochet</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/1086#pullrequestreview-1778151725" title="LGTM">komish</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=redhat-openshift-ecosystem%2Fopenshift-preflight).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/redhat-openshift-ecosystem/openshift-preflight/blob/main/OWNERS)~~ [acornett21,bcrochet,komish]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->

Patch:
diff --git a/certification/results.go b/certification/results.go
index 19f0a34..e931c6d 100644
--- a/certification/results.go
+++ b/certification/results.go
@@ -22,4 +22,5 @@ type Results struct {
 	Passed            []Result
 	Failed            []Result
 	Errors            []Result
+	Warned            []Result
 }
diff --git a/internal/check/certification.go b/internal/check/certification.go
index 9303d86..9391e5d 100644
--- a/internal/check/certification.go
+++ b/internal/check/certification.go
@@ -6,6 +6,13 @@ import (
 	"github.com/redhat-openshift-ecosystem/openshift-preflight/internal/image"
 )
 
+// Indicates the possible levels that can be utilized in the implementation of a check.
+const (
+	LevelBest     = "best"
+	LevelOptional = "optional"
+	LevelWarn     = "warn"
+)
+
 // Check as an interface containing all methods necessary
 // to use and identify a given check.
 type Check interface {
diff --git a/internal/csv/csv.go b/internal/csv/csv.go
index 190ac9a..f0d5e83 100644
--- a/internal/csv/csv.go
+++ b/internal/csv/csv.go
@@ -10,7 +10,19 @@ import (
 	corev1 "k8s.io/api/core/v1"
 )
 
-const InfrastructureFeaturesAnnotation = "operators.openshift.io/infrastructure-features"
+const (
+	InfrastructureFeaturesAnnotation = "operators.openshift.io/infrastructure-features"
+	DisconnectedAnnotation           = "features.operators.openshift.io/disconnected"
+	FIPSCompliantAnnotation          = "features.operators.openshift.io/fips-compliant"
+	ProxyAwareAnnotation             = "features.operators.openshift.io/proxy-aware"
+	TLSProfilesAnnotation            = "features.operators.openshift.io/tls-profiles"
+	TokenAuthAWSAnnotation           = "features.operators.openshift.io/token-auth-aws"
+	TokenAuthAzureAnnotation         = "features.operators.openshift.io/token-auth-azure"
+	TokenAuthGCPAnnotation           = "features.operators.openshift.io/token-auth-gcp"
+	CNFAnnotation                    = "features.operators.openshift.io/cnf"
+	CNIAnnotation                    = "features.operators.openshift.io/cni"
+	CSIAnnotation                    = "features.operators.openshift.io/csi"
+)
 
 // SupportsDisconnected accepts a stringified list of supported features
 // and returns true if "disconnected" is listed as a supported feature.
diff --git a/internal/engine/engine.go b/internal/engine/engine.go
index 4398502..3b4cef9 100644
--- a/internal/engine/engine.go
+++ b/internal/engine/engine.go
@@ -216,33 +216,39 @@ func (c *craneEngine) ExecuteChecks(ctx context.Context) error {
 
 	// execute checks
 	logger.V(log.DBG).Info("executing checks")
-	for _, check := range c.checks {
+	for _, executedCheck := range c.checks {
 		c.results.TestedImage = c.image
 
-		logger.V(log.DBG).Info("running check", "check", check.Name())
-		if check.Metadata().Level == "optional" {
-			logger.Info(fmt.Sprintf("Check %s is not currently being enforced.", check.Name()))
+		logger.V(log.DBG).Info("running check", "check", executedCheck.Name())
+		if executedCheck.Metadata().Level == check.LevelOptional || executedCheck.Metadata().Level == check.LevelWarn {
+			logger.Info(fmt.Sprintf("Check %s is not currently being enforced.", executedCheck.Name()))
 		}
 
 		// run the validation
 		checkStartTime := time.Now()
-		checkPassed, err := check.Validate(ctx, c.imageRef)
+		checkPassed, err := executedCheck.Validate(ctx, c.imageRef)
 		checkElapsedTime := time.Since(checkStartTime)
 
 		if err != nil {
-			logger.WithValues("result", "ERROR", "err", err.Error()).Info("check completed", "check", check.Name())
-			c.results.Errors = appendUnlessOptional(c.results.Errors, certification.Result{Check: check, ElapsedTime: checkElapsedTime})
+			logger.WithValues("result", "ERROR", "err", err.Error()).Info("check completed", "check", executedCheck.Name())
+			c.results.Errors = appendUnlessOptional(c.results.Errors, certification.Result{Check: executedCheck, ElapsedTime: checkElapsedTime})
 			continue
 		}
 
 		if !checkPassed {
-			logger.WithValues("result", "FAILED").Info("check completed", "check", check.Name())
-			c.results.Failed = appendUnlessOptional(c.results.Failed, certification.Result{Check: check, ElapsedTime: checkElapsedTime})
+			// if a test doesn't pass but is of level warn include it in warning results, instead of failed results
+			if executedCheck.Metadata().Level == check.LevelWarn {
+				logger.WithValues("result", "WARNING").Info("check completed", "check", executedCheck.Name())
+				c.results.Warned = appendUnlessOptional(c.results.Warned, certification.Result{Check: executedCheck, ElapsedTime: checkElapsedTime})
+				continue
+			}
+			logger.WithValues("result", "FAILED").Info("check completed", "check", executedCheck.Name())
+			c.results.Failed = appendUnlessOptional(c.results.Failed, certification.Result{Check: executedCheck, ElapsedTime: checkElapsedTime})
 			continue
 		}
 
-		logger.WithValues("result", "PASSED").Info("check completed", "check", check.Name())
-		c.results.Passed = appendUnlessOptional(c.results.Passed, certification.Result{Check: check, ElapsedTime: checkElapsedTime})
+		logger.WithValues("result", "PASSED").Info("check completed", "check", executedCheck.Name())
+		c.results.Passed = appendUnlessOptional(c.results.Passed, certification.Result{Check: executedCheck, ElapsedTime: checkElapsedTime})
 	}
 
 	if len(c.results.Errors) > 0 || len(c.results.Failed) > 0 {
@@ -674,6 +680,7 @@ func InitializeOperatorChecks(ctx context.Context, p policy.Policy, cfg Operator
 			operatorpol.NewSecurityContextConstraintsCheck(),
 			&operatorpol.RelatedImagesCheck{},
 			operatorpol.FollowsRestrictedNetworkEnablementGuidelines{},
+			operatorpol.RequiredAnnotations{},
 		}, nil
 	}
 
diff --git a/internal/formatters/junitxml.go b/internal/formatters/junitxml.go
index 64d0787..879325f 100644
--- a/internal/formatters/junitxml.go
+++ b/internal/formatters/junitxml.go
@@ -18,6 +18,7 @@ type JUnitTestSuite struct {
 	XMLName    xml.Name        `xml:"testsuite"`
 	Tests      int             `xml:"tests,attr"`
 	Failures   int             `xml:"failures,attr"`
+	Warnings   int             `xml:"warnings,attr"`
 	Time       string          `xml:"time,attr"`
 	Name       string          `xml:"name,attr"`
 	Properties []JUnitProperty `xml:"properties>property,omitempty"`
@@ -30,7 +31,8 @@ type JUnitTestCase struct {
 	Name        string            `xml:"name,attr"`
 	Time        string            `xml:"time,attr"`
 	SkipMessage *JUnitSkipMessage `xml:"skipped,omitempty"`
-	Failure     *JUnitFailure     `xml:"failure,omitempty"`
+	Failure     *JUnitMessage     `xml:"failure,omitempty"`
+	Warning     *JUnitMessage     `xml:"warning,omitempty"`
 	SystemOut   string            `xml:"system-out,omitempty"`
 	Message     string            `xml:",chardata"`
 }
@@ -44,18 +46,19 @@ type JUnitProperty struct {
 	Value string `xml:"value,attr"`
 }
 
-type JUnitFailure struct {
+type JUnitMessage struct {
 	Message  string `xml:"message,attr"`
 	Type     string `xml:"type,attr"`
 	Contents string `xml:",chardata"`
 }
 
-func junitXMLFormatter(ctx context.Context, r certification.Results) ([]byte, error) {
+func junitXMLFormatter(_ context.Context, r certification.Results) ([]byte, error) {
 	response := getResponse(r)
 	suites := JUnitTestSuites{}
 	testsuite := JUnitTestSuite{
-		Tests:      len(r.Errors) + len(r.Failed) + len(r.Passed),
+		Tests:      len(r.Errors) + len(r.Failed) + len(r.Passed) + len(r.Warned),
 		Failures:   len(r.Errors) + len(r.Failed),
+		Warnings:   len(r.Warned),
 		Time:       "0s",
 		Name:       "Red Hat Certification",
 		Properties: []JUnitProperty{},
@@ -80,7 +83,7 @@ func junitXMLFormatter(ctx context.Context, r certification.Results) ([]byte, er
 			Classname: response.Image,
 			Name:      result.Name(),
 			Time:      result.ElapsedTime.String(),
-			Failure: &JUnitFailure{
+			Failure: &JUnitMessage{
 				Message:  "Failed",
 				Type:     "",
 				Contents: fmt.Sprintf("%s: Suggested Fix: %s", result.Help().Message, result.Help().Suggestion),
@@ -90,6 +93,21 @@ func junitXMLFormatter(ctx context.Context, r certification.Results) ([]byte, er
 		totalDuration += result.ElapsedTime
 	}
 
+	for _, result := range r.Warned {
+		testCase := JUnitTestCase{
+			Classname: response.Image,
+			Name:      result.Name(),
+			Time:      result.ElapsedTime.String(),
+			Warning: &JUnitMessage{
+				Message:  "Warn",
+				Type:     "",
+				Contents: fmt.Sprintf("%s: Suggested Fix: %s", result.Help().Message, result.Help().Suggestion),
+			},
+		}
+		testsuite.TestCases = append(testsuite.TestCases, testCase)
+		totalDuration += result.ElapsedTime
+	}
+
 	testsuite.Time = fmt.Sprintf("%f", totalDuration.Seconds())
 	suites.Suites = append(suites.Suites, testsuite)
 
diff --git a/internal/formatters/util.go b/internal/formatters/util.go
index 3cb59ad..3c1cf02 100644
--- a/internal/formatters/util.go
+++ b/internal/formatters/util.go
@@ -11,6 +11,7 @@ func getResponse(r certification.Results) UserResponse {
 	passedChecks := make([]checkExecutionInfo, 0, len(r.Passed))
 	failedChecks := make([]checkExecutionInfo, 0, len(r.Failed))
 	erroredChecks := make([]checkExecutionInfo, 0, len(r.Errors))
+	warnedChecks := make([]checkExecutionInfo, 0, len(r.Warned))
 
 	if len(r.Passed) > 0 {
 		for _, check := range r.Passed {
@@ -47,15 +48,30 @@ func getResponse(r certification.Results) UserResponse {
 		}
 	}
 
+	if len(r.Warned) > 0 {
+		for _, check := range r.Warned {
+			warnedChecks = append(warnedChecks, checkExecutionInfo{
+				Name:             check.Name(),
+				ElapsedTime:      float64(check.ElapsedTime.Milliseconds()),
+				Description:      check.Metadata().Description,
+				Help:             check.Help().Message,
+				Suggestion:       check.Help().Suggestion,
+				KnowledgeBaseURL: check.Metadata().KnowledgeBaseURL,
+				CheckURL:         check.Metadata().CheckURL,
+			})
+		}
+	}
+
 	response := UserResponse{
 		Image:             r.TestedImage,
 		Passed:            r.PassedOverall,
 		LibraryInfo:       version.Version,
 		CertificationHash: r.CertificationHash,
 		Results: resultsText{
-			Passed: passedChecks,
-			Failed: failedChecks,
-			Errors: erroredChecks,
+			Passed:   passedChecks,
+			Failed:   failedChecks,
+			Errors:   erroredChecks,
+			Warnings: warnedChecks,
 		},
 	}
 
@@ -73,9 +89,10 @@ type UserResponse struct {
 
 // resultsText represents the results of check execution against the asset.
 type resultsText struct {
-	Passed []checkExecutionInfo `json:"passed" xml:"passed"`
-	Failed []checkExecutionInfo `json:"failed" xml:"failed"`
-	Errors []checkExecutionInfo `json:"errors" xml:"errors"`
+	Passed   []checkExecutionInfo `json:"passed" xml:"passed"`
+	Failed   []checkExecutionInfo `json:"failed" xml:"failed"`
+	Errors   []checkExecutionInfo `json:"errors" xml:"errors"`
+	Warnings []checkExecutionInfo `json:"warning,omitempty" xml:"warning,omitempty"`
 }
 
 // checkExecutionInfo contains all possible output fields that a user might see in their result.
diff --git a/internal/policy/operator/required_annotations.go b/internal/policy/operator/required_annotations.go
new file mode 100644
index 0000000..03fa6eb
--- /dev/null
+++ b/internal/policy/operator/required_annotations.go
@@ -0,0 +1,102 @@
+package operator
+
+import (
+	"context"
+	"fmt"
+
+	"github.com/go-logr/logr"
+	"github.com/operator-framework/api/pkg/manifests"
+	operatorsv1alpha1 "github.com/operator-framework/api/pkg/operators/v1alpha1"
+
+	"github.com/redhat-openshift-ecosystem/openshift-preflight/internal/check"
+	libcsv "github.com/redhat-openshift-ecosystem/openshift-preflight/internal/csv"
+	"github.com/redhat-openshift-ecosystem/openshift-preflight/internal/image"
+	"github.com/redhat-openshift-ecosystem/openshift-preflight/internal/log"
+)
+
+var infraAnnotations = map[string]string{
+	libcsv.DisconnectedAnnotation:   "required",
+	libcsv.FIPSCompliantAnnotation:  "required",
+	libcsv.ProxyAwareAnnotation:     "required",
+	libcsv.TLSProfilesAnnotation:    "required",
+	libcsv.TokenAuthAWSAnnotation:   "required",
+	libcsv.TokenAuthAzureAnnotation: "required",
+	libcsv.TokenAuthGCPAnnotation:   "required",
+	libcsv.CNFAnnotation:            "optional",
+	libcsv.CNIAnnotation:            "optional",
+	libcsv.CSIAnnotation:            "optional",
+}
+
+var _ check.Check = &RequiredAnnotations{}
+
+type RequiredAnnotations struct{}
+
+func (h RequiredAnnotations) Validate(ctx context.Context, imageReference image.ImageReference) (result bool, err error) {
+	return h.validate(ctx, imageReference.ImageFSPath)
+}
+
+func (h RequiredAnnotations) getBundleCSV(_ context.Context, bundlepath string) (*operatorsv1alpha1.ClusterServiceVersion, error) {
+	bundle, err := manifests.GetBundleFromDir(bundlepath)
+	if err != nil {
+		return nil, err
+	}
+	return bundle.CSV, nil
+}
+
+func (h RequiredAnnotations) validate(ctx context.Context, bundledir string) (bool, error) {
+	logger := logr.FromContextOrDiscard(ctx)
+	csv, err := h.getBundleCSV(ctx, bundledir)
+	if err != nil {
+		return false, err
+	}
+
+	var missingAnnotations []string
+	incorrectValues := map[string]string{}
+	for annotation, status := range infraAnnotations {
+		value, ok := csv.GetAnnotations()[annotation]
+		if !ok {
+			// only add the required annotations to the missing list
+			if status == "required" {
+				missingAnnotations = append(missingAnnotations, annotation)
+			}
+			continue
+		}
+		// the only string values allowed are lower case 'true' or 'false'
+		if !(value == "true" || value == "false") {
+			incorrectValues[annotation] = value
+		}
+	}
+
+	if len(missingAnnotations) > 0 {
+		logger.V(log.DBG).Info("expected annotations are missing", "missingAnnotations", missingAnnotations)
+	}
+
+	if len(incorrectValues) > 0 {
+		for key, value := range incorrectValues {
+			logger.V(log.DBG).Info(fmt.Sprintf("expected annotation: %s to have either 'true' or 'false' value, but had value of: %s.", key, value))
+		}
+	}
+
+	return len(missingAnnotations) == 0 && len(incorrectValues) == 0, nil
+}
+
+func (h RequiredAnnotations) Name() string {
+	return "RequiredAnnotations"
+}
+
+func (h RequiredAnnotations) Metadata() check.Metadata {
+	return check.Metadata{
+		Description: "Checks that the CSV has all of the required feature annotations.",
+		// TODO: This will start as warn, but will need to move to `best`
+		Level:            check.LevelWarn,
+		KnowledgeBaseURL: "https://access.redhat.com/documentation/en-us/red_hat_software_certification/8.69/html-single/red_hat_openshift_software_certification_policy_guide/index#con-operator-requirements_openshift-sw-cert-policy-products-managed",
+		CheckURL:         "https://access.redhat.com/documentation/en-us/red_hat_software_certification/8.69/html-single/red_hat_openshift_software_certification_policy_guide/index#con-operator-requirements_openshift-sw-cert-policy-products-managed",
+	}
+}
+
+func (h RequiredAnnotations) Help() check.HelpText {
+	return check.HelpText{
+		Message:    "Check that the CSV has all of the required feature annotations.",
+		Suggestion: "Add all of the required annotations, and make sure the value is set to either 'true' or 'false'",
+	}
+}


Test patch:
diff --git a/internal/engine/engine_test.go b/internal/engine/engine_test.go
index 3d2dfb8..5f2ea47 100644
--- a/internal/engine/engine_test.go
+++ b/internal/engine/engine_test.go
@@ -108,6 +108,24 @@ var _ = Describe("Execute Checks tests", func() {
 			check.HelpText{},
 		)
 
+		warningCheckPassing := check.NewGenericCheck(
+			"warnCheckPassing",
+			func(context.Context, image.ImageReference) (bool, error) {
+				return true, nil
+			},
+			check.Metadata{Level: check.LevelWarn},
+			check.HelpText{},
+		)
+
+		warningCheckFailing := check.NewGenericCheck(
+			"warnCheckFailing",
+			func(context.Context, image.ImageReference) (bool, error) {
+				return false, nil
+			},
+			check.Metadata{Level: check.LevelWarn},
+			check.HelpText{},
+		)
+
 		emptyConfig := runtime.Config{}
 		engine = craneEngine{
 			dockerConfig: emptyConfig.DockerConfig,
@@ -118,6 +136,8 @@ var _ = Describe("Execute Checks tests", func() {
 				failedCheck,
 				optionalCheckPassing,
 				optionalCheckFailing,
+				warningCheckPassing,
+				warningCheckFailing,
 			},
 			isBundle:  false,
 			isScratch: false,
@@ -127,9 +147,10 @@ var _ = Describe("Execute Checks tests", func() {
 		It("should succeed", func() {
 			err := engine.ExecuteChecks(testcontext)
 			Expect(err).ToNot(HaveOccurred())
-			Expect(engine.results.Passed).To(HaveLen(1))
+			Expect(engine.results.Passed).To(HaveLen(2))
 			Expect(engine.results.Failed).To(HaveLen(1))
 			Expect(engine.results.Errors).To(HaveLen(1))
+			Expect(engine.results.Warned).To(HaveLen(1))
 			Expect(engine.results.CertificationHash).To(BeEmpty())
 		})
 		Context("it is a bundle", func() {
@@ -330,6 +351,7 @@ var _ = Describe("Check Name Queries", func() {
 			"SecurityContextConstraintsInCSV",
 			"AllImageRefsInRelatedImages",
 			"FollowsRestrictedNetworkEnablementGuidelines",
+			"RequiredAnnotations",
 		}),
 		Entry("scratch container policy", ScratchContainerPolicy, []string{
 			"HasLicense",
diff --git a/internal/formatters/junitxml_test.go b/internal/formatters/junitxml_test.go
index 06d6613..9e81059 100644
--- a/internal/formatters/junitxml_test.go
+++ b/internal/formatters/junitxml_test.go
@@ -78,6 +78,38 @@ var _ = Describe("JUnitXML Formatter", func() {
 						ElapsedTime: 0,
 					},
 				},
+				Warned: []certification.Result{
+					{
+						Check: check.NewGenericCheck(
+							"WarningCheckPass",
+							func(ctx context.Context, ir image.ImageReference) (bool, error) { return true, nil },
+							check.Metadata{
+								Description:      "description",
+								KnowledgeBaseURL: "kburl",
+								CheckURL:         "checkurl",
+							},
+							check.HelpText{
+								Message:    "helptext",
+								Suggestion: "suggestion",
+							}),
+						ElapsedTime: 0,
+					},
+					{
+						Check: check.NewGenericCheck(
+							"WarningCheckFail",
+							func(ctx context.Context, ir image.ImageReference) (bool, error) { return false, nil },
+							check.Metadata{
+								Description:      "description",
+								KnowledgeBaseURL: "kburl",
+								CheckURL:         "checkurl",
+							},
+							check.HelpText{
+								Message:    "helptext",
+								Suggestion: "suggestion",
+							}),
+						ElapsedTime: 0,
+					},
+				},
 			}
 		})
 		It("should format without error", func() {
diff --git a/internal/policy/operator/required_annotations_test.go b/internal/policy/operator/required_annotations_test.go
new file mode 100644
index 0000000..b79cc5b
--- /dev/null
+++ b/internal/policy/operator/required_annotations_test.go
@@ -0,0 +1,60 @@
+package operator
+
+import (
+	"context"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("RequiredAnnotations", func() {
+	var h RequiredAnnotations
+	BeforeEach(func() {
+		h = RequiredAnnotations{}
+	})
+
+	AssertMetaData(h)
+
+	When("Getting the CSV from a bundle", func() {
+		It("Should fail if a CSV is not found", func() {
+			_, err := h.getBundleCSV(context.TODO(), "./testdata/doesnotexist")
+			Expect(err).To(HaveOccurred())
+		})
+
+		It("Should successfully find a CSV in a valid bundle", func() {
+			csv, err := h.getBundleCSV(context.TODO(), "./testdata/disconnected_bundle")
+			Expect(err).ToNot(HaveOccurred())
+			Expect(csv.ObjectMeta.Name).ToNot(BeEmpty())
+		})
+	})
+
+	When("Validating that a CSV has all of the required annotations", func() {
+		var bundlepath string
+		It("Should succeed with a bundle that has been prepared as expected", func() {
+			bundlepath = "./testdata/required_annotations_bundle"
+			passed, err := h.validate(context.TODO(), bundlepath)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(passed).To(BeTrue())
+		})
+
+		It("Should fail with a bundle that has not been prepared as expected", func() {
+			bundlepath = "./testdata/invalid_bundle"
+			passed, err := h.validate(context.TODO(), bundlepath)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(passed).To(BeFalse())
+		})
+
+		It("Should fail with a bundle that has not been prepared as expected, and is missing an entry", func() {
+			bundlepath = "./testdata/incorrect_required_annotations_bundle"
+			passed, err := h.validate(context.TODO(), bundlepath)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(passed).To(BeFalse())
+		})
+		It("Should fail with a bundle that has an incorrect optional value", func() {
+			bundlepath = "./testdata/incorrect_optional_annotations_bundle"
+			passed, err := h.validate(context.TODO(), bundlepath)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(passed).To(BeFalse())
+		})
+	})
+})
diff --git a/internal/policy/operator/testdata/incorrect_optional_annotations_bundle/manifests/optional-annotations-operator.clusterserviceversion.yaml b/internal/policy/operator/testdata/incorrect_optional_annotations_bundle/manifests/optional-annotations-operator.clusterserviceversion.yaml
new file mode 100644
index 0000000..5756dbc
--- /dev/null
+++ b/internal/policy/operator/testdata/incorrect_optional_annotations_bundle/manifests/optional-annotations-operator.clusterserviceversion.yaml
@@ -0,0 +1,283 @@
+apiVersion: operators.coreos.com/v1alpha1
+kind: ClusterServiceVersion
+metadata:
+  annotations:
+    alm-examples: |-
+      [
+        {
+          "apiVersion": "tools.opdev.io/v1alpha1",
+          "kind": "optionalAnnotationsApp",
+          "metadata": {
+            "labels": {
+              "app.kubernetes.io/created-by": "optional-annotations-operator",
+              "app.kubernetes.io/instance": "optionalannotationsapp-sample",
+              "app.kubernetes.io/managed-by": "kustomize",
+              "app.kubernetes.io/name": "optionalannotationsapp",
+              "app.kubernetes.io/part-of": "optional-annotations-operator"
+            },
+            "name": "optionalannotationsapp-sample"
+          },
+          "spec": null
+        }
+      ]
+    capabilities: Basic Install
+    createdAt: "2022-12-16T20:14:33Z"
+    features.operators.openshift.io/disconnected: "false"
+    features.operators.openshift.io/fips-compliant: "false"
+    features.operators.openshift.io/proxy-aware: "false"
+    features.operators.openshift.io/tls-profiles: "false"
+    features.operators.openshift.io/token-auth-aws: "false"
+    features.operators.openshift.io/token-auth-azure: "false"
+    features.operators.openshift.io/token-auth-gcp: "false"
+    features.operators.openshift.io/cnf: "foo"
+    operators.operatorframework.io/builder: operator-sdk-v1.31.0
+    operators.operatorframework.io/project_layout: go.kubebuilder.io/v3
+  name: optional-annotations-operator.v0.0.1
+  namespace: placeholder
+spec:
+  apiservicedefinitions: {}
+  customresourcedefinitions:
+    owned:
+      - description: optionalAnnotationsApp is the Schema for the optionalannotationsapps API
+        displayName: Disconnected Friendly App
+        kind: optionalAnnotationsApp
+        name: optionalAnnotationsapps.tools.opdev.io
+        version: v1alpha1
+  description: Deploys a minimal optional-annotations-operator-aware operator
+  displayName: optional Annotations Operator
+  icon:
+    - base64data: ""
+      mediatype: ""
+  install:
+    spec:
+      clusterPermissions:
+        - rules:
+            - apiGroups:
+                - apps
+              resources:
+                - deployments
+              verbs:
+                - create
+                - delete
+                - get
+                - list
+                - patch
+                - update
+                - watch
+            - apiGroups:
+                - apps
+              resources:
+                - deployments/finalizers
+              verbs:
+                - update
+            - apiGroups:
+                - tools.opdev.io
+              resources:
+                - optionalAnnotationsapps
+              verbs:
+                - create
+                - delete
+                - get
+                - list
+                - patch
+                - update
+                - watch
+            - apiGroups:
+                - tools.opdev.io
+              resources:
+                - optionalAnnotationsapps/finalizers
+              verbs:
+                - update
+            - apiGroups:
+                - tools.opdev.io
+              resources:
+                - optionalAnnotationsapps/status
+              verbs:
+                - get
+                - patch
+                - update
+            - apiGroups:
+                - authentication.k8s.io
+              resources:
+                - tokenreviews
+              verbs:
+                - create
+            - apiGroups:
+                - authorization.k8s.io
+              resources:
+                - subjectaccessreviews
+              verbs:
+                - create
+          serviceAccountName: optional-annotations-operator-controller-manager
+      deployments:
+        - label:
+            app.kubernetes.io/component: manager
+            app.kubernetes.io/created-by: optional-annotations-operator
+            app.kubernetes.io/instance: controller-manager
+            app.kubernetes.io/managed-by: kustomize
+            app.kubernetes.io/name: deployment
+            app.kubernetes.io/part-of: optional-annotations-operator
+            control-plane: controller-manager
+          name: optional-annotations-operator-controller-manager
+          spec:
+            replicas: 1
+            selector:
+              matchLabels:
+                control-plane: controller-manager
+            strategy: {}
+            template:
+              metadata:
+                annotations:
+                  kubectl.kubernetes.io/default-container: manager
+                labels:
+                  control-plane: controller-manager
+              spec:
+                affinity:
+                  nodeAffinity:
+                    optionalDuringSchedulingIgnoredDuringExecution:
+                      nodeSelectorTerms:
+                        - matchExpressions:
+                            - key: kubernetes.io/arch
+                              operator: In
+                              values:
+                                - amd64
+                                - arm64
+                                - ppc64le
+                                - s390x
+                            - key: kubernetes.io/os
+                              operator: In
+                              values:
+                                - linux
+                containers:
+                  - args:
+                      - --secure-listen-address=0.0.0.0:8443
+                      - --upstream=http://127.0.0.1:8080/
+                      - --logtostderr=true
+                      - --v=0
+                    image: gcr.io/kubebuilder/kube-rbac-proxy@sha256:d99a8d144816b951a67648c12c0b988936ccd25cf3754f3cd85ab8c01592248f
+                    name: kube-rbac-proxy
+                    ports:
+                      - containerPort: 8443
+                        name: https
+                        protocol: TCP
+                    resources:
+                      limits:
+                        cpu: 500m
+                        memory: 128Mi
+                      requests:
+                        cpu: 5m
+                        memory: 64Mi
+                    securityContext:
+                      allowPrivilegeEscalation: false
+                      capabilities:
+                        drop:
+                          - ALL
+                  - args:
+                      - --health-probe-bind-address=:8081
+                      - --metrics-bind-address=127.0.0.1:8080
+                      - --leader-elect
+                    command:
+                      - /manager
+                    env:
+                      - name: RELATED_IMAGE_FEDORA
+                        value: quay.io/fedora/fedora@sha256:ce08a91085403ecbc637eb2a96bd3554d75537871a12a14030b89243501050f2
+                      - name: DFA_SLEEPER_IMAGE
+                        value: $(RELATED_IMAGE_FEDORA)
+                      - name: DFA_BUSYBOX_IMAGE
+                        value: $(RELATED_IMAGE_FEDORA)
+                    image: quay.io/opdev/optional-annotations-operator-cm@sha256:b4d060da584f7f5f7935e8fb78a0b62b1abb829717ad522190ac7747abd9fbd1
+                    livenessProbe:
+                      httpGet:
+                        path: /healthz
+                        port: 8081
+                      initialDelaySeconds: 15
+                      periodSeconds: 20
+                    name: manager
+                    readinessProbe:
+                      httpGet:
+                        path: /readyz
+                        port: 8081
+                      initialDelaySeconds: 5
+                      periodSeconds: 10
+                    resources:
+                      limits:
+                        cpu: 500m
+                        memory: 128Mi
+                      requests:
+                        cpu: 10m
+                        memory: 64Mi
+                    securityContext:
+                      allowPrivilegeEscalation: false
+                      capabilities:
+                        drop:
+                          - ALL
+                securityContext:
+                  runAsNonRoot: true
+                serviceAccountName: optional-annotations-operator-controller-manager
+                terminationGracePeriodSeconds: 10
+      permissions:
+        - rules:
+            - apiGroups:
+                - ""
+              resources:
+                - configmaps
+              verbs:
+                - get
+                - list
+                - watch
+                - create
+                - update
+                - patch
+                - delete
+            - apiGroups:
+                - coordination.k8s.io
+              resources:
+                - leases
+              verbs:
+                - get
+                - list
+                - watch
+                - create
+                - update
+                - patch
+                - delete
+            - apiGroups:
+                - ""
+              resources:
+                - events
+              verbs:
+                - create
+                - patch
+          serviceAccountName: optional-annotations-operator-controller-manager
+    strategy: deployment
+  installModes:
+    - supported: false
+      type: OwnNamespace
+    - supported: false
+      type: SingleNamespace
+    - supported: false
+      type: MultiNamespace
+    - supported: true
+      type: AllNamespaces
+  keywords:
+    - sample
+    - example
+    - disconnected
+  links:
+    - name: optional Annotations Operator
+      url: https://optional-annotations-operator.domain
+  maintainers:
+    - email: admins@example.com
+      name: Admins
+  maturity: alpha
+  provider:
+    name: The Operator Enablement Team
+    url: https://github.com/opdev
+  relatedImages:
+    - image: quay.io/fedora/fedora@sha256:ce08a91085403ecbc637eb2a96bd3554d75537871a12a14030b89243501050f2
+      name: fedora
+    - image: gcr.io/kubebuilder/kube-rbac-proxy@sha256:d99a8d144816b951a67648c12c0b988936ccd25cf3754f3cd85ab8c01592248f
+      name: kube-rbac-proxy
+    - image: quay.io/opdev/optional-annotations-operator-cm@sha256:b4d060da584f7f5f7935e8fb78a0b62b1abb829717ad522190ac7747abd9fbd1
+      name: manager
+  version: 0.0.1
diff --git a/internal/policy/operator/testdata/incorrect_optional_annotations_bundle/metadata/annotations.yaml b/internal/policy/operator/testdata/incorrect_optional_annotations_bundle/metadata/annotations.yaml
new file mode 100644
index 0000000..13571b0
--- /dev/null
+++ b/internal/policy/operator/testdata/incorrect_optional_annotations_bundle/metadata/annotations.yaml
@@ -0,0 +1,14 @@
+annotations:
+  # Core bundle annotations.
+  operators.operatorframework.io.bundle.mediatype.v1: registry+v1
+  operators.operatorframework.io.bundle.manifests.v1: manifests/
+  operators.operatorframework.io.bundle.metadata.v1: metadata/
+  operators.operatorframework.io.bundle.package.v1: required-annotations-operator
+  operators.operatorframework.io.bundle.channels.v1: alpha
+  operators.operatorframework.io.metrics.builder: operator-sdk-v1.31.0
+  operators.operatorframework.io.metrics.mediatype.v1: metrics+v1
+  operators.operatorframework.io.metrics.project_layout: go.kubebuilder.io/v3
+
+  # Annotations for testing.
+  operators.operatorframework.io.test.mediatype.v1: scorecard+v1
+  operators.operatorframework.io.test.config.v1: tests/scorecard/
diff --git a/internal/policy/operator/testdata/incorrect_required_annotations_bundle/manifests/required-annotations-operator.clusterserviceversion.yaml b/internal/policy/operator/testdata/incorrect_required_annotations_bundle/manifests/required-annotations-operator.clusterserviceversion.yaml
new file mode 100644
index 0000000..31db368
--- /dev/null
+++ b/internal/policy/operator/testdata/incorrect_required_annotations_bundle/manifests/required-annotations-operator.clusterserviceversion.yaml
@@ -0,0 +1,282 @@
+apiVersion: operators.coreos.com/v1alpha1
+kind: ClusterServiceVersion
+metadata:
+  annotations:
+    alm-examples: |-
+      [
+        {
+          "apiVersion": "tools.opdev.io/v1alpha1",
+          "kind": "RequiredAnnotationsApp",
+          "metadata": {
+            "labels": {
+              "app.kubernetes.io/created-by": "required-annotations-operator",
+              "app.kubernetes.io/instance": "requiredannotationsapp-sample",
+              "app.kubernetes.io/managed-by": "kustomize",
+              "app.kubernetes.io/name": "requiredannotationsapp",
+              "app.kubernetes.io/part-of": "required-annotations-operator"
+            },
+            "name": "requiredannotationsapp-sample"
+          },
+          "spec": null
+        }
+      ]
+    capabilities: Basic Install
+    createdAt: "2022-12-16T20:14:33Z"
+    features.operators.openshift.io/disconnected: "foo"
+    features.operators.openshift.io/fips-compliant: "bar"
+    features.operators.openshift.io/proxy-aware: "foo"
+    features.operators.openshift.io/tls-profiles: "bar"
+    features.operators.openshift.io/token-auth-aws: "cat"
+    features.operators.openshift.io/token-auth-azure: "hat"
+    features.operators.openshift.io/token-auth-gcp: ""
+    operators.operatorframework.io/builder: operator-sdk-v1.31.0
+    operators.operatorframework.io/project_layout: go.kubebuilder.io/v3
+  name: required-annotations-operator.v0.0.1
+  namespace: placeholder
+spec:
+  apiservicedefinitions: {}
+  customresourcedefinitions:
+    owned:
+      - description: RequiredAnnotationsApp is the Schema for the requiredannotationsapps API
+        displayName: Disconnected Friendly App
+        kind: RequiredAnnotationsApp
+        name: RequiredAnnotationsapps.tools.opdev.io
+        version: v1alpha1
+  description: Deploys a minimal required-annotations-operator-aware operator
+  displayName: Required Annotations Operator
+  icon:
+    - base64data: ""
+      mediatype: ""
+  install:
+    spec:
+      clusterPermissions:
+        - rules:
+            - apiGroups:
+                - apps
+              resources:
+                - deployments
+              verbs:
+                - create
+                - delete
+                - get
+                - list
+                - patch
+                - update
+                - watch
+            - apiGroups:
+                - apps
+              resources:
+                - deployments/finalizers
+              verbs:
+                - update
+            - apiGroups:
+                - tools.opdev.io
+              resources:
+                - RequiredAnnotationsapps
+              verbs:
+                - create
+                - delete
+                - get
+                - list
+                - patch
+                - update
+                - watch
+            - apiGroups:
+                - tools.opdev.io
+              resources:
+                - RequiredAnnotationsapps/finalizers
+              verbs:
+                - update
+            - apiGroups:
+                - tools.opdev.io
+              resources:
+                - RequiredAnnotationsapps/status
+              verbs:
+                - get
+                - patch
+                - update
+            - apiGroups:
+                - authentication.k8s.io
+              resources:
+                - tokenreviews
+              verbs:
+                - create
+            - apiGroups:
+                - authorization.k8s.io
+              resources:
+                - subjectaccessreviews
+              verbs:
+                - create
+          serviceAccountName: required-annotations-operator-controller-manager
+      deployments:
+        - label:
+            app.kubernetes.io/component: manager
+            app.kubernetes.io/created-by: required-annotations-operator
+            app.kubernetes.io/instance: controller-manager
+            app.kubernetes.io/managed-by: kustomize
+            app.kubernetes.io/name: deployment
+            app.kubernetes.io/part-of: required-annotations-operator
+            control-plane: controller-manager
+          name: required-annotations-operator-controller-manager
+          spec:
+            replicas: 1
+            selector:
+              matchLabels:
+                control-plane: controller-manager
+            strategy: {}
+            template:
+              metadata:
+                annotations:
+                  kubectl.kubernetes.io/default-container: manager
+                labels:
+                  control-plane: controller-manager
+              spec:
+                affinity:
+                  nodeAffinity:
+                    requiredDuringSchedulingIgnoredDuringExecution:
+                      nodeSelectorTerms:
+                        - matchExpressions:
+                            - key: kubernetes.io/arch
+                              operator: In
+                              values:
+                                - amd64
+                                - arm64
+                                - ppc64le
+                                - s390x
+                            - key: kubernetes.io/os
+                              operator: In
+                              values:
+                                - linux
+                containers:
+                  - args:
+                      - --secure-listen-address=0.0.0.0:8443
+                      - --upstream=http://127.0.0.1:8080/
+                      - --logtostderr=true
+                      - --v=0
+                    image: gcr.io/kubebuilder/kube-rbac-proxy@sha256:d99a8d144816b951a67648c12c0b988936ccd25cf3754f3cd85ab8c01592248f
+                    name: kube-rbac-proxy
+                    ports:
+                      - containerPort: 8443
+                        name: https
+                        protocol: TCP
+                    resources:
+                      limits:
+                        cpu: 500m
+                        memory: 128Mi
+                      requests:
+                        cpu: 5m
+                        memory: 64Mi
+                    securityContext:
+                      allowPrivilegeEscalation: false
+                      capabilities:
+                        drop:
+                          - ALL
+                  - args:
+                      - --health-probe-bind-address=:8081
+                      - --metrics-bind-address=127.0.0.1:8080
+                      - --leader-elect
+                    command:
+                      - /manager
+                    env:
+                      - name: RELATED_IMAGE_FEDORA
+                        value: quay.io/fedora/fedora@sha256:ce08a91085403ecbc637eb2a96bd3554d75537871a12a14030b89243501050f2
+                      - name: DFA_SLEEPER_IMAGE
+                        value: $(RELATED_IMAGE_FEDORA)
+                      - name: DFA_BUSYBOX_IMAGE
+                        value: $(RELATED_IMAGE_FEDORA)
+                    image: quay.io/opdev/required-annotations-operator-cm@sha256:b4d060da584f7f5f7935e8fb78a0b62b1abb829717ad522190ac7747abd9fbd1
+                    livenessProbe:
+                      httpGet:
+                        path: /healthz
+                        port: 8081
+                      initialDelaySeconds: 15
+                      periodSeconds: 20
+                    name: manager
+                    readinessProbe:
+                      httpGet:
+                        path: /readyz
+                        port: 8081
+                      initialDelaySeconds: 5
+                      periodSeconds: 10
+                    resources:
+                      limits:
+                        cpu: 500m
+                        memory: 128Mi
+                      requests:
+                        cpu: 10m
+                        memory: 64Mi
+                    securityContext:
+                      allowPrivilegeEscalation: false
+                      capabilities:
+                        drop:
+                          - ALL
+                securityContext:
+                  runAsNonRoot: true
+                serviceAccountName: required-annotations-operator-controller-manager
+                terminationGracePeriodSeconds: 10
+      permissions:
+        - rules:
+            - apiGroups:
+                - ""
+              resources:
+                - configmaps
+              verbs:
+                - get
+                - list
+                - watch
+                - create
+                - update
+                - patch
+                - delete
+            - apiGroups:
+                - coordination.k8s.io
+              resources:
+                - leases
+              verbs:
+                - get
+                - list
+                - watch
+                - create
+                - update
+                - patch
+                - delete
+            - apiGroups:
+                - ""
+              resources:
+                - events
+              verbs:
+                - create
+                - patch
+          serviceAccountName: required-annotations-operator-controller-manager
+    strategy: deployment
+  installModes:
+    - supported: false
+      type: OwnNamespace
+    - supported: false
+      type: SingleNamespace
+    - supported: false
+      type: MultiNamespace
+    - supported: true
+      type: AllNamespaces
+  keywords:
+    - sample
+    - example
+    - disconnected
+  links:
+    - name: Required Annotations Operator
+      url: https://required-annotations-operator.domain
+  maintainers:
+    - email: admins@example.com
+      name: Admins
+  maturity: alpha
+  provider:
+    name: The Operator Enablement Team
+    url: https://github.com/opdev
+  relatedImages:
+    - image: quay.io/fedora/fedora@sha256:ce08a91085403ecbc637eb2a96bd3554d75537871a12a14030b89243501050f2
+      name: fedora
+    - image: gcr.io/kubebuilder/kube-rbac-proxy@sha256:d99a8d144816b951a67648c12c0b988936ccd25cf3754f3cd85ab8c01592248f
+      name: kube-rbac-proxy
+    - image: quay.io/opdev/required-annotations-operator-cm@sha256:b4d060da584f7f5f7935e8fb78a0b62b1abb829717ad522190ac7747abd9fbd1
+      name: manager
+  version: 0.0.1
diff --git a/internal/policy/operator/testdata/incorrect_required_annotations_bundle/metadata/annotations.yaml b/internal/policy/operator/testdata/incorrect_required_annotations_bundle/metadata/annotations.yaml
new file mode 100644
index 0000000..13571b0
--- /dev/null
+++ b/internal/policy/operator/testdata/incorrect_required_annotations_bundle/metadata/annotations.yaml
@@ -0,0 +1,14 @@
+annotations:
+  # Core bundle annotations.
+  operators.operatorframework.io.bundle.mediatype.v1: registry+v1
+  operators.operatorframework.io.bundle.manifests.v1: manifests/
+  operators.operatorframework.io.bundle.metadata.v1: metadata/
+  operators.operatorframework.io.bundle.package.v1: required-annotations-operator
+  operators.operatorframework.io.bundle.channels.v1: alpha
+  operators.operatorframework.io.metrics.builder: operator-sdk-v1.31.0
+  operators.operatorframework.io.metrics.mediatype.v1: metrics+v1
+  operators.operatorframework.io.metrics.project_layout: go.kubebuilder.io/v3
+
+  # Annotations for testing.
+  operators.operatorframework.io.test.mediatype.v1: scorecard+v1
+  operators.operatorframework.io.test.config.v1: tests/scorecard/
diff --git a/internal/policy/operator/testdata/required_annotations_bundle/manifests/required-annotations-operator.clusterserviceversion.yaml b/internal/policy/operator/testdata/required_annotations_bundle/manifests/required-annotations-operator.clusterserviceversion.yaml
new file mode 100644
index 0000000..7a3407a
--- /dev/null
+++ b/internal/policy/operator/testdata/required_annotations_bundle/manifests/required-annotations-operator.clusterserviceversion.yaml
@@ -0,0 +1,282 @@
+apiVersion: operators.coreos.com/v1alpha1
+kind: ClusterServiceVersion
+metadata:
+  annotations:
+    alm-examples: |-
+      [
+        {
+          "apiVersion": "tools.opdev.io/v1alpha1",
+          "kind": "RequiredAnnotationsApp",
+          "metadata": {
+            "labels": {
+              "app.kubernetes.io/created-by": "required-annotations-operator",
+              "app.kubernetes.io/instance": "requiredannotationsapp-sample",
+              "app.kubernetes.io/managed-by": "kustomize",
+              "app.kubernetes.io/name": "requiredannotationsapp",
+              "app.kubernetes.io/part-of": "required-annotations-operator"
+            },
+            "name": "requiredannotationsapp-sample"
+          },
+          "spec": null
+        }
+      ]
+    capabilities: Basic Install
+    createdAt: "2022-12-16T20:14:33Z"
+    features.operators.openshift.io/disconnected: "true"
+    features.operators.openshift.io/fips-compliant: "false"
+    features.operators.openshift.io/proxy-aware: "false"
+    features.operators.openshift.io/tls-profiles: "false"
+    features.operators.openshift.io/token-auth-aws: "false"
+    features.operators.openshift.io/token-auth-azure: "false"
+    features.operators.openshift.io/token-auth-gcp: "false"
+    operators.operatorframework.io/builder: operator-sdk-v1.31.0
+    operators.operatorframework.io/project_layout: go.kubebuilder.io/v3
+  name: required-annotations-operator.v0.0.1
+  namespace: placeholder
+spec:
+  apiservicedefinitions: {}
+  customresourcedefinitions:
+    owned:
+      - description: RequiredAnnotationsApp is the Schema for the requiredannotationsapps API
+        displayName: Disconnected Friendly App
+        kind: RequiredAnnotationsApp
+        name: RequiredAnnotationsapps.tools.opdev.io
+        version: v1alpha1
+  description: Deploys a minimal required-annotations-operator-aware operator
+  displayName: Required Annotations Operator
+  icon:
+    - base64data: ""
+      mediatype: ""
+  install:
+    spec:
+      clusterPermissions:
+        - rules:
+            - apiGroups:
+                - apps
+              resources:
+                - deployments
+              verbs:
+                - create
+                - delete
+                - get
+                - list
+                - patch
+                - update
+                - watch
+            - apiGroups:
+                - apps
+              resources:
+                - deployments/finalizers
+              verbs:
+                - update
+            - apiGroups:
+                - tools.opdev.io
+              resources:
+                - RequiredAnnotationsapps
+              verbs:
+                - create
+                - delete
+                - get
+                - list
+                - patch
+                - update
+                - watch
+            - apiGroups:
+                - tools.opdev.io
+              resources:
+                - RequiredAnnotationsapps/finalizers
+              verbs:
+                - update
+            - apiGroups:
+                - tools.opdev.io
+              resources:
+                - RequiredAnnotationsapps/status
+              verbs:
+                - get
+                - patch
+                - update
+            - apiGroups:
+                - authentication.k8s.io
+              resources:
+                - tokenreviews
+              verbs:
+                - create
+            - apiGroups:
+                - authorization.k8s.io
+              resources:
+                - subjectaccessreviews
+              verbs:
+                - create
+          serviceAccountName: required-annotations-operator-controller-manager
+      deployments:
+        - label:
+            app.kubernetes.io/component: manager
+            app.kubernetes.io/created-by: required-annotations-operator
+            app.kubernetes.io/instance: controller-manager
+            app.kubernetes.io/managed-by: kustomize
+            app.kubernetes.io/name: deployment
+            app.kubernetes.io/part-of: required-annotations-operator
+            control-plane: controller-manager
+          name: required-annotations-operator-controller-manager
+          spec:
+            replicas: 1
+            selector:
+              matchLabels:
+                control-plane: controller-manager
+            strategy: {}
+            template:
+              metadata:
+                annotations:
+                  kubectl.kubernetes.io/default-container: manager
+                labels:
+                  control-plane: controller-manager
+              spec:
+                affinity:
+                  nodeAffinity:
+                    requiredDuringSchedulingIgnoredDuringExecution:
+                      nodeSelectorTerms:
+                        - matchExpressions:
+                            - key: kubernetes.io/arch
+                              operator: In
+                              values:
+                                - amd64
+                                - arm64
+                                - ppc64le
+                                - s390x
+                            - key: kubernetes.io/os
+                              operator: In
+                              values:
+                                - linux
+                containers:
+                  - args:
+                      - --secure-listen-address=0.0.0.0:8443
+                      - --upstream=http://127.0.0.1:8080/
+                      - --logtostderr=true
+                      - --v=0
+                    image: gcr.io/kubebuilder/kube-rbac-proxy@sha256:d99a8d144816b951a67648c12c0b988936ccd25cf3754f3cd85ab8c01592248f
+                    name: kube-rbac-proxy
+                    ports:
+                      - containerPort: 8443
+                        name: https
+                        protocol: TCP
+                    resources:
+                      limits:
+                        cpu: 500m
+                        memory: 128Mi
+                      requests:
+                        cpu: 5m
+                        memory: 64Mi
+                    securityContext:
+                      allowPrivilegeEscalation: false
+                      capabilities:
+                        drop:
+                          - ALL
+                  - args:
+                      - --health-probe-bind-address=:8081
+                      - --metrics-bind-address=127.0.0.1:8080
+                      - --leader-elect
+                    command:
+                      - /manager
+                    env:
+                      - name: RELATED_IMAGE_FEDORA
+                        value: quay.io/fedora/fedora@sha256:ce08a91085403ecbc637eb2a96bd3554d75537871a12a14030b89243501050f2
+                      - name: DFA_SLEEPER_IMAGE
+                        value: $(RELATED_IMAGE_FEDORA)
+                      - name: DFA_BUSYBOX_IMAGE
+                        value: $(RELATED_IMAGE_FEDORA)
+                    image: quay.io/opdev/required-annotations-operator-cm@sha256:b4d060da584f7f5f7935e8fb78a0b62b1abb829717ad522190ac7747abd9fbd1
+                    livenessProbe:
+                      httpGet:
+                        path: /healthz
+                        port: 8081
+                      initialDelaySeconds: 15
+                      periodSeconds: 20
+                    name: manager
+                    readinessProbe:
+                      httpGet:
+                        path: /readyz
+                        port: 8081
+                      initialDelaySeconds: 5
+                      periodSeconds: 10
+                    resources:
+                      limits:
+                        cpu: 500m
+                        memory: 128Mi
+                      requests:
+                        cpu: 10m
+                        memory: 64Mi
+                    securityContext:
+                      allowPrivilegeEscalation: false
+                      capabilities:
+                        drop:
+                          - ALL
+                securityContext:
+                  runAsNonRoot: true
+                serviceAccountName: required-annotations-operator-controller-manager
+                terminationGracePeriodSeconds: 10
+      permissions:
+        - rules:
+            - apiGroups:
+                - ""
+              resources:
+                - configmaps
+              verbs:
+                - get
+                - list
+                - watch
+                - create
+                - update
+                - patch
+                - delete
+            - apiGroups:
+                - coordination.k8s.io
+              resources:
+                - leases
+              verbs:
+                - get
+                - list
+                - watch
+                - create
+                - update
+                - patch
+                - delete
+            - apiGroups:
+                - ""
+              resources:
+                - events
+              verbs:
+                - create
+                - patch
+          serviceAccountName: required-annotations-operator-controller-manager
+    strategy: deployment
+  installModes:
+    - supported: false
+      type: OwnNamespace
+    - supported: false
+      type: SingleNamespace
+    - supported: false
+      type: MultiNamespace
+    - supported: true
+      type: AllNamespaces
+  keywords:
+    - sample
+    - example
+    - disconnected
+  links:
+    - name: Required Annotations Operator
+      url: https://required-annotations-operator.domain
+  maintainers:
+    - email: admins@example.com
+      name: Admins
+  maturity: alpha
+  provider:
+    name: The Operator Enablement Team
+    url: https://github.com/opdev
+  relatedImages:
+    - image: quay.io/fedora/fedora@sha256:ce08a91085403ecbc637eb2a96bd3554d75537871a12a14030b89243501050f2
+      name: fedora
+    - image: gcr.io/kubebuilder/kube-rbac-proxy@sha256:d99a8d144816b951a67648c12c0b988936ccd25cf3754f3cd85ab8c01592248f
+      name: kube-rbac-proxy
+    - image: quay.io/opdev/required-annotations-operator-cm@sha256:b4d060da584f7f5f7935e8fb78a0b62b1abb829717ad522190ac7747abd9fbd1
+      name: manager
+  version: 0.0.1
diff --git a/internal/policy/operator/testdata/required_annotations_bundle/metadata/annotations.yaml b/internal/policy/operator/testdata/required_annotations_bundle/metadata/annotations.yaml
new file mode 100644
index 0000000..13571b0
--- /dev/null
+++ b/internal/policy/operator/testdata/required_annotations_bundle/metadata/annotations.yaml
@@ -0,0 +1,14 @@
+annotations:
+  # Core bundle annotations.
+  operators.operatorframework.io.bundle.mediatype.v1: registry+v1
+  operators.operatorframework.io.bundle.manifests.v1: manifests/
+  operators.operatorframework.io.bundle.metadata.v1: metadata/
+  operators.operatorframework.io.bundle.package.v1: required-annotations-operator
+  operators.operatorframework.io.bundle.channels.v1: alpha
+  operators.operatorframework.io.metrics.builder: operator-sdk-v1.31.0
+  operators.operatorframework.io.metrics.mediatype.v1: metrics+v1
+  operators.operatorframework.io.metrics.project_layout: go.kubebuilder.io/v3
+
+  # Annotations for testing.
+  operators.operatorframework.io.test.mediatype.v1: scorecard+v1
+  operators.operatorframework.io.test.config.v1: tests/scorecard/