Repository: stackrox/stackrox
Pull request #15232: ROX-26974: Populate API fields
Author: login: lvalerom
Labels: name: area/central
name: area/ui
Created: 2025-05-08T07:11:38Z | Merged: 2025-05-28T17:31:28Z

### Description

<!--
A detailed explanation of the changes in your PR. Feel free to remove this
section if the title of your PR is sufficiently descriptive. To learn more
about contributing to this project, check "*.md" files under:
    https://github.com/stackrox/stackrox/tree/master/.github
-->

- depends on #15231

This PR populates the `v2` protos with the `FailedClusters`.

### User-facing documentation

- [x] [CHANGELOG.md](https://github.com/stackrox/stackrox/blob/master/CHANGELOG.md) is updated **OR** update is not needed
- [x] [documentation PR](https://spaces.redhat.com/display/StackRox/Submitting+a+User+Documentation+Pull+Request) is created and is linked above **OR** is not needed

### Testing and quality

- [x] the change is production ready: the change is [GA](https://github.com/stackrox/stackrox/blob/master/PR_GA.md), or otherwise the functionality is gated by a [feature flag](https://github.com/stackrox/stackrox/blob/master/pkg/features/README.md)
- [ ] CI results are [inspected](https://docs.google.com/document/d/1d5ga073jkv4CO1kAJqp8MPGpC6E1bwyrCGZ7S5wKg3w/edit?tab=t.0#heading=h.w4ercgtcg0xp)

#### Automated testing

<!--
If no tests have been contributed, please explain why unless it's obvious,
e.g., the PR is a one-line comment change.
-->

- [ ] added unit tests
- [ ] added e2e tests
- [ ] added regression tests
- [ ] added compatibility tests
- [x] modified existing tests

#### How I validated my change

<!--
Use this space to explain **how you validated** that **your change functions
exactly how you expect it**. Feel free to attach JSON snippets, curl commands,
screenshots, etc. Apply a simple benchmark: would the information you provided
convince any reviewer or any external reader that you did enough to validate
your change.

It is acceptable to assume trust and keep this section light, e.g. as a
bullet-point list.

It is acceptable to skip testing in cases when CI is sufficient, or it's a
markdown or code comment change only. It is also acceptable to skip testing for
changes that are too taxing to test before merging. In such case you are
responsible for the change after it gets merged which includes reverting,
fixing, etc. Make sure you validate the change ASAP after it gets merged or
explain in PR when the validation will be performed. Explain here why you
skipped testing in case you did so.

Have you created automated tests for your change? Explain here which validation
activities you did manually and why so.
-->

- [x] Manual testing:
  - Deploy in two clusters with Compliance Operator
  - Create a Scan Schedule with both cluster
  - Make one of the scans fail mid scan (e.g. disconnect sensor from central)
  - Query central API and observe the status is set to PARTIAL ERROR



Diff:
diff --git a/central/complianceoperator/v2/scanconfigurations/service/convert.go b/central/complianceoperator/v2/scanconfigurations/service/convert.go
index 526a02d03c7ae..4675812a809a3 100644
--- a/central/complianceoperator/v2/scanconfigurations/service/convert.go
+++ b/central/complianceoperator/v2/scanconfigurations/service/convert.go
@@ -2,6 +2,7 @@ package service
 
 import (
 	"context"
+	"strings"
 
 	"github.com/pkg/errors"
 	blobDS "github.com/stackrox/rox/central/blob/datastore"
@@ -472,7 +473,26 @@ func convertStorageSnapshotsToV2Snapshots(ctx context.Context, snapshots []*stor
 func shallCheckDownload(reportStatus *storage.ComplianceOperatorReportStatus) bool {
 	runState := reportStatus.GetRunState()
 	return reportStatus.GetReportNotificationMethod() == storage.ComplianceOperatorReportStatus_DOWNLOAD &&
-		(runState == storage.ComplianceOperatorReportStatus_GENERATED || runState == storage.ComplianceOperatorReportStatus_DELIVERED)
+		(runState == storage.ComplianceOperatorReportStatus_GENERATED ||
+			runState == storage.ComplianceOperatorReportStatus_DELIVERED ||
+			runState == storage.ComplianceOperatorReportStatus_PARTIAL_ERROR)
+}
+
+func failedClusterReasonsJoinFunc(reasons []string) string {
+	return strings.Join(reasons, "; ")
+}
+
+func convertStorageFailedClustersToV2FailedClusters(failedClusters []*storage.ComplianceOperatorReportSnapshotV2_FailedCluster) []*v2.FailedCluster {
+	ret := make([]*v2.FailedCluster, 0, len(failedClusters))
+	for _, cluster := range failedClusters {
+		ret = append(ret, &v2.FailedCluster{
+			ClusterId:       cluster.GetClusterId(),
+			ClusterName:     cluster.GetClusterName(),
+			OperatorVersion: cluster.GetOperatorVersion(),
+			Reason:          failedClusterReasonsJoinFunc(cluster.GetReasons()),
+		})
+	}
+	return ret
 }
 
 func convertStorageSnapshotToV2Snapshot(ctx context.Context, snapshot *storage.ComplianceOperatorReportSnapshotV2,
@@ -520,6 +540,7 @@ func convertStorageSnapshotToV2Snapshot(ctx context.Context, snapshot *storage.C
 			ErrorMsg:                 snapshot.GetReportStatus().GetErrorMsg(),
 			ReportRequestType:        storageReportRequestTypeToV2[snapshot.GetReportStatus().GetReportRequestType()],
 			ReportNotificationMethod: storageReportNotificationMethodToV2[snapshot.GetReportStatus().GetReportNotificationMethod()],
+			FailedClusters:           convertStorageFailedClustersToV2FailedClusters(snapshot.GetFailedClusters()),
 		},
 		ReportData: configStatus,
 		User: &v2.SlimUser{
diff --git a/central/complianceoperator/v2/scanconfigurations/service/service_impl_test.go b/central/complianceoperator/v2/scanconfigurations/service/service_impl_test.go
index bbb53a2206a94..5f112237d7dcc 100644
--- a/central/complianceoperator/v2/scanconfigurations/service/service_impl_test.go
+++ b/central/complianceoperator/v2/scanconfigurations/service/service_impl_test.go
@@ -11,6 +11,7 @@ import (
 	benchmarkMocks "github.com/stackrox/rox/central/complianceoperator/v2/benchmarks/datastore/mocks"
 	managerMocks "github.com/stackrox/rox/central/complianceoperator/v2/compliancemanager/mocks"
 	profileDatastore "github.com/stackrox/rox/central/complianceoperator/v2/profiles/datastore/mocks"
+	"github.com/stackrox/rox/central/complianceoperator/v2/report"
 	snapshotMocks "github.com/stackrox/rox/central/complianceoperator/v2/report/datastore/mocks"
 	reportManagerMocks "github.com/stackrox/rox/central/complianceoperator/v2/report/manager/mocks"
 	scanConfigMocks "github.com/stackrox/rox/central/complianceoperator/v2/scanconfigurations/datastore/mocks"
@@ -730,6 +731,14 @@ func (s *ComplianceScanConfigServiceTestSuite) TestGetReportHistory() {
 					StartedAt:                now,
 					CompletedAt:              now,
 				},
+				FailedClusters: []*storage.ComplianceOperatorReportSnapshotV2_FailedCluster{
+					{
+						ClusterId:       "cluster-1",
+						ClusterName:     "cluster-1",
+						OperatorVersion: "v1.6.0",
+						Reasons:         []string{report.INTERNAL_ERROR},
+					},
+				},
 			},
 		}
 		sc := &storage.ComplianceOperatorScanConfigurationV2{
@@ -754,6 +763,86 @@ func (s *ComplianceScanConfigServiceTestSuite) TestGetReportHistory() {
 						ReportNotificationMethod: v2.NotificationMethod_EMAIL,
 						StartedAt:                now,
 						CompletedAt:              now,
+						FailedClusters: []*v2.FailedCluster{
+							{
+								ClusterId:       "cluster-1",
+								ClusterName:     "cluster-1",
+								OperatorVersion: "v1.6.0",
+								Reason:          report.INTERNAL_ERROR,
+							},
+						},
+					},
+					ReportData: &v2.ComplianceScanConfigurationStatus{
+						Id:       scanConfigID,
+						ScanName: scanConfigID,
+						ScanConfig: &v2.BaseComplianceScanConfigurationSettings{
+							OneTimeScan: false,
+							Profiles:    []string{},
+							Notifiers:   []*v2.NotifierConfiguration{},
+						},
+						ClusterStatus: []*v2.ClusterScanStatus{},
+						ModifiedBy:    &v2.SlimUser{},
+					},
+					User:                &v2.SlimUser{},
+					IsDownloadAvailable: false,
+				},
+			},
+		}, res)
+	})
+
+	s.Run("Success with failed cluster and multiple errors", func() {
+		scanConfigID := "scan-config-1"
+		now := protocompat.TimestampNow()
+		snapshots := []*storage.ComplianceOperatorReportSnapshotV2{
+			{
+				ReportId:            "snapshot-1",
+				ScanConfigurationId: scanConfigID,
+				ReportStatus: &storage.ComplianceOperatorReportStatus{
+					ReportRequestType:        storage.ComplianceOperatorReportStatus_SCHEDULED,
+					ReportNotificationMethod: storage.ComplianceOperatorReportStatus_EMAIL,
+					StartedAt:                now,
+					CompletedAt:              now,
+				},
+				FailedClusters: []*storage.ComplianceOperatorReportSnapshotV2_FailedCluster{
+					{
+						ClusterId:       "cluster-1",
+						ClusterName:     "cluster-1",
+						OperatorVersion: "v1.6.0",
+						Reasons:         []string{report.INTERNAL_ERROR, report.COMPLIANCE_VERSION_ERROR},
+					},
+				},
+			},
+		}
+		sc := &storage.ComplianceOperatorScanConfigurationV2{
+			Id:             scanConfigID,
+			ScanConfigName: scanConfigID,
+		}
+
+		s.snapshotDS.EXPECT().SearchSnapshots(allAccessContext, gomock.Any()).Return(snapshots, nil)
+		s.scanConfigDatastore.EXPECT().GetScanConfiguration(allAccessContext, scanConfigID).Return(sc, true, nil)
+		s.scanConfigDatastore.EXPECT().GetScanConfigClusterStatus(allAccessContext, scanConfigID).Return(nil, nil)
+		s.suiteDataStore.EXPECT().GetSuites(allAccessContext, gomock.Any()).Return(nil, nil)
+
+		res, err := s.service.GetReportHistory(allAccessContext, &v2.ComplianceReportHistoryRequest{Id: scanConfigID})
+		s.Require().NoError(err)
+		protoassert.Equal(s.T(), &v2.ComplianceReportHistoryResponse{
+			ComplianceReportSnapshots: []*v2.ComplianceReportSnapshot{
+				{
+					ReportJobId:  "snapshot-1",
+					ScanConfigId: scanConfigID,
+					ReportStatus: &v2.ComplianceReportStatus{
+						ReportRequestType:        v2.ComplianceReportStatus_SCHEDULED,
+						ReportNotificationMethod: v2.NotificationMethod_EMAIL,
+						StartedAt:                now,
+						CompletedAt:              now,
+						FailedClusters: []*v2.FailedCluster{
+							{
+								ClusterId:       "cluster-1",
+								ClusterName:     "cluster-1",
+								OperatorVersion: "v1.6.0",
+								Reason:          failedClusterReasonsJoinFunc([]string{report.INTERNAL_ERROR, report.COMPLIANCE_VERSION_ERROR}),
+							},
+						},
 					},
 					ReportData: &v2.ComplianceScanConfigurationStatus{
 						Id:       scanConfigID,


Reviews:
user: login: mtodor
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T16:12:02Z
user: login: mtodor
body: Sorry, I forgot to change to approve in the previous review. All my comments are not changing behaviour - they are all nitpicks.
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T16:15:57Z
user: login: lvalerom
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T16:45:57Z
user: login: lvalerom
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T16:46:41Z
user: login: lvalerom
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T16:46:55Z
user: login: mtodor
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-28T08:41:10Z

Review comments:
diff_hunk: @@ -520,6 +523,18 @@ func convertStorageSnapshotToV2Snapshot(ctx context.Context, snapshot *storage.C
 			ErrorMsg:                 snapshot.GetReportStatus().GetErrorMsg(),
 			ReportRequestType:        storageReportRequestTypeToV2[snapshot.GetReportStatus().GetReportRequestType()],
 			ReportNotificationMethod: storageReportNotificationMethodToV2[snapshot.GetReportStatus().GetReportNotificationMethod()],
+			FailedClusters: func() []*v2.FailedCluster {
path: central/complianceoperator/v2/scanconfigurations/service/convert.go
user: login: mtodor
body: nitpick: I would suggest moving this anonymous function to a default private package function. It will make the code a bit more readable.

And we can simply call here:
```suggestion
			FailedClusters: getApiFailedClusters(snapshot)
```
_please consider better naming from what I have suggested_
created_at: 2025-05-27T16:00:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -520,6 +523,18 @@ func convertStorageSnapshotToV2Snapshot(ctx context.Context, snapshot *storage.C
 			ErrorMsg:                 snapshot.GetReportStatus().GetErrorMsg(),
 			ReportRequestType:        storageReportRequestTypeToV2[snapshot.GetReportStatus().GetReportRequestType()],
 			ReportNotificationMethod: storageReportNotificationMethodToV2[snapshot.GetReportStatus().GetReportNotificationMethod()],
+			FailedClusters: func() []*v2.FailedCluster {
+				var ret []*v2.FailedCluster
+				for _, cluster := range snapshot.GetFailedClusters() {
+					ret = append(ret, &v2.FailedCluster{
+						ClusterId:       cluster.GetClusterId(),
+						ClusterName:     cluster.GetClusterName(),
+						OperatorVersion: cluster.GetOperatorVersion(),
+						Reason:          strings.Join(cluster.GetReasons(), ", "),
path: central/complianceoperator/v2/scanconfigurations/service/convert.go
user: login: mtodor
body: I'm not familiar with this - is `,` usually used to join error messages? My thinking: `,` is a common punctuation mark in a sentence, and it can be part of an error message. If we join with `,` it can be a bit odd to read multiple errors.

Should we maybe consider `;` or something else?

**EDIT:** maybe someone from the UI team can provide a suggestion.
created_at: 2025-05-27T16:10:51Z
author_association: CONTRIBUTOR
diff_hunk: @@ -730,6 +731,14 @@ func (s *ComplianceScanConfigServiceTestSuite) TestGetReportHistory() {
 					StartedAt:                now,
 					CompletedAt:              now,
 				},
+				FailedClusters: []*storage.ComplianceOperatorReportSnapshotV2_FailedCluster{
+					{
+						ClusterId:       "cluster-1",
+						ClusterName:     "cluster-1",
+						OperatorVersion: "v1.6.0",
+						Reasons:         []string{report.INTERNAL_ERROR},
path: central/complianceoperator/v2/scanconfigurations/service/service_impl_test.go
user: login: mtodor
body: Please consider adding a case for multiple error messages.
created_at: 2025-05-27T16:11:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -520,6 +523,18 @@ func convertStorageSnapshotToV2Snapshot(ctx context.Context, snapshot *storage.C
 			ErrorMsg:                 snapshot.GetReportStatus().GetErrorMsg(),
 			ReportRequestType:        storageReportRequestTypeToV2[snapshot.GetReportStatus().GetReportRequestType()],
 			ReportNotificationMethod: storageReportNotificationMethodToV2[snapshot.GetReportStatus().GetReportNotificationMethod()],
+			FailedClusters: func() []*v2.FailedCluster {
path: central/complianceoperator/v2/scanconfigurations/service/convert.go
user: login: lvalerom
body: Done
created_at: 2025-05-27T16:45:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -520,6 +523,18 @@ func convertStorageSnapshotToV2Snapshot(ctx context.Context, snapshot *storage.C
 			ErrorMsg:                 snapshot.GetReportStatus().GetErrorMsg(),
 			ReportRequestType:        storageReportRequestTypeToV2[snapshot.GetReportStatus().GetReportRequestType()],
 			ReportNotificationMethod: storageReportNotificationMethodToV2[snapshot.GetReportStatus().GetReportNotificationMethod()],
+			FailedClusters: func() []*v2.FailedCluster {
+				var ret []*v2.FailedCluster
+				for _, cluster := range snapshot.GetFailedClusters() {
+					ret = append(ret, &v2.FailedCluster{
+						ClusterId:       cluster.GetClusterId(),
+						ClusterName:     cluster.GetClusterName(),
+						OperatorVersion: cluster.GetOperatorVersion(),
+						Reason:          strings.Join(cluster.GetReasons(), ", "),
path: central/complianceoperator/v2/scanconfigurations/service/convert.go
user: login: lvalerom
body: Good point. I don't think any of the error messages have `,` but nevertheless I think it's better to change it to `; `.
created_at: 2025-05-27T16:46:41Z
author_association: CONTRIBUTOR
diff_hunk: @@ -730,6 +731,14 @@ func (s *ComplianceScanConfigServiceTestSuite) TestGetReportHistory() {
 					StartedAt:                now,
 					CompletedAt:              now,
 				},
+				FailedClusters: []*storage.ComplianceOperatorReportSnapshotV2_FailedCluster{
+					{
+						ClusterId:       "cluster-1",
+						ClusterName:     "cluster-1",
+						OperatorVersion: "v1.6.0",
+						Reasons:         []string{report.INTERNAL_ERROR},
path: central/complianceoperator/v2/scanconfigurations/service/service_impl_test.go
user: login: lvalerom
body: Done
created_at: 2025-05-27T16:46:55Z
author_association: CONTRIBUTOR

PR comments:
user: login: openshift-ci[bot]
created_at: 2025-05-08T07:11:42Z
body: Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with `/test all`
author_association: NONE
user: login: rhacs-bot
created_at: 2025-05-08T07:43:45Z
body: [//]: # (meta:type=build)

Images are ready for the commit at 044d0d4.

To use with deploy scripts, first `export MAIN_IMAGE_TAG=4.8.x-835-g044d0d4619`.
author_association: CONTRIBUTOR
user: login: codecov[bot]
created_at: 2025-05-08T08:20:45Z
body: ## [Codecov](https://app.codecov.io/gh/stackrox/stackrox/pull/15232?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=stackrox) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 49.26%. Comparing base [(`37eeeb6`)](https://app.codecov.io/gh/stackrox/stackrox/commit/37eeeb6496786da1c12e42e4bb960f0009283b03?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=stackrox) to head [(`044d0d4`)](https://app.codecov.io/gh/stackrox/stackrox/commit/044d0d46190a1220eb6bf13f57379c07964cb515?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=stackrox).
> Report is 4 commits behind head on master.

<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##           master   #15232   +/-   ##
=======================================
  Coverage   49.25%   49.26%           
=======================================
  Files        2581     2581           
  Lines      189326   189342   +16     
=======================================
+ Hits        93251    93272   +21     
+ Misses      88735    88731    -4     
+ Partials     7340     7339    -1     
```

| [Flag](https://app.codecov.io/gh/stackrox/stackrox/pull/15232/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=stackrox) | Coverage Δ | |
|---|---|---|
| [go-unit-tests](https://app.codecov.io/gh/stackrox/stackrox/pull/15232/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=stackrox) | `49.26% <100.00%> (+<0.01%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=stackrox#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/stackrox/stackrox/pull/15232?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=stackrox).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=stackrox).

<details><summary> :rocket: New features to boost your workflow: </summary>

- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
author_association: NONE
user: login: openshift-ci[bot]
created_at: 2025-05-28T17:13:31Z
body: @lvalerom: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
ci/prow/gke-qa-e2e-tests | 044d0d46190a1220eb6bf13f57379c07964cb515 | [link](https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/stackrox_stackrox/15232/pull-ci-stackrox-stackrox-master-gke-qa-e2e-tests/1927749348808986624) | false | `/test gke-qa-e2e-tests`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=stackrox&repo=stackrox&pr=15232). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:lvalerom).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
author_association: NONE