Repository: kubesphere/ks-devops
Pull request #517 (2022-03-25 06:28:51)

Provide applications summary API
### What type of PR is this?

/kind feature

### What this PR does / why we need it:

Provide applications summary API. The request and response look like below:

**Request URL**
```bash
http://localhost:9090/kapis/gitops.kubesphere.io/v1alpha1/namespaces/{namepsace}/applications/-/summary
```

**Response Body**

```json
{
  "total": 1,
  "healthStatus": {
    "Healthy": 1
  },
  "syncStatus": {
    "Synced": 1
  }
}
```

### Which issue(s) this PR fixes:
<!--
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
Please leave it or change # to be None if there is no corresponding issue that exists
-->
Fixes #

### Special notes for reviewers:

Please check the following list before waiting reviewers:

- [ ] Already committed the CRD files to [the Helm Chart](https://github.com/kubesphere-sigs/ks-devops-helm-chart/) if you created some new CRDs
- [ ] Already [added the permission](https://github.com/kubesphere/ks-installer/blob/9e063b085a0e43fdb3d0d9e3e7f4149146f14b9c/roles/ks-core/prepare/files/ks-init/role-templates.yaml) for the new API
- [ ] Already added the RBAC markers for the new controllers

### Does this PR introduce a user-facing change??

```release-note
None
```
/cc @kubesphere/sig-devops 


ks-ci-bot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubesphere/ks-devops/pull/517#" title="Author self-approved">JohnNiang</a>*
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please assign **linuxsuren** after the PR has been reviewed.
You can assign the PR to them by writing `/assign @linuxsuren` in a comment when ready.

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubesphere%2Fks-devops).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubesphere/ks-devops/blob/master/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["linuxsuren"]} -->
codecov[bot]: # [Codecov](https://codecov.io/gh/kubesphere/ks-devops/pull/517?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) Report
> Merging [#517](https://codecov.io/gh/kubesphere/ks-devops/pull/517?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) (86ca6aa) into [master](https://codecov.io/gh/kubesphere/ks-devops/commit/7d1a226539089e81d0a856c1bc94b0b393b9b4c0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) (7d1a226) will **decrease** coverage by `0.04%`.
> The diff coverage is `73.91%`.

```diff
@@            Coverage Diff             @@
##           master     #517      +/-   ##
==========================================
- Coverage   77.67%   77.62%   -0.05%     
==========================================
  Files         140      140              
  Lines       22845    22868      +23     
==========================================
+ Hits        17745    17752       +7     
- Misses       4909     4920      +11     
- Partials      191      196       +5     
```

| Flag | Coverage Δ | |
|---|---|---|
| unittests | `77.62% <73.91%> (-0.05%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/kubesphere/ks-devops/pull/517?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) | Coverage Δ | |
|---|---|---|
| [pkg/kapis/gitops/v1alpha1/argocd/handler.go](https://codecov.io/gh/kubesphere/ks-devops/pull/517/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-cGtnL2thcGlzL2dpdG9wcy92MWFscGhhMS9hcmdvY2QvaGFuZGxlci5nbw==) | `89.65% <66.66%> (-6.00%)` | :arrow_down: |
| [pkg/kapis/gitops/v1alpha1/argocd/route.go](https://codecov.io/gh/kubesphere/ks-devops/pull/517/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-cGtnL2thcGlzL2dpdG9wcy92MWFscGhhMS9hcmdvY2Qvcm91dGUuZ28=) | `100.00% <100.00%> (ø)` | |
| [...ontrollers/jenkins/pipeline/pipeline\_controller.go](https://codecov.io/gh/kubesphere/ks-devops/pull/517/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-Y29udHJvbGxlcnMvamVua2lucy9waXBlbGluZS9waXBlbGluZV9jb250cm9sbGVyLmdv) | `39.39% <0.00%> (-3.79%)` | :arrow_down: |
| [.../jenkins/devopsproject/devopsproject\_controller.go](https://codecov.io/gh/kubesphere/ks-devops/pull/517/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-Y29udHJvbGxlcnMvamVua2lucy9kZXZvcHNwcm9qZWN0L2Rldm9wc3Byb2plY3RfY29udHJvbGxlci5nbw==) | `40.62% <0.00%> (-2.61%)` | :arrow_down: |
| [pkg/kapis/devops/v1alpha3/register.go](https://codecov.io/gh/kubesphere/ks-devops/pull/517/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-cGtnL2thcGlzL2Rldm9wcy92MWFscGhhMy9yZWdpc3Rlci5nbw==) | `100.00% <0.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/kubesphere/ks-devops/pull/517?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kubesphere/ks-devops/pull/517?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere). Last update [7d1a226...86ca6aa](https://codecov.io/gh/kubesphere/ks-devops/pull/517?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere).


Patch:
diff --git a/controllers/argocd/application_controller.go b/controllers/argocd/application_controller.go
index 2f763d1..2970a07 100644
--- a/controllers/argocd/application_controller.go
+++ b/controllers/argocd/application_controller.go
@@ -22,7 +22,6 @@ import (
 	"github.com/go-logr/logr"
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
 	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
-	"k8s.io/apimachinery/pkg/runtime/schema"
 	"k8s.io/apimachinery/pkg/types"
 	"k8s.io/client-go/tools/record"
 	"kubesphere.io/devops/pkg/api/gitops/v1alpha1"
@@ -61,9 +60,7 @@ func (r *ApplicationReconciler) Reconcile(req ctrl.Request) (result ctrl.Result,
 func (r *ApplicationReconciler) reconcileArgoApplication(app *v1alpha1.Application) (err error) {
 	ctx := context.Background()
 
-	argoApp := &unstructured.Unstructured{}
-	argoApp.SetKind("Application")
-	argoApp.SetAPIVersion("argoproj.io/v1alpha1")
+	argoApp := createBareArgoCDApplicationObject()
 
 	if err = r.Get(ctx, types.NamespacedName{
 		Namespace: app.GetNamespace(),
@@ -105,12 +102,7 @@ func createUnstructuredApplication(app *v1alpha1.Application) (result *unstructu
 		argoApp.Spec.Destination.Server = ""
 	}
 
-	newArgoApp := &unstructured.Unstructured{}
-	newArgoApp.SetGroupVersionKind(schema.GroupVersionKind{
-		Group:   "argoproj.io",
-		Version: "v1alpha1",
-		Kind:    "Application",
-	})
+	newArgoApp := createBareArgoCDApplicationObject()
 	newArgoApp.SetName(app.GetName())
 	newArgoApp.SetNamespace(app.GetNamespace())
 
diff --git a/controllers/argocd/argocd-application-status-controller.go b/controllers/argocd/argocd-application-status-controller.go
index 33aa611..da39981 100644
--- a/controllers/argocd/argocd-application-status-controller.go
+++ b/controllers/argocd/argocd-application-status-controller.go
@@ -21,9 +21,11 @@ import (
 	"encoding/json"
 	"github.com/go-logr/logr"
 	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
+	"k8s.io/apimachinery/pkg/runtime/schema"
 	"k8s.io/apimachinery/pkg/types"
 	"k8s.io/client-go/tools/record"
 	"kubesphere.io/devops/pkg/api/gitops/v1alpha1"
+	"reflect"
 	ctrl "sigs.k8s.io/controller-runtime"
 	"sigs.k8s.io/controller-runtime/pkg/client"
 )
@@ -57,8 +59,8 @@ func (r *ApplicationStatusReconciler) Reconcile(req ctrl.Request) (result ctrl.R
 	var status map[string]interface{}
 	if status, _, err = unstructured.NestedMap(argoCDApp.Object, "status"); err == nil {
 		var statusData []byte
-		if statusData, err = json.Marshal(status); err == nil {
-			app = app.DeepCopy()
+		if statusData, err = json.Marshal(status); err == nil && !reflect.DeepEqual([]byte(app.Status.ArgoApp), statusData) {
+			//app = app.DeepCopy()
 			if app.GetLabels() == nil {
 				// make sure the labels are not nil
 				app.SetLabels(map[string]string{})
@@ -71,15 +73,29 @@ func (r *ApplicationStatusReconciler) Reconcile(req ctrl.Request) (result ctrl.R
 			if healthStatus, found, _ := unstructured.NestedString(status, "health", "status"); found {
 				app.GetLabels()[v1alpha1.HealthStatusLabelKey] = healthStatus
 			}
+
+			// unset operation field if it was absent
+			if _, found, err := unstructured.NestedMap(argoCDApp.Object, "operation"); err != nil {
+				return ctrl.Result{}, err
+			} else if !found && app.Spec.ArgoApp != nil {
+				app.Spec.ArgoApp.Operation = nil
+			}
+
+			// update labels
+			if err = r.Update(ctx, app); err != nil {
+				return
+			}
+
+			// update status subresource
 			app.Status.ArgoApp = string(statusData)
-			err = r.Update(ctx, app)
+			err = r.Status().Update(ctx, app)
 		}
 	}
 	return
 }
 
 func getArgoCDApplication(client client.Reader, namespacedName types.NamespacedName) (app *unstructured.Unstructured, err error) {
-	app = getArgoCDApplicationObject()
+	app = createBareArgoCDApplicationObject()
 
 	if err = client.Get(context.Background(), namespacedName, app); err != nil {
 		app = nil
@@ -97,19 +113,22 @@ func (r *ApplicationStatusReconciler) GetGroupName() string {
 	return controllerGroupName
 }
 
-func getArgoCDApplicationObject() *unstructured.Unstructured {
-	cluster := &unstructured.Unstructured{}
-	cluster.SetKind("Application")
-	cluster.SetAPIVersion("argoproj.io/v1alpha1")
-	return cluster.DeepCopy()
+func createBareArgoCDApplicationObject() *unstructured.Unstructured {
+	argoApp := &unstructured.Unstructured{}
+	argoApp.SetGroupVersionKind(schema.GroupVersionKind{
+		Group:   "argoproj.io",
+		Version: "v1alpha1",
+		Kind:    "Application",
+	})
+	return argoApp
 }
 
 // SetupWithManager init the logger, recorder and filters
 func (r *ApplicationStatusReconciler) SetupWithManager(mgr ctrl.Manager) error {
-	cluster := getArgoCDApplicationObject()
+	argoApp := createBareArgoCDApplicationObject()
 	r.log = ctrl.Log.WithName(r.GetName())
 	r.recorder = mgr.GetEventRecorderFor(r.GetName())
 	return ctrl.NewControllerManagedBy(mgr).
-		For(cluster).
+		For(argoApp).
 		Complete(r)
 }
diff --git a/pkg/kapis/gitops/v1alpha1/argocd/handler.go b/pkg/kapis/gitops/v1alpha1/argocd/handler.go
index b906cf1..a7755c0 100644
--- a/pkg/kapis/gitops/v1alpha1/argocd/handler.go
+++ b/pkg/kapis/gitops/v1alpha1/argocd/handler.go
@@ -135,6 +135,40 @@ func (h *handler) getClusters(req *restful.Request, res *restful.Response) {
 	common.Response(req, res, argoClusters, err)
 }
 
+func (h *handler) applicationSummary(request *restful.Request, response *restful.Response) {
+	namespace := common.GetPathParameter(request, common.NamespacePathParameter)
+
+	summary, err := h.populateApplicationSummary(namespace)
+	common.Response(request, response, summary, err)
+}
+
+func (h *handler) populateApplicationSummary(namespace string) (*ApplicationsSummary, error) {
+	applicationList := &v1alpha1.ApplicationList{}
+	if err := h.List(context.Background(), applicationList, client.InNamespace(namespace)); err != nil {
+		return nil, err
+	}
+	summary := &ApplicationsSummary{
+		HealthStatus: map[string]int{},
+		SyncStatus:   map[string]int{},
+	}
+	summary.Total = len(applicationList.Items)
+
+	for i := range applicationList.Items {
+		app := &applicationList.Items[i]
+		// accumulate health status
+		healthStatus := app.GetLabels()[v1alpha1.HealthStatusLabelKey]
+		if healthStatus != "" {
+			summary.HealthStatus[healthStatus]++
+		}
+		// accumulate sync status
+		syncStatus := app.GetLabels()[v1alpha1.SyncStatusLabelKey]
+		if syncStatus != "" {
+			summary.SyncStatus[syncStatus]++
+		}
+	}
+	return summary, nil
+}
+
 type handler struct {
 	client.Client
 }
diff --git a/pkg/kapis/gitops/v1alpha1/argocd/route.go b/pkg/kapis/gitops/v1alpha1/argocd/route.go
index 40389d2..90d1fb9 100644
--- a/pkg/kapis/gitops/v1alpha1/argocd/route.go
+++ b/pkg/kapis/gitops/v1alpha1/argocd/route.go
@@ -37,6 +37,13 @@ type ApplicationPageResult struct {
 	TotalItems int                    `json:"totalItems"`
 }
 
+// ApplicationsSummary is the model of application summary response.
+type ApplicationsSummary struct {
+	Total        int            `json:"total"`
+	HealthStatus map[string]int `json:"healthStatus"`
+	SyncStatus   map[string]int `json:"syncStatus"`
+}
+
 // RegisterRoutes is for registering Argo CD Application routes into WebService.
 func RegisterRoutes(service *restful.WebService, options *common.Options) {
 	handler := newHandler(options)
@@ -53,6 +60,12 @@ func RegisterRoutes(service *restful.WebService, options *common.Options) {
 		Doc("Search applications").
 		Returns(http.StatusOK, api.StatusOK, ApplicationPageResult{}))
 
+	service.Route(service.GET("/namespaces/{namespace}/application-summary").
+		To(handler.applicationSummary).
+		Param(common.NamespacePathParameter).
+		Doc("Fetch applications summary").
+		Returns(http.StatusOK, api.StatusOK, ApplicationsSummary{}))
+
 	service.Route(service.POST("/namespaces/{namespace}/applications").
 		To(handler.createApplication).
 		Param(common.NamespacePathParameter).


Test patch:
diff --git a/controllers/argocd/argocd-application-status-controller_test.go b/controllers/argocd/argocd-application-status-controller_test.go
index 85abde7..35df0eb 100644
--- a/controllers/argocd/argocd-application-status-controller_test.go
+++ b/controllers/argocd/argocd-application-status-controller_test.go
@@ -99,7 +99,7 @@ func Test_getArgoCDApplicationObject(t *testing.T) {
 	}}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
-			tt.verify(t, getArgoCDApplicationObject())
+			tt.verify(t, createBareArgoCDApplicationObject())
 		})
 	}
 }
diff --git a/pkg/kapis/gitops/v1alpha1/argocd/route_test.go b/pkg/kapis/gitops/v1alpha1/argocd/route_test.go
index a0397d0..45dd23c 100644
--- a/pkg/kapis/gitops/v1alpha1/argocd/route_test.go
+++ b/pkg/kapis/gitops/v1alpha1/argocd/route_test.go
@@ -55,7 +55,7 @@ func TestRegisterRoutes(t *testing.T) {
 			options: &common.Options{GenericClient: fake.NewFakeClientWithScheme(schema)},
 		},
 		verify: func(t *testing.T, service *restful.WebService) {
-			assert.Equal(t, 6, len(service.Routes()))
+			assert.Greater(t, len(service.Routes()), 0)
 		},
 	}}
 	for _, tt := range tests {
@@ -76,6 +76,10 @@ func TestAPIs(t *testing.T) {
 		ObjectMeta: metav1.ObjectMeta{
 			Namespace: "ns",
 			Name:      "app",
+			Labels: map[string]string{
+				v1alpha1.HealthStatusLabelKey: "Healthy",
+				v1alpha1.SyncStatusLabelKey:   "Synced",
+			},
 		},
 	}
 
@@ -303,6 +307,17 @@ func TestAPIs(t *testing.T) {
  }
 ]`, string(body))
 		},
+	}, {
+		name: "get applications summary",
+		request: request{
+			method: http.MethodGet,
+			uri:    "/namespaces/ns/application-summary",
+		},
+		k8sclient:    fake.NewFakeClientWithScheme(schema, app.DeepCopy()),
+		responseCode: http.StatusOK,
+		verify: func(t *testing.T, body []byte) {
+			assert.JSONEq(t, `{"total": 1, "healthStatus": { "Healthy": 1 }, "syncStatus": { "Synced": 1 }}`, string(body))
+		},
 	}}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {