Repository: kubesphere/ks-devops
Pull request #791 (2022-08-19 09:11:39)

Support object in the step paramters
<!-- Thanks for sending a pull request! Here are some tips for you:

1. If you want **faster** PR reviews, read how: https://github.com/kubesphere/community/blob/master/developer-guide/development/the-pr-author-guide-to-getting-through-code-review.md
2. In case you want to know how your PR got reviewed, read: https://github.com/kubesphere/community/blob/master/developer-guide/development/code-review-guide.md
3. Here are some coding conventions followed by the KubeSphere community: https://github.com/kubesphere/community/blob/master/developer-guide/development/coding-conventions.md
4. Additional open-source best practice: https://github.com/LinuxSuRen/open-source-best-practice
-->

### What type of PR is this?
<!-- 
Add one of the following kinds:
/kind bug
/kind cleanup
/kind documentation
/kind feature
/kind design

Optionally add one or more of the following kinds if applicable:
/kind api-change
/kind deprecation
/kind failing-test
/kind flake
/kind regression
-->
/kind chore

### What this PR does / why we need it:

### Which issue(s) this PR fixes:
<!--
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
Please leave it or change # to be None if there is no corresponding issue that exists
-->
Fixes #

### Special notes for reviewers:
<!--
You can use the following command to let the DevOps SIG members help you to review your PR.
/cc @kubesphere/sig-devops 
And please avoid cc any individual.
-->
Please check the following list before waiting reviewers:

- [ ] Already committed the CRD files to [the Helm Chart](https://github.com/kubesphere-sigs/ks-devops-helm-chart/) if you created some new CRDs
- [ ] Already [added the permission](https://github.com/kubesphere/ks-installer/blob/9e063b085a0e43fdb3d0d9e3e7f4149146f14b9c/roles/ks-core/prepare/files/ks-init/role-templates.yaml) for the new API
- [ ] Already added the RBAC markers for the new controllers

### Does this PR introduce a user-facing change??
<!--
If no, just write "None" in the release-note block below.
If yes, a release note is required:
Enter your extended-release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".

Please keep the note be same as your PR title if you believe it should be in the release notes.
-->
```release-note
None
```


ks-ci-bot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by:
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please ask for approval from **linuxsuren** after the PR has been reviewed.

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubesphere%2Fks-devops).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubesphere/ks-devops/blob/master/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["linuxsuren"]} -->
codecov[bot]: # [Codecov](https://codecov.io/gh/kubesphere/ks-devops/pull/791?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) Report
> Merging [#791](https://codecov.io/gh/kubesphere/ks-devops/pull/791?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) (01f845c) into [master](https://codecov.io/gh/kubesphere/ks-devops/commit/5c830fe3e3074731eedfc72d056f2c99430029cb?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) (5c830fe) will **increase** coverage by `0.10%`.
> The diff coverage is `100.00%`.

```diff
@@            Coverage Diff             @@
##           master     #791      +/-   ##
==========================================
+ Coverage   51.04%   51.14%   +0.10%     
==========================================
  Files         171      171              
  Lines       10143    10141       -2     
==========================================
+ Hits         5177     5187      +10     
+ Misses       4676     4662      -14     
- Partials      290      292       +2     
```

| Flag | Coverage Δ | |
|---|---|---|
| unittests | `51.14% <100.00%> (+0.10%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/kubesphere/ks-devops/pull/791?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) | Coverage Δ | |
|---|---|---|
| [pkg/api/devops/v1alpha3/steptemplate\_render.go](https://codecov.io/gh/kubesphere/ks-devops/pull/791/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-cGtnL2FwaS9kZXZvcHMvdjFhbHBoYTMvc3RlcHRlbXBsYXRlX3JlbmRlci5nbw==) | `95.40% <ø> (ø)` | |
| [pkg/jwt/token/jwt.go](https://codecov.io/gh/kubesphere/ks-devops/pull/791/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-cGtnL2p3dC90b2tlbi9qd3QuZ28=) | `75.47% <ø> (+20.92%)` | :arrow_up: |
| [pkg/kapis/devops/v1alpha3/steptemplate/handler.go](https://codecov.io/gh/kubesphere/ks-devops/pull/791/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-cGtnL2thcGlzL2Rldm9wcy92MWFscGhhMy9zdGVwdGVtcGxhdGUvaGFuZGxlci5nbw==) | `80.00% <100.00%> (ø)` | |
| [...ns/devopscredential/devopscredential\_controller.go](https://codecov.io/gh/kubesphere/ks-devops/pull/791/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-Y29udHJvbGxlcnMvamVua2lucy9kZXZvcHNjcmVkZW50aWFsL2Rldm9wc2NyZWRlbnRpYWxfY29udHJvbGxlci5nbw==) | `33.33% <0.00%> (-3.41%)` | :arrow_down: |
| [.../jenkins/devopsproject/devopsproject\_controller.go](https://codecov.io/gh/kubesphere/ks-devops/pull/791/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-Y29udHJvbGxlcnMvamVua2lucy9kZXZvcHNwcm9qZWN0L2Rldm9wc3Byb2plY3RfY29udHJvbGxlci5nbw==) | `43.22% <0.00%> (+2.60%)` | :arrow_up: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere)


Patch:
diff --git a/cmd/tools/tpl/render.go b/cmd/tools/tpl/render.go
index 22f3599..dea8a11 100644
--- a/cmd/tools/tpl/render.go
+++ b/cmd/tools/tpl/render.go
@@ -18,12 +18,13 @@ package main
 
 import (
 	"fmt"
+	"io/ioutil"
+	"path/filepath"
+
 	"github.com/spf13/cobra"
 	"gopkg.in/yaml.v2"
-	"io/ioutil"
 	v1 "k8s.io/api/core/v1"
 	"kubesphere.io/devops/pkg/api/devops/v1alpha3"
-	"path/filepath"
 )
 
 type renderOption struct {
@@ -68,7 +69,7 @@ func (o *renderOption) runE(cmd *cobra.Command, args []string) (err error) {
 		}
 
 		var output string
-		if output, err = stepTemplate.Spec.Render(map[string]string{}, &v1.Secret{}); err != nil {
+		if output, err = stepTemplate.Spec.Render(map[string]interface{}{}, &v1.Secret{}); err != nil {
 			err = fmt.Errorf("failed to render ClusterStepTemplate from file: %s, error %v", item, err)
 			return
 		}
diff --git a/codecov.yml b/codecov.yml
index b2f711b..81ebe8c 100644
--- a/codecov.yml
+++ b/codecov.yml
@@ -3,6 +3,8 @@ ignore:
   - "*generated*.go"
   - "**/*generated*.go"         # glob accepted
   - "pkg/client/clientset/versioned"
+  - "pkg/client/listers"
+  - "pkg/client/informers"
 
 # https://docs.codecov.com/docs/commit-status
 # it's hard to have a coverage for some code lines
diff --git a/pkg/api/devops/v1alpha3/steptemplate_render.go b/pkg/api/devops/v1alpha3/steptemplate_render.go
index e0bb0c3..5a8cf32 100644
--- a/pkg/api/devops/v1alpha3/steptemplate_render.go
+++ b/pkg/api/devops/v1alpha3/steptemplate_render.go
@@ -20,13 +20,14 @@ import (
 	"bytes"
 	"encoding/json"
 	"fmt"
-	v1 "k8s.io/api/core/v1"
 	"strings"
 	"text/template"
+
+	v1 "k8s.io/api/core/v1"
 )
 
 // Render renders the template and returns the result
-func (t *StepTemplateSpec) Render(param map[string]string, secret *v1.Secret) (output string, err error) {
+func (t *StepTemplateSpec) Render(param map[string]interface{}, secret *v1.Secret) (output string, err error) {
 	// taking the default parameter values
 	for i := range t.Parameters {
 		item := t.Parameters[i]
@@ -114,7 +115,7 @@ func wrapWithCredential(secretType, secretName, target string) string {
 	return jsonFormat(target)
 }
 
-func dslRender(dslTpl string, param map[string]string, secret *v1.Secret) (output string, err error) {
+func dslRender(dslTpl string, param map[string]interface{}, secret *v1.Secret) (output string, err error) {
 	output = dslTpl
 
 	var tpl *template.Template
@@ -134,7 +135,7 @@ func dslRender(dslTpl string, param map[string]string, secret *v1.Secret) (outpu
 	return
 }
 
-func shellRender(shellTpl string, param map[string]string, secret *v1.Secret) (output string, err error) {
+func shellRender(shellTpl string, param map[string]interface{}, secret *v1.Secret) (output string, err error) {
 	if output, err = dslRender(shellTpl, param, secret); err == nil {
 		output = fmt.Sprintf(`{
 "arguments": [
diff --git a/pkg/jwt/token/jwt.go b/pkg/jwt/token/jwt.go
index 4ae28b6..de3f195 100644
--- a/pkg/jwt/token/jwt.go
+++ b/pkg/jwt/token/jwt.go
@@ -24,7 +24,6 @@ import (
 
 	"github.com/form3tech-oss/jwt-go"
 	"k8s.io/apiserver/pkg/authentication/user"
-	"k8s.io/klog/v2"
 )
 
 const (
@@ -93,7 +92,6 @@ func (s *jwtTokenIssuer) Verify(tokenString string) (user.Info, TokenType, error
 	// verify token signature and expiration time
 	_, err := jwt.ParseWithClaims(tokenString, clm, s.keyFunc)
 	if err != nil {
-		klog.V(4).Info(err)
 		return nil, "", err
 	}
 	return &user.DefaultInfo{Name: clm.Username, Groups: clm.Groups, Extra: clm.Extra}, clm.TokenType, nil
@@ -122,7 +120,6 @@ func (s *jwtTokenIssuer) IssueTo(user user.Info, tokenType TokenType, expiresIn
 
 	tokenString, err := token.SignedString(s.secret)
 	if err != nil {
-		klog.V(4).Info(err)
 		return "", err
 	}
 
diff --git a/pkg/kapis/devops/v1alpha3/steptemplate/handler.go b/pkg/kapis/devops/v1alpha3/steptemplate/handler.go
index e851c22..b07b051 100644
--- a/pkg/kapis/devops/v1alpha3/steptemplate/handler.go
+++ b/pkg/kapis/devops/v1alpha3/steptemplate/handler.go
@@ -19,6 +19,8 @@ package steptemplate
 import (
 	"context"
 	"fmt"
+	"net/http"
+
 	"github.com/emicklei/go-restful"
 	v1 "k8s.io/api/core/v1"
 	"k8s.io/apimachinery/pkg/runtime"
@@ -26,7 +28,6 @@ import (
 	"kubesphere.io/devops/pkg/api/devops/v1alpha3"
 	"kubesphere.io/devops/pkg/apiserver/query"
 	resourcesV1alpha3 "kubesphere.io/devops/pkg/models/resources/v1alpha3"
-	"net/http"
 )
 
 func (h *handler) clusterStepTemplates(req *restful.Request, resp *restful.Response) {
@@ -75,7 +76,7 @@ func (h *handler) renderClusterStepTemplate(req *restful.Request, resp *restful.
 		fmt.Printf("something goes wrong when getting secret, error: %v\n", err)
 	}
 
-	param := map[string]string{}
+	param := map[string]interface{}{}
 	// get the parameters from request
 	if err = req.ReadEntity(&param); err != nil {
 		// TODO considering have logger output instead of the std output


Test patch:
diff --git a/pkg/api/devops/v1alpha3/steptemplate_render_test.go b/pkg/api/devops/v1alpha3/steptemplate_render_test.go
index 1a8c30b..b429d6c 100644
--- a/pkg/api/devops/v1alpha3/steptemplate_render_test.go
+++ b/pkg/api/devops/v1alpha3/steptemplate_render_test.go
@@ -17,11 +17,13 @@ limitations under the License.
 package v1alpha3
 
 import (
-	"github.com/google/go-cmp/cmp"
 	"io/ioutil"
+	"testing"
+
+	"github.com/google/go-cmp/cmp"
+	"github.com/stretchr/testify/assert"
 	v1 "k8s.io/api/core/v1"
 	v12 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	"testing"
 )
 
 func Test_handler_stepTemplateRender(t *testing.T) {
@@ -31,7 +33,7 @@ func Test_handler_stepTemplateRender(t *testing.T) {
 	}
 	stepTemplateWithParameters := &StepTemplateSpec{
 		Template: `docker login -u $USERNAMEVARIABLE -p $PASSWORDVARIABLE
-docker build {{.param.context}} -t {{.param.tag}}`,
+docker build {{.param.context}} -t {{.param.tag}} -f {{.param.dockerfile.path}}`,
 		Runtime:   "shell",
 		Container: "base",
 		Secret: SecretInStep{
@@ -42,6 +44,8 @@ docker build {{.param.context}} -t {{.param.tag}}`,
 			Name: "context",
 		}, {
 			Name: "tag",
+		}, {
+			Name: "dockerfile",
 		}},
 	}
 	stepTemplateWithDSL := &StepTemplateSpec{
@@ -51,7 +55,7 @@ docker build {{.param.context}} -t {{.param.tag}}`,
 
 	type args struct {
 		stepTemplate *StepTemplateSpec
-		param        map[string]string
+		param        map[string]interface{}
 		secret       *v1.Secret
 	}
 	tests := []struct {
@@ -81,9 +85,12 @@ docker build {{.param.context}} -t {{.param.tag}}`,
 		name: "docker build command with parameters",
 		args: args{
 			stepTemplate: stepTemplateWithParameters,
-			param: map[string]string{
+			param: map[string]interface{}{
 				"context": "dir",
 				"tag":     "image:tag",
+				"dockerfile": map[string]string{
+					"path": "Dockerfile",
+				},
 			},
 			secret: &v1.Secret{
 				ObjectMeta: v12.ObjectMeta{
@@ -175,6 +182,11 @@ func Test_wrapWithCredential(t *testing.T) {
 	}
 }
 
+func TestJSONFormat(t *testing.T) {
+	assert.Equal(t, "abc", jsonFormat("abc"))
+	assert.Equal(t, "abc", jsonFormat(" abc "))
+}
+
 func readFile(file string) string {
 	if data, err := ioutil.ReadFile(file); err == nil {
 		return string(data)
diff --git a/pkg/api/devops/v1alpha3/testdata/docker-login.json b/pkg/api/devops/v1alpha3/testdata/docker-login.json
index 4007744..632a73c 100644
--- a/pkg/api/devops/v1alpha3/testdata/docker-login.json
+++ b/pkg/api/devops/v1alpha3/testdata/docker-login.json
@@ -16,7 +16,7 @@
               "key": "script",
               "value": {
                 "isLiteral": true,
-                "value": "docker login -u $USERNAMEVARIABLE -p $PASSWORDVARIABLE\ndocker build dir -t image:tag"
+                "value": "docker login -u $USERNAMEVARIABLE -p $PASSWORDVARIABLE\ndocker build dir -t image:tag -f Dockerfile"
               }
             }
           ],
diff --git a/pkg/jwt/token/jwt_test.go b/pkg/jwt/token/jwt_test.go
index 37832eb..3e52888 100644
--- a/pkg/jwt/token/jwt_test.go
+++ b/pkg/jwt/token/jwt_test.go
@@ -17,9 +17,11 @@ limitations under the License.
 package token
 
 import (
-	"github.com/form3tech-oss/jwt-go"
 	"testing"
 
+	"github.com/form3tech-oss/jwt-go"
+	"github.com/stretchr/testify/assert"
+
 	"github.com/google/go-cmp/cmp"
 	"k8s.io/apiserver/pkg/authentication/user"
 )
@@ -38,7 +40,6 @@ func TestTokenVerifyWithoutCacheValidate(t *testing.T) {
 	}
 
 	got, _, err := issuer.Verify(tokenString)
-
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -46,6 +47,11 @@ func TestTokenVerifyWithoutCacheValidate(t *testing.T) {
 	if diff := cmp.Diff(got, admin); diff != "" {
 		t.Error("token validate failed")
 	}
+
+	var usr user.Info
+	usr, _, err = issuer.VerifyWithoutClaimsValidation(tokenString)
+	assert.Nil(t, err)
+	assert.Equal(t, "admin", usr.GetName())
 }
 
 func Test_getUserFromClaims(t *testing.T) {