Repository: kubesphere/ks-devops
Pull request #658 (2022-06-10 08:19:57)

Support to render step templates via CLI
<!-- Thanks for sending a pull request! Here are some tips for you:

1. If you want **faster** PR reviews, read how: https://github.com/kubesphere/community/blob/master/developer-guide/development/the-pr-author-guide-to-getting-through-code-review.md
2. In case you want to know how your PR got reviewed, read: https://github.com/kubesphere/community/blob/master/developer-guide/development/code-review-guide.md
3. Here are some coding conventions followed by the KubeSphere community: https://github.com/kubesphere/community/blob/master/developer-guide/development/coding-conventions.md
4. Additional open-source best practice: https://github.com/LinuxSuRen/open-source-best-practice
-->

### What type of PR is this?
<!-- 
Add one of the following kinds:
/kind bug
/kind cleanup
/kind documentation
/kind design
/kind chore

Optionally add one or more of the following kinds if applicable:
/kind api-change
/kind deprecation
/kind failing-test
/kind flake
/kind regression
-->
/kind feature

### What this PR does / why we need it:

### Which issue(s) this PR fixes:
<!--
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
Please leave it or change # to be None if there is no corresponding issue that exists
-->
Fixes #

### Special notes for reviewers:
<!--
You can use the following command to let the DevOps SIG members help you to review your PR.
/cc @kubesphere/sig-devops 
And please avoid cc any individual.
-->
Please check the following list before waiting reviewers:

- [ ] Already committed the CRD files to [the Helm Chart](https://github.com/kubesphere-sigs/ks-devops-helm-chart/) if you created some new CRDs
- [ ] Already [added the permission](https://github.com/kubesphere/ks-installer/blob/9e063b085a0e43fdb3d0d9e3e7f4149146f14b9c/roles/ks-core/prepare/files/ks-init/role-templates.yaml) for the new API
- [ ] Already added the RBAC markers for the new controllers

### Does this PR introduce a user-facing change??
<!--
If no, just write "None" in the release-note block below.
If yes, a release note is required:
Enter your extended-release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".

Please keep the note be same as your PR title if you believe it should be in the release notes.
-->
```release-note
None
```


ks-ci-bot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by:
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please ask for approval from **linuxsuren** after the PR has been reviewed.

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubesphere%2Fks-devops).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubesphere/ks-devops/blob/master/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["linuxsuren"]} -->
codecov[bot]: # [Codecov](https://codecov.io/gh/kubesphere/ks-devops/pull/658?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) Report
> Merging [#658](https://codecov.io/gh/kubesphere/ks-devops/pull/658?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) (fb849cf) into [master](https://codecov.io/gh/kubesphere/ks-devops/commit/90a6c5fe64b5fe55d894355a4ddd600738d56c92?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) (90a6c5f) will **decrease** coverage by `0.02%`.
> The diff coverage is `92.59%`.

```diff
@@            Coverage Diff             @@
##           master     #658      +/-   ##
==========================================
- Coverage   77.55%   77.52%   -0.03%     
==========================================
  Files         153      154       +1     
  Lines       23527    23529       +2     
==========================================
- Hits        18246    18241       -5     
- Misses       5055     5060       +5     
- Partials      226      228       +2     
```

| Flag | Coverage Δ | |
|---|---|---|
| unittests | `77.52% <92.59%> (-0.03%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/kubesphere/ks-devops/pull/658?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) | Coverage Δ | |
|---|---|---|
| [pkg/api/devops/v1alpha3/steptemplate\_types.go](https://codecov.io/gh/kubesphere/ks-devops/pull/658/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-cGtnL2FwaS9kZXZvcHMvdjFhbHBoYTMvc3RlcHRlbXBsYXRlX3R5cGVzLmdv) | `100.00% <ø> (ø)` | |
| [pkg/api/devops/v1alpha3/steptemplate\_render.go](https://codecov.io/gh/kubesphere/ks-devops/pull/658/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-cGtnL2FwaS9kZXZvcHMvdjFhbHBoYTMvc3RlcHRlbXBsYXRlX3JlbmRlci5nbw==) | `92.45% <92.45%> (ø)` | |
| [pkg/kapis/devops/v1alpha3/steptemplate/handler.go](https://codecov.io/gh/kubesphere/ks-devops/pull/658/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-cGtnL2thcGlzL2Rldm9wcy92MWFscGhhMy9zdGVwdGVtcGxhdGUvaGFuZGxlci5nbw==) | `82.50% <100.00%> (-7.61%)` | :arrow_down: |
| [...ontrollers/jenkins/pipeline/pipeline\_controller.go](https://codecov.io/gh/kubesphere/ks-devops/pull/658/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-Y29udHJvbGxlcnMvamVua2lucy9waXBlbGluZS9waXBlbGluZV9jb250cm9sbGVyLmdv) | `39.39% <0.00%> (-3.79%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/kubesphere/ks-devops/pull/658?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kubesphere/ks-devops/pull/658?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere). Last update [90a6c5f...fb849cf](https://codecov.io/gh/kubesphere/ks-devops/pull/658?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere).


Patch:
diff --git a/Makefile b/Makefile
index 03cb26e..4b3fd99 100644
--- a/Makefile
+++ b/Makefile
@@ -122,6 +122,12 @@ docker-build-push: docker-build-push-apiserver docker-build-push-controller
 atest:
 	atest run -p 'test/api/*.yaml'
 
+build-tpl:
+	mkdir -p bin
+	go build -o bin/tpl cmd/tools/tpl/*.go
+copy-tpl: build-tpl
+	cp bin/tpl /usr/local/bin/
+
 swagger-ui:
 	git clone https://github.com/swagger-api/swagger-ui -b v2.2.10 --depth 1 bin/swagger-ui
 
diff --git a/cmd/tools/tpl/README.md b/cmd/tools/tpl/README.md
new file mode 100644
index 0000000..3eaa48b
--- /dev/null
+++ b/cmd/tools/tpl/README.md
@@ -0,0 +1,19 @@
+This is a command to render Pipeline and step templates.
+
+See the templates from [kubesphere-sigs/pipeline-templates](https://github.com/kubesphere-sigs/pipeline-templates/).
+
+## Get started
+
+You can build and copy this command to system path:
+
+```shell
+make build-tpl
+```
+
+## TODO
+If you're interested in this tool, please feel free to consider creating the following features with us:
+
+* Validation for the template resource YAML file
+* Provide parameters select when rendering a step template
+* Support to render the Pipeline template
+* Provide a wizard to create Step or Pipeline template
diff --git a/cmd/tools/tpl/render.go b/cmd/tools/tpl/render.go
new file mode 100644
index 0000000..22f3599
--- /dev/null
+++ b/cmd/tools/tpl/render.go
@@ -0,0 +1,78 @@
+/*
+Copyright 2022 The KubeSphere Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package main
+
+import (
+	"fmt"
+	"github.com/spf13/cobra"
+	"gopkg.in/yaml.v2"
+	"io/ioutil"
+	v1 "k8s.io/api/core/v1"
+	"kubesphere.io/devops/pkg/api/devops/v1alpha3"
+	"path/filepath"
+)
+
+type renderOption struct {
+	pattern string
+}
+
+func createRenderCommand() (cmd *cobra.Command) {
+	opt := &renderOption{}
+	cmd = &cobra.Command{
+		Use:     "render",
+		Short:   "Render Pipeline and Step templates",
+		Aliases: []string{"r"},
+		RunE:    opt.runE,
+	}
+
+	flags := cmd.Flags()
+	flags.StringVarP(&opt.pattern, "pattern", "p", "*.yaml",
+		"The template file path pattern")
+	return
+}
+
+func (o *renderOption) runE(cmd *cobra.Command, args []string) (err error) {
+	var files []string
+	if files, err = filepath.Glob(o.pattern); err != nil {
+		err = fmt.Errorf("failed to find file with pattern: %s, error: %v", o.pattern, err)
+		return
+	}
+
+	for i := range files {
+		item := files[i]
+
+		var data []byte
+		if data, err = ioutil.ReadFile(item); err != nil {
+			err = fmt.Errorf("failed to read file: %s, error %v", item, err)
+			return
+		}
+
+		stepTemplate := &v1alpha3.ClusterStepTemplate{}
+		if err = yaml.Unmarshal(data, stepTemplate); err != nil {
+			err = fmt.Errorf("failed to parse ClusterStepTemplate from file: %s, error %v", item, err)
+			return
+		}
+
+		var output string
+		if output, err = stepTemplate.Spec.Render(map[string]string{}, &v1.Secret{}); err != nil {
+			err = fmt.Errorf("failed to render ClusterStepTemplate from file: %s, error %v", item, err)
+			return
+		}
+		cmd.Println(output)
+	}
+	return
+}
diff --git a/cmd/tools/tpl/root.go b/cmd/tools/tpl/root.go
new file mode 100644
index 0000000..92964d5
--- /dev/null
+++ b/cmd/tools/tpl/root.go
@@ -0,0 +1,33 @@
+/*
+Copyright 2022 The KubeSphere Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package main
+
+import (
+	"github.com/spf13/cobra"
+	"os"
+)
+
+func main() {
+	cmd := &cobra.Command{
+		Use: "tpl",
+	}
+	cmd.SetOut(os.Stdout)
+	cmd.AddCommand(createRenderCommand())
+	if err := cmd.Execute(); err != nil {
+		os.Exit(1)
+	}
+}
diff --git a/pkg/api/devops/v1alpha3/steptemplate_render.go b/pkg/api/devops/v1alpha3/steptemplate_render.go
new file mode 100644
index 0000000..084396a
--- /dev/null
+++ b/pkg/api/devops/v1alpha3/steptemplate_render.go
@@ -0,0 +1,112 @@
+/*
+Copyright 2022 The KubeSphere Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package v1alpha3
+
+import (
+	"bytes"
+	"fmt"
+	v1 "k8s.io/api/core/v1"
+	"strings"
+	"text/template"
+)
+
+// Render renders the template and returns the result
+func (t *StepTemplateSpec) Render(param map[string]string, secret *v1.Secret) (output string, err error) {
+	// taking the default parameter values
+	for i := range t.Parameters {
+		item := t.Parameters[i]
+		if _, ok := param[item.Name]; !ok && item.DefaultValue != "" {
+			param[item.Name] = item.DefaultValue
+		}
+	}
+
+	switch t.Runtime {
+	case "dsl":
+		output, err = dslRender(t.Template, param, secret)
+	case "shell":
+		fallthrough
+	default:
+		output, err = shellRender(t.Template, param, secret)
+	}
+
+	if t.Secret.Wrap {
+		output = wrapWithCredential(t.Secret.Type, secret.Name, output)
+	}
+
+	if err == nil && t.Container != "" {
+		output = fmt.Sprintf(`container("%s") {
+%s
+}`, t.Container, addIndent(output))
+	}
+	return
+}
+
+func addIndent(txt string) string {
+	items := strings.Split(txt, "\n")
+	return "\t" + strings.Join(items, "\n\t")
+}
+
+func wrapWithCredential(secretType, secretName, target string) string {
+	switch secretType {
+	case string(v1.SecretTypeBasicAuth), string(SecretTypeBasicAuth):
+		return fmt.Sprintf(`withCredential[usernamePassword(credentialsId : "%s" ,passwordVariable : 'PASSWORDVARIABLE' ,usernameVariable : 'USERNAMEVARIABLE')]) {
+%s
+}`, secretName, addIndent(target))
+	case string(v1.SecretTypeBootstrapToken), string(SecretTypeSecretText):
+		return fmt.Sprintf(`withCredential[string(credentialsId : "%s" ,variable : 'VARIABLE')]) {
+%s
+}`, secretName, addIndent(target))
+	case string(SecretTypeKubeConfig):
+		return fmt.Sprintf(`withCredential[kubeconfigContent(credentialsId : "%s" ,variable : 'VARIABLE')]) {
+%s
+}`, secretName, addIndent(target))
+	case string(SecretTypeSSHAuth):
+		return fmt.Sprintf(`withCredential[sshUserPrivateKey(credentialsId : "%s" ,keyFileVariable : 'KEYFILEVARIABLE' ,passphraseVariable : 'PASSPHRASEVARIABLE' ,usernameVariable : 'SSHUSERPRIVATEKEY')]) {
+%s
+}`, secretName, addIndent(target))
+	}
+	return target
+}
+
+func dslRender(dslTpl string, param map[string]string, secret *v1.Secret) (output string, err error) {
+	output = dslTpl
+
+	var tpl *template.Template
+	if tpl, err = template.New("shell").Parse(output); err != nil {
+		return
+	}
+
+	data := map[string]interface{}{
+		"param":  param,
+		"secret": secret,
+	}
+
+	buf := bytes.NewBuffer([]byte{})
+	if err = tpl.Execute(buf, data); err == nil {
+		output = strings.TrimSpace(buf.String())
+	}
+	return
+}
+
+func shellRender(shellTpl string, param map[string]string, secret *v1.Secret) (output string, err error) {
+	if output, err = dslRender(shellTpl, param, secret); err == nil {
+		output = fmt.Sprintf(`sh '''
+%s
+'''`, addIndent(output))
+	}
+	return
+}
diff --git a/pkg/api/devops/v1alpha3/steptemplate_types.go b/pkg/api/devops/v1alpha3/steptemplate_types.go
index 1f03b12..6189645 100644
--- a/pkg/api/devops/v1alpha3/steptemplate_types.go
+++ b/pkg/api/devops/v1alpha3/steptemplate_types.go
@@ -41,7 +41,7 @@ type ParameterInStep struct {
 	Name         string `json:"name"`
 	Required     bool   `json:"required,omitempty"`
 	Display      string `json:"display,omitempty"`
-	DefaultValue string `json:"defaultValue,omitempty"`
+	DefaultValue string `json:"defaultValue,omitempty" yaml:"defaultValue"`
 }
 
 // StepTemplateStatus defines the observed state of ClusterStepTemplate
diff --git a/pkg/client/devops/jenkins/options.go b/pkg/client/devops/jenkins/options.go
index 07e2421..967a556 100644
--- a/pkg/client/devops/jenkins/options.go
+++ b/pkg/client/devops/jenkins/options.go
@@ -71,6 +71,10 @@ func (s *Options) Validate() []error {
 		errors = append(errors, fmt.Errorf("jenkins's username or password is empty"))
 	}
 
+	if s.Password == "119d76305a05e7a2a65d096f71feb77921" {
+		errors = append(errors, fmt.Errorf("the token of the Jenkins needs to update"))
+	}
+
 	if s.MaxConnections <= 0 {
 		errors = append(errors, fmt.Errorf("jenkins's maximum connections should be greater than 0"))
 	}
diff --git a/pkg/kapis/devops/v1alpha3/scm/scmhandler.go b/pkg/kapis/devops/v1alpha3/scm/scmhandler.go
index 9c70b00..5cd2f63 100644
--- a/pkg/kapis/devops/v1alpha3/scm/scmhandler.go
+++ b/pkg/kapis/devops/v1alpha3/scm/scmhandler.go
@@ -101,7 +101,9 @@ func (h *handler) getRepositories(scm, server, org, secret, namespace string, pa
 		var listRepositoryFunc listRepository
 		if user, err = h.getCurrentUsername(c); err == nil {
 			if user == org && !strings.HasPrefix(scm, "bitbucket") {
-				listRepositoryFunc = c.Repositories.ListUser
+				listRepositoryFunc = func(ctx context.Context, s string, options goscm.ListOptions) ([]*goscm.Repository, *goscm.Response, error) {
+					return c.Repositories.List(ctx, options)
+				}
 			} else {
 				listRepositoryFunc = c.Repositories.ListOrganisation
 			}
diff --git a/pkg/kapis/devops/v1alpha3/steptemplate/handler.go b/pkg/kapis/devops/v1alpha3/steptemplate/handler.go
index e809719..5170e2c 100644
--- a/pkg/kapis/devops/v1alpha3/steptemplate/handler.go
+++ b/pkg/kapis/devops/v1alpha3/steptemplate/handler.go
@@ -17,11 +17,9 @@ limitations under the License.
 package steptemplate
 
 import (
-	"bytes"
 	"context"
 	"fmt"
 	"github.com/emicklei/go-restful"
-	"html/template"
 	v1 "k8s.io/api/core/v1"
 	"k8s.io/apimachinery/pkg/types"
 	"kubesphere.io/devops/pkg/api/devops/v1alpha3"
@@ -62,15 +60,7 @@ func (h *handler) renderClusterStepTemplate(req *restful.Request, resp *restful.
 		fmt.Printf("something goes wrong when getting secret, error: %v\n", err)
 	}
 
-	// get default param value from the template
 	param := make(map[string]string)
-	for i := range clusterStepTemplate.Spec.Parameters {
-		item := clusterStepTemplate.Spec.Parameters[i]
-		if item.DefaultValue != "" {
-			param[item.Name] = item.DefaultValue
-		}
-	}
-
 	// get the parameters from request
 	if err = req.ReadEntity(&param); err != nil {
 		// TODO considering have logger output instead of the std output
@@ -78,7 +68,7 @@ func (h *handler) renderClusterStepTemplate(req *restful.Request, resp *restful.
 	}
 
 	var output string
-	output, err = stepTemplateRender(&clusterStepTemplate.Spec, param, secret)
+	output, err = clusterStepTemplate.Spec.Render(param, secret)
 	writeResponse(map[string]string{
 		"data": output,
 	}, err, resp)
@@ -104,76 +94,3 @@ func writeResponse(object interface{}, err error, resp *restful.Response) {
 		_ = resp.WriteError(http.StatusInternalServerError, err)
 	}
 }
-
-func stepTemplateRender(stepTemplate *v1alpha3.StepTemplateSpec, param map[string]string, secret *v1.Secret) (output string, err error) {
-	switch stepTemplate.Runtime {
-	case "dsl":
-		output, err = dslRender(stepTemplate.Template, param, secret)
-	case "shell":
-		fallthrough
-	default:
-		output, err = shellRender(stepTemplate.Template, param, secret)
-	}
-
-	if stepTemplate.Secret.Wrap {
-		output = wrapWithCredential(stepTemplate.Secret.Type, secret.Name, output)
-	}
-
-	if err == nil && stepTemplate.Container != "" {
-		output = fmt.Sprintf(`container("%s") {
-	%s
-}`, stepTemplate.Container, output)
-	}
-	return
-}
-
-func wrapWithCredential(secretType, secretName, target string) string {
-	switch secretType {
-	case string(v1.SecretTypeBasicAuth), string(v1alpha3.SecretTypeBasicAuth):
-		return fmt.Sprintf(`withCredential[usernamePassword(credentialsId : "%s" ,passwordVariable : 'PASSWORDVARIABLE' ,usernameVariable : 'USERNAMEVARIABLE')]) {
-	%s
-}`, secretName, target)
-	case string(v1.SecretTypeBootstrapToken), string(v1alpha3.SecretTypeSecretText):
-		return fmt.Sprintf(`withCredential[string(credentialsId : "%s" ,variable : 'VARIABLE')]) {
-	%s
-}`, secretName, target)
-	case string(v1alpha3.SecretTypeKubeConfig):
-		return fmt.Sprintf(`withCredential[kubeconfigContent(credentialsId : "%s" ,variable : 'VARIABLE')]) {
-	%s
-}`, secretName, target)
-	case string(v1alpha3.SecretTypeSSHAuth):
-		return fmt.Sprintf(`withCredential[sshUserPrivateKey(credentialsId : "%s" ,keyFileVariable : 'KEYFILEVARIABLE' ,passphraseVariable : 'PASSPHRASEVARIABLE' ,usernameVariable : 'SSHUSERPRIVATEKEY')]) {
-	%s
-}`, secretName, target)
-	}
-	return target
-}
-
-func dslRender(dslTpl string, param map[string]string, secret *v1.Secret) (output string, err error) {
-	output = dslTpl
-
-	var tpl *template.Template
-	if tpl, err = template.New("shell").Parse(output); err != nil {
-		return
-	}
-
-	data := map[string]interface{}{
-		"param":  param,
-		"secret": secret,
-	}
-
-	buf := bytes.NewBuffer([]byte{})
-	if err = tpl.Execute(buf, data); err == nil {
-		output = buf.String()
-	}
-	return
-}
-
-func shellRender(shellTpl string, param map[string]string, secret *v1.Secret) (output string, err error) {
-	if output, err = dslRender(shellTpl, param, secret); err == nil {
-		output = fmt.Sprintf(`sh '''
-%s
-'''`, output)
-	}
-	return
-}


Test patch:
diff --git a/pkg/kapis/devops/v1alpha3/steptemplate/handler_test.go b/pkg/api/devops/v1alpha3/steptemplate_render_test.go
similarity index 77%
rename from pkg/kapis/devops/v1alpha3/steptemplate/handler_test.go
rename to pkg/api/devops/v1alpha3/steptemplate_render_test.go
index 9248448..1ee8c06 100644
--- a/pkg/kapis/devops/v1alpha3/steptemplate/handler_test.go
+++ b/pkg/api/devops/v1alpha3/steptemplate_render_test.go
@@ -14,42 +14,42 @@ See the License for the specific language governing permissions and
 limitations under the License.
 */
 
-package steptemplate
+package v1alpha3
 
 import (
+	"github.com/stretchr/testify/assert"
 	v1 "k8s.io/api/core/v1"
 	v12 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	"kubesphere.io/devops/pkg/api/devops/v1alpha3"
 	"testing"
 )
 
 func Test_handler_stepTemplateRender(t *testing.T) {
-	stepTemplate := &v1alpha3.StepTemplateSpec{
+	stepTemplate := &StepTemplateSpec{
 		Template: `echo 1`,
 		Runtime:  "shell",
 	}
-	stepTemplateWithParameters := &v1alpha3.StepTemplateSpec{
+	stepTemplateWithParameters := &StepTemplateSpec{
 		Template: `docker login -u $USERNAMEVARIABLE -p $PASSWORDVARIABLE
 docker build {{.param.context}} -t {{.param.tag}}`,
 		Runtime:   "shell",
 		Container: "base",
-		Secret: v1alpha3.SecretInStep{
+		Secret: SecretInStep{
 			Wrap: true,
 			Type: string(v1.SecretTypeBasicAuth),
 		},
-		Parameters: []v1alpha3.ParameterInStep{{
+		Parameters: []ParameterInStep{{
 			Name: "context",
 		}, {
 			Name: "tag",
 		}},
 	}
-	stepTemplateWithDSL := &v1alpha3.StepTemplateSpec{
+	stepTemplateWithDSL := &StepTemplateSpec{
 		Template: `echo "1"`,
 		Runtime:  "dsl",
 	}
 
 	type args struct {
-		stepTemplate *v1alpha3.StepTemplateSpec
+		stepTemplate *StepTemplateSpec
 		param        map[string]string
 		secret       *v1.Secret
 	}
@@ -64,7 +64,7 @@ docker build {{.param.context}} -t {{.param.tag}}`,
 			stepTemplate: stepTemplate,
 		},
 		wantOutput: `sh '''
-echo 1
+	echo 1
 '''`,
 		wantErr: false,
 	}, {
@@ -88,11 +88,11 @@ echo 1
 		},
 		wantOutput: `container("base") {
 	withCredential[usernamePassword(credentialsId : "docker" ,passwordVariable : 'PASSWORDVARIABLE' ,usernameVariable : 'USERNAMEVARIABLE')]) {
-	sh '''
-docker login -u $USERNAMEVARIABLE -p $PASSWORDVARIABLE
-docker build dir -t image:tag
-'''
-}
+		sh '''
+			docker login -u $USERNAMEVARIABLE -p $PASSWORDVARIABLE
+			docker build dir -t image:tag
+		'''
+	}
 }`,
 		wantErr: false,
 	}, {
@@ -105,7 +105,7 @@ docker build dir -t image:tag
 	}}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
-			gotOutput, err := stepTemplateRender(tt.args.stepTemplate, tt.args.param, tt.args.secret)
+			gotOutput, err := tt.args.stepTemplate.Render(tt.args.param, tt.args.secret)
 			if (err != nil) != tt.wantErr {
 				t.Errorf("stepTemplateRender() error = %v, wantErr %v", err, tt.wantErr)
 				return
@@ -137,7 +137,7 @@ func Test_wrapWithCredential(t *testing.T) {
 	}, {
 		name: "secret as kubeconfig type",
 		args: args{
-			secretType: string(v1alpha3.SecretTypeKubeConfig),
+			secretType: string(SecretTypeKubeConfig),
 			secretName: "config",
 			target:     "echo 1",
 		},
@@ -147,7 +147,7 @@ func Test_wrapWithCredential(t *testing.T) {
 	}, {
 		name: "secret as secret text type",
 		args: args{
-			secretType: string(v1alpha3.SecretTypeSecretText),
+			secretType: string(SecretTypeSecretText),
 			secretName: "config",
 			target:     "echo 1",
 		},
@@ -157,7 +157,7 @@ func Test_wrapWithCredential(t *testing.T) {
 	}, {
 		name: "secret as ssh auth type",
 		args: args{
-			secretType: string(v1alpha3.SecretTypeSSHAuth),
+			secretType: string(SecretTypeSSHAuth),
 			secretName: "config",
 			target:     "echo 1",
 		},
@@ -173,3 +173,35 @@ func Test_wrapWithCredential(t *testing.T) {
 		})
 	}
 }
+
+func Test_addIndent(t *testing.T) {
+	type args struct {
+		txt string
+	}
+	tests := []struct {
+		name string
+		args args
+		want string
+	}{{
+		name: "one line",
+		args: args{
+			txt: "one line",
+		},
+		want: "\tone line",
+	}, {
+		name: "three lines",
+		args: args{
+			txt: `one line
+two line
+three line`,
+		},
+		want: `	one line
+	two line
+	three line`,
+	}}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			assert.Equalf(t, tt.want, addIndent(tt.args.txt), "addIndent(%v)", tt.args.txt)
+		})
+	}
+}
diff --git a/pkg/kapis/devops/v1alpha3/scm/scmhandler_test.go b/pkg/kapis/devops/v1alpha3/scm/scmhandler_test.go
index 4611d3d..0c29df4 100644
--- a/pkg/kapis/devops/v1alpha3/scm/scmhandler_test.go
+++ b/pkg/kapis/devops/v1alpha3/scm/scmhandler_test.go
@@ -278,9 +278,11 @@ func TestSCMAPI(t *testing.T) {
 			}
 
 			gock.New("https://api.github.com").
-				Get("users/octocat/repos").
+				Get("/user/repos").
 				MatchParam("per_page", "1").
 				MatchParam("page", "1").
+				MatchParam("visibility", "all").
+				MatchParam("affiliation", "owner").
 				Reply(200).
 				Type("application/json").
 				SetHeaders(mockHeaders).
diff --git a/pkg/kapis/devops/v1alpha3/steptemplate/route_test.go b/pkg/kapis/devops/v1alpha3/steptemplate/route_test.go
index bcf04d9..c68691e 100644
--- a/pkg/kapis/devops/v1alpha3/steptemplate/route_test.go
+++ b/pkg/kapis/devops/v1alpha3/steptemplate/route_test.go
@@ -110,7 +110,7 @@ func TestAPIs(t *testing.T) {
 		},
 		verify: func(bytes []byte, t *testing.T) {
 			assert.Equal(t, `{
- "data": "sh '''\necho 2\n'''"
+ "data": "sh '''\n\techo 2\n'''"
 }`, string(bytes))
 		},
 		wantCode: http.StatusOK,