Repository: kubesphere/ks-devops
Pull request #214 (2021-09-02 00:40:36)

Support multi-branch pipeline build and fetching build
### What this PR dose

- Support multi-branch pipeline build and fetching build

### Why we need it

This PR is part of #41

### Special notes

Upstream PR is https://github.com/jenkins-zh/jenkins-client/pull/16.

- [x] Bump jenkins-zh/jenkins-client dependency while upstream PR is merged.

### Step to test

```bash
kubectl apply -f config/samples/devops_v1alpha4_pipelinerun_multi_branch_demo.yaml
watch kubectl -n demo-devopsproject get pipelinerun demo-pipelinerun -o yaml
```

/kind feature

ks-ci-bot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubesphere/ks-devops/pull/214#" title="Author self-approved">JohnNiang</a>*
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please assign **linuxsuren** after the PR has been reviewed.
You can assign the PR to them by writing `/assign @linuxsuren` in a comment when ready.

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubesphere%2Fks-devops).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubesphere/ks-devops/blob/master/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["linuxsuren"]} -->
ks-ci-bot: @JohnNiang: PR needs rebase.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
codecov[bot]: # [Codecov](https://codecov.io/gh/kubesphere/ks-devops/pull/214?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) Report
> Merging [#214](https://codecov.io/gh/kubesphere/ks-devops/pull/214?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) (57fe606) into [master](https://codecov.io/gh/kubesphere/ks-devops/commit/db195546d263e5fafa6829108bbca8ea982adc34?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) (db19554) will **increase** coverage by `0.00%`.
> The diff coverage is `4.54%`.

[![Impacted file tree graph](https://codecov.io/gh/kubesphere/ks-devops/pull/214/graphs/tree.svg?width=650&height=150&src=pr&token=qSK4RBbyNg&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere)](https://codecov.io/gh/kubesphere/ks-devops/pull/214?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere)

```diff
@@          Coverage Diff           @@
##           master    #214   +/-   ##
======================================
  Coverage    5.69%   5.69%           
======================================
  Files          80      80           
  Lines       21399   21416   +17     
======================================
+ Hits         1218    1219    +1     
- Misses      20108   20124   +16     
  Partials       73      73           
```

| Flag | Coverage Δ | |
|---|---|---|
| unittests | `5.69% <4.54%> (+<0.01%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/kubesphere/ks-devops/pull/214?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere) | Coverage Δ | |
|---|---|---|
| [...lers/jenkins/pipelinerun/pipelinerun\_controller.go](https://codecov.io/gh/kubesphere/ks-devops/pull/214/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-Y29udHJvbGxlcnMvamVua2lucy9waXBlbGluZXJ1bi9waXBlbGluZXJ1bl9jb250cm9sbGVyLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/api/devops/v1alpha4/pipelinerun\_types.go](https://codecov.io/gh/kubesphere/ks-devops/pull/214/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere#diff-cGtnL2FwaS9kZXZvcHMvdjFhbHBoYTQvcGlwZWxpbmVydW5fdHlwZXMuZ28=) | `9.67% <100.00%> (+3.22%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/kubesphere/ks-devops/pull/214?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kubesphere/ks-devops/pull/214?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere). Last update [db19554...57fe606](https://codecov.io/gh/kubesphere/ks-devops/pull/214?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kubesphere).


Patch:
diff --git a/config/samples/devops_v1alpha4_pipelinerun_multi_branch_demo.yaml b/config/samples/devops_v1alpha4_pipelinerun_multi_branch_demo.yaml
new file mode 100644
index 0000000..e171561
--- /dev/null
+++ b/config/samples/devops_v1alpha4_pipelinerun_multi_branch_demo.yaml
@@ -0,0 +1,53 @@
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: demo-devopsproject
+---
+apiVersion: devops.kubesphere.io/v1alpha3
+kind: DevOpsProject
+metadata:
+  annotations:
+    kubesphere.io/creator: admin
+  name: demo-devopsproject
+status:
+  adminNamespace: demo-devopsproject
+
+---
+apiVersion: devops.kubesphere.io/v1alpha3
+kind: Pipeline
+metadata:
+  annotations:
+    kubesphere.io/creator: admin
+  name: demo-multi-branch-pipeline
+  namespace: demo-devopsproject
+spec:
+  multi_branch_pipeline:
+    discarder:
+      days_to_keep: '-1'
+      num_to_keep: '-1'
+    git_source:
+      discover_branches: true
+      git_clone_option:
+        depth: 1
+        timeout: 20
+      url: 'https://gitlab.com/johnniang/jenkinsfile-demo.git'
+    name: demo-multi-branch-pipeline
+    script_path: Jenkinsfile
+    source_type: git
+  type: multi-branch-pipeline
+
+---
+apiVersion: devops.kubesphere.io/v1alpha4
+kind: PipelineRun
+metadata:
+  name: demo-pipelinerun
+  namespace: demo-devopsproject
+spec:
+  pipelineRef:
+    name: demo-multi-branch-pipeline
+  pipelineSpec:
+    type: multi-branch-pipeline
+  scm:
+    refName: main
+    refType: branch
diff --git a/controllers/jenkins/pipelinerun/pipelinerun_controller.go b/controllers/jenkins/pipelinerun/pipelinerun_controller.go
index 492dc42..dd7a3fa 100644
--- a/controllers/jenkins/pipelinerun/pipelinerun_controller.go
+++ b/controllers/jenkins/pipelinerun/pipelinerun_controller.go
@@ -161,25 +161,52 @@ func (r *Reconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
 		return ctrl.Result{}, err
 	}
 
-	// requeue immediately
+	// requeue after 1 second
 	return ctrl.Result{RequeueAfter: 1 * time.Second}, nil
 }
 
-func (r *Reconciler) triggerJenkinsJob(devopsProjectName, pipelineName string, prSpec *devopsv1alpha4.PipelineRunSpec) (*job.PipelineBuild, error) {
-	boClient := job.BlueOceanClient{JenkinsCore: r.JenkinsCore, Organization: "jenkins"}
-	return boClient.Build(job.BuildOption{
-		Pipelines:  []string{devopsProjectName, pipelineName},
+func (r *Reconciler) triggerJenkinsJob(projectName, pipelineName string, prSpec *devopsv1alpha4.PipelineRunSpec) (*job.PipelineBuild, error) {
+	c := job.BlueOceanClient{JenkinsCore: r.JenkinsCore, Organization: "jenkins"}
+
+	branch, err := getBranch(prSpec)
+	if err != nil {
+		return nil, err
+	}
+
+	return c.Build(job.BuildOption{
+		Pipelines:  []string{projectName, pipelineName},
 		Parameters: parameterConverter{parameters: prSpec.Parameters}.convert(),
+		Branch:     branch,
 	})
 }
 
+func getBranch(prSpec *devopsv1alpha4.PipelineRunSpec) (string, error) {
+	var branch = ""
+	if prSpec.IsMultiBranchPipeline() {
+		if prSpec.SCM == nil || prSpec.SCM.RefName == "" {
+			return "", fmt.Errorf("failed to obtain SCM reference name for multi-branch Pipeline")
+		}
+		branch = prSpec.SCM.RefName
+	}
+	return branch, nil
+}
+
 func (r *Reconciler) getPipelineRunResult(projectName, pipelineName string, pr *devopsv1alpha4.PipelineRun) (*job.PipelineBuild, error) {
 	runIDStr, exists := pr.GetPipelineRunID()
 	if !exists {
 		return nil, fmt.Errorf("unable to get PipelineRun result due to not found run ID")
 	}
 	boClient := job.BlueOceanClient{JenkinsCore: r.JenkinsCore, Organization: "jenkins"}
-	return boClient.GetBuild(runIDStr, projectName, pipelineName)
+
+	branch, err := getBranch(&pr.Spec)
+	if err != nil {
+		return nil, err
+	}
+	return boClient.GetBuild(job.GetBuildOption{
+		RunID:     runIDStr,
+		Pipelines: []string{projectName, pipelineName},
+		Branch:    branch,
+	})
 }
 
 func (r *Reconciler) updateLabelsAndAnnotations(ctx context.Context, pr *devopsv1alpha4.PipelineRun) error {
diff --git a/go.mod b/go.mod
index 89ea74d..9d24993 100644
--- a/go.mod
+++ b/go.mod
@@ -21,7 +21,7 @@ require (
 	github.com/golang/mock v1.6.0
 	github.com/google/go-cmp v0.5.5
 	github.com/google/go-querystring v1.1.0 // indirect
-	github.com/jenkins-zh/jenkins-client v0.0.3-0.20210830140903-1ae131e1f28b
+	github.com/jenkins-zh/jenkins-client v0.0.3-0.20210902064041-1fcb0fa89b38
 	github.com/kubesphere/sonargo v0.0.2
 	github.com/onsi/ginkgo v1.16.4
 	github.com/onsi/gomega v1.15.0
diff --git a/go.sum b/go.sum
index 02415ae..8c827ff 100644
--- a/go.sum
+++ b/go.sum
@@ -385,8 +385,8 @@ github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANyt
 github.com/influxdata/influxdb1-client v0.0.0-20191209144304-8bf82d3c094d/go.mod h1:qj24IKcXYK6Iy9ceXlo3Tc+vtHo9lIhSX5JddghvEPo=
 github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
 github.com/jenkins-zh/jenkins-cli v0.0.32/go.mod h1:uE1mH9PNITrg0sugv6HXuM/CSddg0zxXoYu3w57I3JY=
-github.com/jenkins-zh/jenkins-client v0.0.3-0.20210830140903-1ae131e1f28b h1:+5d4wUkSGy/Ql5/ZBDAPjXA+Aab39V3nvw5Ec6TCHx8=
-github.com/jenkins-zh/jenkins-client v0.0.3-0.20210830140903-1ae131e1f28b/go.mod h1:ICBk7OOoTafVP//f/VfKZ34c0ff8vJwVnOsF9btiMYU=
+github.com/jenkins-zh/jenkins-client v0.0.3-0.20210902064041-1fcb0fa89b38 h1:0b4F5kF+Kx295haTJH3pogw9NcpSuGfIkcKo8N98964=
+github.com/jenkins-zh/jenkins-client v0.0.3-0.20210902064041-1fcb0fa89b38/go.mod h1:ICBk7OOoTafVP//f/VfKZ34c0ff8vJwVnOsF9btiMYU=
 github.com/jenkins-zh/jenkins-formulas v0.0.5/go.mod h1:zS8fm8u5L6FcjZM0QznXsLV9T2UtSVK+hT6Sm76iUZ4=
 github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
 github.com/jessevdk/go-flags v1.5.0/go.mod h1:Fw0T6WPc1dYxT4mKEZRfG5kJhaTDP9pj1c2EWnYs/m4=
diff --git a/pkg/api/devops/v1alpha4/pipelinerun_types.go b/pkg/api/devops/v1alpha4/pipelinerun_types.go
index 17884e1..4d6af66 100644
--- a/pkg/api/devops/v1alpha4/pipelinerun_types.go
+++ b/pkg/api/devops/v1alpha4/pipelinerun_types.go
@@ -153,7 +153,7 @@ func (pr *PipelineRun) Buildable() bool {
 
 // IsMultiBranchPipeline indicates if the PipelineRun belongs a multi-branch pipeline.
 func (prSpec *PipelineRunSpec) IsMultiBranchPipeline() bool {
-	return prSpec.SCM != nil && len(prSpec.SCM.RefName) > 0
+	return prSpec.PipelineSpec != nil && prSpec.PipelineSpec.Type == v1alpha3.MultiBranchPipelineType
 }
 
 func (pr *PipelineRun) GetPipelineRunID() (pipelineRunID string, exist bool) {


Test patch:
diff --git a/controllers/jenkins/pipelinerun/pipelinerun_controller_test.go b/controllers/jenkins/pipelinerun/pipelinerun_controller_test.go
index 0a6b50f..f4fec70 100644
--- a/controllers/jenkins/pipelinerun/pipelinerun_controller_test.go
+++ b/controllers/jenkins/pipelinerun/pipelinerun_controller_test.go
@@ -1,1 +1,80 @@
 package pipelinerun
+
+import (
+	"kubesphere.io/devops/pkg/api/devops/v1alpha3"
+	devopsv1alpha4 "kubesphere.io/devops/pkg/api/devops/v1alpha4"
+	"testing"
+)
+
+func Test_getBranch(t *testing.T) {
+	type args struct {
+		prSpec *devopsv1alpha4.PipelineRunSpec
+	}
+	tests := []struct {
+		name    string
+		args    args
+		want    string
+		wantErr bool
+	}{{
+		name: "No SCM Pipeline",
+		args: args{
+			prSpec: &devopsv1alpha4.PipelineRunSpec{
+				PipelineSpec: &v1alpha3.PipelineSpec{
+					Type: v1alpha3.NoScmPipelineType,
+				},
+			},
+		},
+		want: "",
+	}, {
+		name: "No SCM Pipeline but SCM set",
+		args: args{
+			prSpec: &devopsv1alpha4.PipelineRunSpec{
+				PipelineSpec: &v1alpha3.PipelineSpec{
+					Type: v1alpha3.NoScmPipelineType,
+				},
+				SCM: &devopsv1alpha4.SCM{
+					RefName: "main",
+					RefType: "branch",
+				},
+			},
+		},
+		want: "",
+	}, {
+		name: "Multi-branch Pipeline but not SCM set",
+		args: args{
+			prSpec: &devopsv1alpha4.PipelineRunSpec{
+				PipelineSpec: &v1alpha3.PipelineSpec{
+					Type: v1alpha3.MultiBranchPipelineType,
+				},
+			},
+		},
+		wantErr: true,
+	}, {
+		name: "Multi-branch Pipeline and SCM set",
+		args: args{
+			prSpec: &devopsv1alpha4.PipelineRunSpec{
+				PipelineSpec: &v1alpha3.PipelineSpec{
+					Type: v1alpha3.MultiBranchPipelineType,
+				},
+				SCM: &devopsv1alpha4.SCM{
+					RefName: "main",
+					RefType: "branch",
+				},
+			},
+		},
+		want: "main",
+	},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			got, err := getBranch(tt.args.prSpec)
+			if (err != nil) != tt.wantErr {
+				t.Errorf("getBranch() error = %v, wantErr %v", err, tt.wantErr)
+				return
+			}
+			if got != tt.want {
+				t.Errorf("getBranch() got = %v, want %v", got, tt.want)
+			}
+		})
+	}
+}
diff --git a/pkg/api/devops/v1alpha4/pipelinerun_types_test.go b/pkg/api/devops/v1alpha4/pipelinerun_types_test.go
index ee6f131..a5b32ac 100644
--- a/pkg/api/devops/v1alpha4/pipelinerun_types_test.go
+++ b/pkg/api/devops/v1alpha4/pipelinerun_types_test.go
@@ -1,1 +1,55 @@
 package v1alpha4
+
+import (
+	v1 "k8s.io/api/core/v1"
+	"kubesphere.io/devops/pkg/api/devops/v1alpha3"
+	"testing"
+)
+
+func TestPipelineRunSpec_IsMultiBranchPipeline(t *testing.T) {
+	type fields struct {
+		PipelineRef  *v1.ObjectReference
+		PipelineSpec *v1alpha3.PipelineSpec
+		Parameters   []Parameter
+		SCM          *SCM
+	}
+	tests := []struct {
+		name   string
+		fields fields
+		want   bool
+	}{{
+		name: "No SCM Pipeline",
+		fields: fields{
+			PipelineSpec: &v1alpha3.PipelineSpec{
+				Type: v1alpha3.NoScmPipelineType,
+			},
+		},
+		want: false,
+	}, {
+		name:   "No PipelineSpec",
+		fields: fields{},
+		want:   false,
+	}, {
+		name: "SCM Pipeline",
+		fields: fields{
+			PipelineSpec: &v1alpha3.PipelineSpec{
+				Type: v1alpha3.MultiBranchPipelineType,
+			},
+		},
+		want: true,
+	},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			prSpec := &PipelineRunSpec{
+				PipelineRef:  tt.fields.PipelineRef,
+				PipelineSpec: tt.fields.PipelineSpec,
+				Parameters:   tt.fields.Parameters,
+				SCM:          tt.fields.SCM,
+			}
+			if got := prSpec.IsMultiBranchPipeline(); got != tt.want {
+				t.Errorf("IsMultiBranchPipeline() = %v, want %v", got, tt.want)
+			}
+		})
+	}
+}