Repository: jenkins-x/lighthouse
Pull request #1437 (2022-03-17 22:07:18)

fix: Page through statuses when polling
This PR ensures that Lighthouse poller finds all statuses when polling instead of just using the first page

jenkins-x-bot: Hi @dippynark. Thanks for your PR.

I'm waiting for a [jenkins-x](https://github.com/orgs/jenkins-x/people) or [todo](https://github.com/orgs/todo/people) member to verify that this patch is reasonable to test. If it is, they should reply with `/ok-to-test` on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should [join the org](https://github.com/orgs/todo/people) to skip this step.

Once the patch is verified, the new status will be reflected by the `ok-to-test` label.

I understand the commands that are listed [here](https://jenkins-x.io/v3/develop/reference/chatops/?repo=jenkins-x%2Flighthouse).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [jenkins-x/lighthouse](https://github.com/jenkins-x/lighthouse/issues/new?title=Command%20issue:) repository.
</details>

jenkins-x-bot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by:
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please assign **hferentschik**
You can assign the PR to them by writing `/assign @hferentschik` in a comment when ready.

The full list of commands accepted by this bot can be found [here](https://jenkins-x.io/v3/develop/reference/chatops/?repo=jenkins-x%2Flighthouse).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/jenkins-x/lighthouse/blob/main/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["hferentschik"]} -->
tomhobson: /ok-to-test
jenkins-x-bot: @dippynark: The following test **failed**, say `/retest` to rerun them all:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pr | af1fbff051a5262eda3ae00979ce7ae2a7316c66 | [link](http://dashboard-jx.infra.jenkins-x.rocks/namespaces/jx/pipelineruns/jenkins-x-lighthouse-pr-1437-pr-gm5sg) | `/test`



<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [jenkins-x/lighthouse](https://github.com/jenkins-x/lighthouse/issues/new?title=Command%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
jenkins-x-bot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by:
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please assign **garethjevans**
You can assign the PR to them by writing `/assign @garethjevans` in a comment when ready.

The full list of commands accepted by this bot can be found [here](https://jenkins-x.io/v3/develop/reference/chatops/?repo=jenkins-x%2Flighthouse).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/jenkins-x/lighthouse/blob/main/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["garethjevans"]} -->
ankitm123: Can u rebase with the latest master? Else ur tests wont start (because of a recent tekton upgrade)
jenkins-x-bot: @dippynark: The following test **failed**, say `/retest` to rerun them all:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pr | 447b0a3695db14470095b0695f2b61a2fb8f1bc1 | [link](http://dashboard-jx.infra.jenkins-x.rocks/namespaces/jx/pipelineruns/jenkins-x-lighthouse-pr-1437-pr-vff6k) | `/test`



<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [jenkins-x/lighthouse](https://github.com/jenkins-x/lighthouse/issues/new?title=Command%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
jenkins-x-bot: @dippynark: The following test **failed**, say `/retest` to rerun them all:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pr | 447b0a3695db14470095b0695f2b61a2fb8f1bc1 | [link](http://dashboard-jx.infra.jenkins-x.rocks/namespaces/jx/pipelineruns/jenkins-x-lighthouse-pr-1437-pr-vff6k) | `/test`



<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [jenkins-x/lighthouse](https://github.com/jenkins-x/lighthouse/issues/new?title=Command%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->

Patch:
diff --git a/pkg/poller/poller.go b/pkg/poller/poller.go
index e09cb5d4..5305982c 100644
--- a/pkg/poller/poller.go
+++ b/pkg/poller/poller.go
@@ -17,6 +17,7 @@ import (
 
 var (
 	censor = func(content []byte) []byte { return content }
+	StatusesPageSize = 100
 )
 
 type pollingController struct {
@@ -283,10 +284,7 @@ func (c *pollingController) pollPullRequestPushHook(ctx context.Context, l *logr
 }
 
 func (c *pollingController) hasStatusForSHA(ctx context.Context, l *logrus.Entry, fullName string, sha string, isRelease bool) (bool, error) {
-	opts := scm.ListOptions{
-		Page: 1,
-	}
-	statuses, _, err := c.scmClient.Repositories.ListStatus(ctx, fullName, sha, opts)
+	statuses, err := c.ListAllStatuses(ctx, fullName, sha)
 	if err != nil {
 		return false, errors.Wrapf(err, "failed to list status")
 	}
@@ -299,6 +297,28 @@ func (c *pollingController) hasStatusForSHA(ctx context.Context, l *logrus.Entry
 	return false, nil
 }
 
+func (c *pollingController) ListAllStatuses(ctx context.Context, fullName string, sha string) ([]*scm.Status, error) {
+	allStatuses := []*scm.Status{}
+	page := 1
+	for {
+		opts := scm.ListOptions{
+			Page: page,
+			Size: StatusesPageSize,
+		}
+		statuses, response, err := c.scmClient.Repositories.ListStatus(ctx, fullName, sha, opts)
+		if err != nil {
+			return allStatuses, err
+		}
+		allStatuses = append(allStatuses, statuses...)
+		// Check for an invalid value for the next page
+		if response == nil || response.Page.Next <= page {
+			break
+		}
+		page = response.Page.Next
+	}
+	return allStatuses, nil
+}
+
 func (c *pollingController) isMatchingStatus(s *scm.Status, isRelease bool) bool {
 	if isRelease && c.requireReleaseSuccess {
 		if s.State != scm.StateSuccess {


Test patch:
diff --git a/pkg/poller/poller_test.go b/pkg/poller/poller_test.go
index e0b15fb6..0fb33a5d 100644
--- a/pkg/poller/poller_test.go
+++ b/pkg/poller/poller_test.go
@@ -1,10 +1,12 @@
 package poller_test
 
 import (
+	"fmt"
 	"os/exec"
 	"regexp"
 	"strings"
 	"testing"
+	"context"
 
 	scmfake "github.com/jenkins-x/go-scm/scm/driver/fake"
 
@@ -124,3 +126,25 @@ func TestPollerPullRequests(t *testing.T) {
 	assert.NotNil(t, pushHook, "no PushHook")
 	t.Logf("created PushHook %#v", pushHook)
 }
+
+func TestListAllStatuses(t *testing.T) {
+	sha := "mysha1234"
+	fullName := repoNames[0]
+
+	scmClient, fakeData := scmfake.NewDefault()
+	fakeData.Statuses = map[string][]*scm.Status{sha: {}}
+	// Populate fake data with more entries than the page size to ensure there
+	// are multiple pages of data
+	statusesCount := poller.StatusesPageSize + 1
+	for i := 0; i < statusesCount; i++ {
+		status := scm.Status{}
+		fakeData.Statuses[sha] = append(fakeData.Statuses[sha], &status)
+	}
+
+	p, err := poller.NewPollingController(repoNames, gitServer, scmClient, nil, requireReleaseSuccess, nil, nil)
+	require.NoError(t, err, "failed to create PollingController")
+
+	statuses, err := p.ListAllStatuses(context.TODO(), fullName, sha)
+	assert.Nil(t, err, "failed to list statuses")
+	require.Len(t, statuses, statusesCount, fmt.Sprintf("should have %d statuses", statusesCount))
+}