Repository: runatlantis/atlantis (language: go)

Issue:
"GitHub Allow Mergable Bypass Apply" Broken for Fork PRs
<!--- Please keep this note for the community --->

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
* Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
* If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

### Overview of the Issue

`--gh-allow-mergeable-bypass-apply` as described here: https://www.runatlantis.io/docs/apply-requirements.html#github

Seems broken for *FORK* PRs but works fine with *BRANCH* PRs.

### Reproduction Steps

Repo has `atlantis/apply` as a branch protection mergeable requirement. Open up a branch PR and have it work correctly. Open up a Fork PR and see warning and error in the logs (and the PR is not able to be merged until the branch protection is edited).

### Logs

<details>
  <summary>Logs</summary>

The following warning then error come in the logs:

Error
```
Unable to check pull mergeable status, error: getting pull request status: getting combined status: GET https://api.github.com/repos/ORG/REPO/commits/BRANCH_NAME_FROM_FORK/status: 404 Ref not found []
```

Stacktrace
```
github.com/runatlantis/atlantis/server/events/vcs.(*InstrumentedClient).PullIsMergeable
	github.com/runatlantis/atlantis/server/events/vcs/instrumented_client.go:183
github.com/runatlantis/atlantis/server/events/vcs.(*ClientProxy).PullIsMergeable
	github.com/runatlantis/atlantis/server/events/vcs/proxy.go:72
github.com/runatlantis/atlantis/server/events/vcs.(*pullReqStatusFetcher).FetchPullStatus
	github.com/runatlantis/atlantis/server/events/vcs/pull_status_fetcher.go:28
github.com/runatlantis/atlantis/server/events.(*ApplyCommandRunner).Run
	github.com/runatlantis/atlantis/server/events/apply_command_runner.go:108
github.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand
	github.com/runatlantis/atlantis/server/events/command_runner.go:296
```

Warning
```
unable to get pull request status: fetching mergeability status for repo: ORG/REPO, and pull number: 573: getting pull request status: getting combined status: GET https://api.github.com/repos/ORG/REPO/commits/BRANCH_NAME_FROM_FORK/status: 404 Ref not found []. Continuing with mergeable and approved assumed false
```

Stacktrace
```
github.com/runatlantis/atlantis/server/events.(*ApplyCommandRunner).Run
	github.com/runatlantis/atlantis/server/events/apply_command_runner.go:114
github.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand
	github.com/runatlantis/atlantis/server/events/command_runner.go:296
```

</details>


### Environment details

- Atlantis version: 0.19.9
- Atlantis flags: 
- ENV ATLANTIS_GH_ALLOW_MERGEABLE_BYPASS_APPLY=true



### Additional Context

I tried figuring out the Go code but I hit `githubPR.GetMergeableState()` which does not seem to be defined anywhere?

https://github.com/runatlantis/atlantis/blob/64ce8601a00def31400d60c5d740c3633ce6dccc/server/events/vcs/github_client.go#L382

The GitHub API it is failing on is https://docs.github.com/en/rest/commits/statuses#get-the-combined-status-for-a-specific-reference
The issue is that the commit ref does not exist in the repository the PR is against, but is coming from the fork.


Thanks
"GitHub Allow Mergable Bypass Apply" Broken for Fork PRs
<!--- Please keep this note for the community --->

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
* Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
* If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

### Overview of the Issue

`--gh-allow-mergeable-bypass-apply` as described here: https://www.runatlantis.io/docs/apply-requirements.html#github

Seems broken for *FORK* PRs but works fine with *BRANCH* PRs.

### Reproduction Steps

Repo has `atlantis/apply` as a branch protection mergeable requirement. Open up a branch PR and have it work correctly. Open up a Fork PR and see warning and error in the logs (and the PR is not able to be merged until the branch protection is edited).

### Logs

<details>
  <summary>Logs</summary>

The following warning then error come in the logs:

Error
```
Unable to check pull mergeable status, error: getting pull request status: getting combined status: GET https://api.github.com/repos/ORG/REPO/commits/BRANCH_NAME_FROM_FORK/status: 404 Ref not found []
```

Stacktrace
```
github.com/runatlantis/atlantis/server/events/vcs.(*InstrumentedClient).PullIsMergeable
	github.com/runatlantis/atlantis/server/events/vcs/instrumented_client.go:183
github.com/runatlantis/atlantis/server/events/vcs.(*ClientProxy).PullIsMergeable
	github.com/runatlantis/atlantis/server/events/vcs/proxy.go:72
github.com/runatlantis/atlantis/server/events/vcs.(*pullReqStatusFetcher).FetchPullStatus
	github.com/runatlantis/atlantis/server/events/vcs/pull_status_fetcher.go:28
github.com/runatlantis/atlantis/server/events.(*ApplyCommandRunner).Run
	github.com/runatlantis/atlantis/server/events/apply_command_runner.go:108
github.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand
	github.com/runatlantis/atlantis/server/events/command_runner.go:296
```

Warning
```
unable to get pull request status: fetching mergeability status for repo: ORG/REPO, and pull number: 573: getting pull request status: getting combined status: GET https://api.github.com/repos/ORG/REPO/commits/BRANCH_NAME_FROM_FORK/status: 404 Ref not found []. Continuing with mergeable and approved assumed false
```

Stacktrace
```
github.com/runatlantis/atlantis/server/events.(*ApplyCommandRunner).Run
	github.com/runatlantis/atlantis/server/events/apply_command_runner.go:114
github.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand
	github.com/runatlantis/atlantis/server/events/command_runner.go:296
```

</details>


### Environment details

- Atlantis version: 0.19.9
- Atlantis flags: 
- ENV ATLANTIS_GH_ALLOW_MERGEABLE_BYPASS_APPLY=true



### Additional Context

I tried figuring out the Go code but I hit `githubPR.GetMergeableState()` which does not seem to be defined anywhere?

https://github.com/runatlantis/atlantis/blob/64ce8601a00def31400d60c5d740c3633ce6dccc/server/events/vcs/github_client.go#L382

The GitHub API it is failing on is https://docs.github.com/en/rest/commits/statuses#get-the-combined-status-for-a-specific-reference
The issue is that the commit ref does not exist in the repository the PR is against, but is coming from the fork.


Thanks


Discussion / hints:
I think I had the same issue when I tried to use it, but did not identify it was a fork vs branch problem. Now going back looking at the problem PR it was indeed created from a fork.

To give more information, when I tried to `atlantis apply` on a forked PR that is approved and already satisfy the merge requirement indicated by GitHub, Atlantis responded the following:

```
Ran Apply for project: <project_name> dir: <dir_name> workspace: default

Apply Failed: Pull request must be approved by at least one person other than the author before running apply.
```

not sure if others are seeing the same or a different error message.
and FYI #2436 is the PR that added this feature.
I can confirm this issue is happening on the latest version of Atlantis `v0.20.1`
pain
I was also able to reproduce this issue on a fork PR. I am working on a fix and I plan to create a PR.
I think I had the same issue when I tried to use it, but did not identify it was a fork vs branch problem. Now going back looking at the problem PR it was indeed created from a fork.

To give more information, when I tried to `atlantis apply` on a forked PR that is approved and already satisfy the merge requirement indicated by GitHub, Atlantis responded the following:

```
Ran Apply for project: <project_name> dir: <dir_name> workspace: default

Apply Failed: Pull request must be approved by at least one person other than the author before running apply.
```

not sure if others are seeing the same or a different error message.
and FYI #2436 is the PR that added this feature.
I can confirm this issue is happening on the latest version of Atlantis `v0.20.1`
pain
I was also able to reproduce this issue on a fork PR. I am working on a fix and I plan to create a PR.

Fix patch:
diff --git a/server/events/vcs/github_client.go b/server/events/vcs/github_client.go
index 7f67cb3dcc..aa64dc8cbc 100644
--- a/server/events/vcs/github_client.go
+++ b/server/events/vcs/github_client.go
@@ -397,7 +397,7 @@ func isRequiredCheck(check string, required []string) bool {
 // GetCombinedStatusMinusApply checks Statuses for PR, excluding atlantis apply. Returns true if all other statuses are not in failure.
 func (g *GithubClient) GetCombinedStatusMinusApply(repo models.Repo, pull *github.PullRequest, vcstatusname string) (bool, error) {
 	//check combined status api
-	status, _, err := g.client.Repositories.GetCombinedStatus(g.ctx, repo.Owner, repo.Name, *pull.Head.Ref, nil)
+	status, _, err := g.client.Repositories.GetCombinedStatus(g.ctx, *pull.Head.Repo.Owner.Login, repo.Name, *pull.Head.Ref, nil)
 	if err != nil {
 		return false, errors.Wrap(err, "getting combined status")
 	}
@@ -419,7 +419,7 @@ func (g *GithubClient) GetCombinedStatusMinusApply(repo models.Repo, pull *githu
 	}
 
 	//check check suite/check run api
-	checksuites, _, err := g.client.Checks.ListCheckSuitesForRef(context.Background(), repo.Owner, repo.Name, *pull.Head.Ref, nil)
+	checksuites, _, err := g.client.Checks.ListCheckSuitesForRef(context.Background(), *pull.Head.Repo.Owner.Login, repo.Name, *pull.Head.Ref, nil)
 	if err != nil {
 		return false, errors.Wrap(err, "getting check suites for ref")
 	}
@@ -428,7 +428,7 @@ func (g *GithubClient) GetCombinedStatusMinusApply(repo models.Repo, pull *githu
 	for _, c := range checksuites.CheckSuites {
 		if *c.Status == "completed" {
 			//iterate over the runs inside the suite
-			suite, _, err := g.client.Checks.ListCheckRunsCheckSuite(context.Background(), repo.Owner, repo.Name, *c.ID, nil)
+			suite, _, err := g.client.Checks.ListCheckRunsCheckSuite(context.Background(), *pull.Head.Repo.Owner.Login, repo.Name, *c.ID, nil)
 			if err != nil {
 				return false, errors.Wrap(err, "getting check runs for check suite")
 			}


Test patch:
diff --git a/server/events/vcs/github_client_test.go b/server/events/vcs/github_client_test.go
index c1c30f03d4..b84de928df 100644
--- a/server/events/vcs/github_client_test.go
+++ b/server/events/vcs/github_client_test.go
@@ -692,19 +692,19 @@ func TestGithubClient_PullIsMergeableWithAllowMergeableBypassApply(t *testing.T)
 			testServer := httptest.NewTLSServer(
 				http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 					switch r.RequestURI {
-					case "/api/v3/repos/owner/repo/pulls/1":
+					case "/api/v3/repos/octocat/repo/pulls/1":
 						w.Write([]byte(response)) // nolint: errcheck
 						return
-					case "/api/v3/repos/owner/repo/pulls/1/reviews?per_page=300":
+					case "/api/v3/repos/octocat/repo/pulls/1/reviews?per_page=300":
 						w.Write([]byte("[]")) // nolint: errcheck
 						return
-					case "/api/v3/repos/owner/repo/commits/new-topic/status":
+					case "/api/v3/repos/octocat/repo/commits/new-topic/status":
 						w.Write([]byte(commitJSON)) // nolint: errcheck
 					case "/api/graphql":
 						w.Write([]byte(reviewDecision)) // nolint: errcheck
-					case "/api/v3/repos/owner/repo/branches/main/protection":
+					case "/api/v3/repos/octocat/repo/branches/main/protection":
 						w.Write([]byte(branchProtectionJSON)) // nolint: errcheck
-					case "/api/v3/repos/owner/repo/commits/new-topic/check-suites":
+					case "/api/v3/repos/octocat/repo/commits/new-topic/check-suites":
 						w.Write([]byte(checkSuites)) // nolint: errcheck
 					default:
 						t.Errorf("got unexpected request at %q", r.RequestURI)
@@ -719,8 +719,8 @@ func TestGithubClient_PullIsMergeableWithAllowMergeableBypassApply(t *testing.T)
 			defer disableSSLVerification()()
 
 			actMergeable, err := client.PullIsMergeable(models.Repo{
-				FullName:          "owner/repo",
-				Owner:             "owner",
+				FullName:          "octocat/repo",
+				Owner:             "octocat",
 				Name:              "repo",
 				CloneURL:          "",
 				SanitizedCloneURL: "",