Repository: cli/cli

Issue:
gh repo sync not working
### Describe the bug

```console
$ gh repo sync alizter/dune
HTTP 404: Not Found (https://api.github.com/repos/alizter/dune/git/refs/heads/main)
```
when the repo doesn't need syncing, the command succeeds. Clicking this URL also correctly fetches the json.

NixOS
```
gh version 2.29.0 (1980-01-01)
https://github.com/cli/cli/releases/tag/v2.29.0
```


### Steps to reproduce the behavior

1. Have a repo that needs syncing.
2. Do `gh repo sync user/repo`
3. Should 404

### Expected vs actual behavior

It used to work and sync the repo.



Discussion:
@samcoe: @Alizter Thanks for writing in. We have not made any changes to that command in a long while. Do you have any estimation around when this stopped working? I am inclined to think that perhaps this is a token permission issue or a setting changed on the repo in question. I am been unable to reproduce this on any of my forked repos. @Alizter: @samcoe The issue started happening around yesterday. I suspect that it might be an issue withe the api rather than the cli, but I am unsure where to file for that.@Alizter: Today it appears to work. I would guess then somebody in the GitHub API was tweaking some things causing my queries to fail. Nothing actionable from you guys I suppose.@rjsparks: Consider reopening - I have been encountering this off-and-on for at least a week.@andrewpollock: I've also been encountering this intermittently:

```
$ gh repo sync andrewpollock/osv.dev
HTTP 404: Not Found (https://api.github.com/repos/andrewpollock/osv.dev/git/refs/heads/master)
```

Yet if I go sync my fork via the web interface, the CLI will stop failing. It's been quite vexing.@andrewpollock: In case this helps:

```
$ DEBUG=1 gh repo sync andrewpollock/osv.dev
⣾* Request at 2023-06-14 09:03:50.551435814 +1000 AEST m=+0.081041738
* Request to https://api.github.com/graphql
⣷* Request took 916.596168ms
* Request at 2023-06-14 09:03:51.469220503 +1000 AEST m=+0.998826430
* Request to https://api.github.com/repos/andrewpollock/osv.dev/merge-upstream
⣻* Request took 379.934845ms
* Request at 2023-06-14 09:03:51.849800312 +1000 AEST m=+1.379406238
* Request to https://api.github.com/graphql
⣟* Request took 312.245235ms
* Request at 2023-06-14 09:03:52.162329298 +1000 AEST m=+1.691935223
* Request to https://api.github.com/repos/google/osv.dev/git/refs/heads/master
⣷* Request took 251.153768ms
* Request at 2023-06-14 09:03:52.41393556 +1000 AEST m=+1.943541558
* Request to https://api.github.com/repos/andrewpollock/osv.dev/git/refs/heads/master
⣽* Request took 304.482351ms
HTTP 404: Not Found (https://api.github.com/repos/andrewpollock/osv.dev/git/refs/heads/master)
```@andrewpollock: After I've synced by the web UI:

```
$ DEBUG=1 gh repo sync andrewpollock/osv.dev
* Request at 2023-06-14 09:05:27.150894162 +1000 AEST m=+0.065482152
* Request to https://api.github.com/graphql
⣷* Request took 919.547876ms
* Request at 2023-06-14 09:05:28.071390589 +1000 AEST m=+0.985978644
* Request to https://api.github.com/repos/andrewpollock/osv.dev/merge-upstream
⣻* Request took 296.000559ms
✓ Synced the "andrewpollock:master" branch from "google:master"
```@samcoe: Seeing as various people have been experiencing this issue on and off, I am going to reopen this issue. I don't think I have seen any report that makes me think this is a bug in `gh` specifically but rather a platform bug. Anyone with evidence to the contrary please let me know as my only current course is to direct this to the appropriate internal platform team. 

If anyone has the ability to reproduce the error and willing to share their logs from using `GH_DEBUG=api gh repo sync` that would be appreciate in trying to determine exactly what is failing. Please be mindful to redact any private data from the logs before submitting them here.@andrewpollock: I agree that this sounds like a platform bug based on the nature of the error.@andrewpollock: Neat how it auto-redacts on paste...

```
* Request at 2023-06-14 13:18:03.941322578 +1000 AEST m=+0.069889336
* Request to https://api.github.com/graphql
> POST /graphql HTTP/1.1
> Host: api.github.com
> Accept: application/vnd.github.merge-info-preview+json, application/vnd.github.nebula-preview
> Authorization: token ████████████████████
> Content-Type: application/json; charset=utf-8
> Graphql-Features: merge_queue
> Time-Zone: Australia/Sydney
> User-Agent: GitHub CLI 2.22.1

GraphQL query:
fragment repo on Repository {
    id
    name
    owner { login }
    hasIssuesEnabled
    description
    hasWikiEnabled
    viewerPermission
    defaultBranchRef {
      name
    }
  }

  query RepositoryInfo($owner: String!, $name: String!) {
    repository(owner: $owner, name: $name) {
      ...repo
      parent {
        ...repo
      }
      mergeCommitAllowed
      rebaseMergeAllowed
      squashMergeAllowed
    }
  }
GraphQL variables: {"name":"osv.dev","owner":"andrewpollock"}

< HTTP/2.0 200 OK
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
< Content-Security-Policy: default-src 'none'
< Content-Type: application/json; charset=utf-8
< Date: Wed, 14 Jun 2023 03:18:04 GMT
< Github-Authentication-Token-Expiration: 2023-08-26 23:37:14 UTC
< Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
< Server: GitHub.com
< Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
< Vary: Accept-Encoding, Accept, X-Requested-With
< X-Accepted-Oauth-Scopes: repo
< X-Content-Type-Options: nosniff
< X-Frame-Options: deny
< X-Github-Media-Type: github.merge-info-preview; param=nebula-preview; format=json
< X-Github-Request-Id: CA64:322F:6FD734:763A8E:6489316C
< X-Oauth-Scopes: read:discussion, read:enterprise, read:gpg_key, read:org, read:packages, read:public_key, read:repo_hook, read:ssh_signing_key, repo, user
< X-Ratelimit-Limit: 5000
< X-Ratelimit-Remaining: 4993
< X-Ratelimit-Reset: 1686716250
< X-Ratelimit-Resource: graphql
< X-Ratelimit-Used: 7
< X-Xss-Protection: 0

{
  "data": {
    "repository": {
      "id": "R_kgDOHxUMsA",
      "name": "osv.dev",
      "owner": {
        "login": "andrewpollock"
      },
      "hasIssuesEnabled": false,
      "description": "Open source vulnerability DB and triage service.",
      "hasWikiEnabled": false,
      "viewerPermission": "ADMIN",
      "defaultBranchRef": {
        "name": "master"
      },
      "parent": {
        "id": "MDEwOlJlcG9zaXRvcnkzMjc3NzUyNjk=",
        "name": "osv.dev",
        "owner": {
          "login": "google"
        },
        "hasIssuesEnabled": true,
        "description": "Open source vulnerability DB and triage service.",
        "hasWikiEnabled": false,
        "viewerPermission": "ADMIN",
        "defaultBranchRef": {
          "name": "master"
        }
      },
      "mergeCommitAllowed": true,
      "rebaseMergeAllowed": true,
      "squashMergeAllowed": true
    }
  }
}

* Request took 711.467759ms
* Request at 2023-06-14 13:18:04.653540222 +1000 AEST m=+0.782107023
* Request to https://api.github.com/repos/andrewpollock/osv.dev/merge-upstream
> POST /repos/andrewpollock/osv.dev/merge-upstream HTTP/1.1
> Host: api.github.com
> Accept: application/vnd.github.merge-info-preview+json, application/vnd.github.nebula-preview
> Authorization: token ████████████████████
> Content-Type: application/json; charset=utf-8
> Time-Zone: Australia/Sydney
> User-Agent: GitHub CLI 2.22.1

{
  "branch": "master"
}

< HTTP/2.0 422 Unprocessable Entity
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
< Content-Length: 170
< Content-Security-Policy: default-src 'none'
< Content-Type: application/json; charset=utf-8
< Date: Wed, 14 Jun 2023 03:18:04 GMT
< Github-Authentication-Token-Expiration: 2023-08-26 23:37:14 UTC
< Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
< Server: GitHub.com
< Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
< Vary: Accept-Encoding, Accept, X-Requested-With
< X-Accepted-Oauth-Scopes: 
< X-Content-Type-Options: nosniff
< X-Frame-Options: deny
< X-Github-Api-Version-Selected: 2022-11-28
< X-Github-Media-Type: github.merge-info-preview; param=nebula-preview; format=json
< X-Github-Request-Id: CA64:322F:6FD751:763ABA:6489316C
< X-Oauth-Scopes: read:discussion, read:enterprise, read:gpg_key, read:org, read:packages, read:public_key, read:repo_hook, read:ssh_signing_key, repo, user
< X-Ratelimit-Limit: 5000
< X-Ratelimit-Remaining: 4990
< X-Ratelimit-Reset: 1686716251
< X-Ratelimit-Resource: core
< X-Ratelimit-Used: 10
< X-Xss-Protection: 0

{
  "message": "There was a problem updating the branch",
  "documentation_url": "https://docs.github.com/rest/branches/branches#sync-a-fork-branch-with-the-upstream-repository"
}

* Request took 432.007227ms
* Request at 2023-06-14 13:18:05.085805957 +1000 AEST m=+1.214372729
* Request to https://api.github.com/graphql
> POST /graphql HTTP/1.1
> Host: api.github.com
> Accept: application/vnd.github.merge-info-preview+json, application/vnd.github.nebula-preview
> Authorization: token ████████████████████
> Content-Type: application/json
> Graphql-Features: merge_queue
> Time-Zone: Australia/Sydney
> User-Agent: GitHub CLI 2.22.1

GraphQL query:
query RepositoryFindParent($name:String!$owner:String!){repository(owner: $owner, name: $name){parent{name,owner{login}}}}
GraphQL variables: {"name":"osv.dev","owner":"andrewpollock"}

< HTTP/2.0 200 OK
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
< Content-Security-Policy: default-src 'none'
< Content-Type: application/json; charset=utf-8
< Date: Wed, 14 Jun 2023 03:18:05 GMT
< Github-Authentication-Token-Expiration: 2023-08-26 23:37:14 UTC
< Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
< Server: GitHub.com
< Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
< Vary: Accept-Encoding, Accept, X-Requested-With
< X-Accepted-Oauth-Scopes: repo
< X-Content-Type-Options: nosniff
< X-Frame-Options: deny
< X-Github-Media-Type: github.merge-info-preview; param=nebula-preview; format=json
< X-Github-Request-Id: CA64:322F:6FD77A:763ADA:6489316D
< X-Oauth-Scopes: read:discussion, read:enterprise, read:gpg_key, read:org, read:packages, read:public_key, read:repo_hook, read:ssh_signing_key, repo, user
< X-Ratelimit-Limit: 5000
< X-Ratelimit-Remaining: 4992
< X-Ratelimit-Reset: 1686716250
< X-Ratelimit-Resource: graphql
< X-Ratelimit-Used: 8
< X-Xss-Protection: 0

{
  "data": {
    "repository": {
      "parent": {
        "name": "osv.dev",
        "owner": {
          "login": "google"
        }
      }
    }
  }
}

* Request took 401.046895ms
* Request at 2023-06-14 13:18:05.487124237 +1000 AEST m=+1.615690996
* Request to https://api.github.com/repos/google/osv.dev/git/refs/heads/master
> GET /repos/google/osv.dev/git/refs/heads/master HTTP/1.1
> Host: api.github.com
> Accept: application/vnd.github.merge-info-preview+json, application/vnd.github.nebula-preview
> Authorization: token ████████████████████
> Content-Type: application/json; charset=utf-8
> Time-Zone: Australia/Sydney
> User-Agent: GitHub CLI 2.22.1

< HTTP/2.0 200 OK
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
< Cache-Control: private, max-age=60, s-maxage=60
< Content-Security-Policy: default-src 'none'
< Content-Type: application/json; charset=utf-8
< Date: Wed, 14 Jun 2023 03:18:05 GMT
< Etag: W/"ce8a950703100f8cfd255ed55fe4c4ca1703348931ff40050b3a4bf01cf1f5ef"
< Github-Authentication-Token-Expiration: 2023-08-26 23:37:14 UTC
< Last-Modified: Wed, 14 Jun 2023 02:11:40 GMT
< Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
< Server: GitHub.com
< Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
< Vary: Accept, Authorization, Cookie, X-GitHub-OTP
< Vary: Accept-Encoding, Accept, X-Requested-With
< X-Accepted-Oauth-Scopes: repo
< X-Content-Type-Options: nosniff
< X-Frame-Options: deny
< X-Github-Api-Version-Selected: 2022-11-28
< X-Github-Media-Type: github.merge-info-preview; param=nebula-preview; format=json
< X-Github-Request-Id: CA64:322F:6FD798:763AFB:6489316D
< X-Oauth-Scopes: read:discussion, read:enterprise, read:gpg_key, read:org, read:packages, read:public_key, read:repo_hook, read:ssh_signing_key, repo, user
< X-Poll-Interval: 300
< X-Ratelimit-Limit: 5000
< X-Ratelimit-Remaining: 4989
< X-Ratelimit-Reset: 1686716251
< X-Ratelimit-Resource: core
< X-Ratelimit-Used: 11
< X-Xss-Protection: 0

{
  "ref": "refs/heads/master",
  "node_id": "MDM6UmVmMzI3Nzc1MjY5OnJlZnMvaGVhZHMvbWFzdGVy",
  "url": "https://api.github.com/repos/google/osv.dev/git/refs/heads/master",
  "object": {
    "sha": "a99c162d1372c807db3fa70454293486142efc1d",
    "type": "commit",
    "url": "https://api.github.com/repos/google/osv.dev/git/commits/a99c162d1372c807db3fa70454293486142efc1d"
  }
}

* Request took 329.57386ms
* Request at 2023-06-14 13:18:05.816961456 +1000 AEST m=+1.945528223
* Request to https://api.github.com/repos/andrewpollock/osv.dev/git/refs/heads/master
> PATCH /repos/andrewpollock/osv.dev/git/refs/heads/master HTTP/1.1
> Host: api.github.com
> Accept: application/vnd.github.merge-info-preview+json, application/vnd.github.nebula-preview
> Authorization: token ████████████████████
> Content-Type: application/json; charset=utf-8
> Time-Zone: Australia/Sydney
> User-Agent: GitHub CLI 2.22.1

{
  "force": false,
  "sha": "a99c162d1372c807db3fa70454293486142efc1d"
}

< HTTP/2.0 404 Not Found
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
< Content-Security-Policy: default-src 'none'
< Content-Type: application/json; charset=utf-8
< Date: Wed, 14 Jun 2023 03:18:06 GMT
< Github-Authentication-Token-Expiration: 2023-08-26 23:37:14 UTC
< Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
< Server: GitHub.com
< Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
< Vary: Accept-Encoding, Accept, X-Requested-With
< X-Accepted-Oauth-Scopes: repo
< X-Content-Type-Options: nosniff
< X-Frame-Options: deny
< X-Github-Api-Version-Selected: 2022-11-28
< X-Github-Media-Type: github.merge-info-preview; param=nebula-preview; format=json
< X-Github-Request-Id: CA64:322F:6FD7AF:763B17:6489316D
< X-Oauth-Scopes: read:discussion, read:enterprise, read:gpg_key, read:org, read:packages, read:public_key, read:repo_hook, read:ssh_signing_key, repo, user
< X-Ratelimit-Limit: 5000
< X-Ratelimit-Remaining: 4988
< X-Ratelimit-Reset: 1686716251
< X-Ratelimit-Resource: core
< X-Ratelimit-Used: 12
< X-Xss-Protection: 0

{
  "message": "Not Found",
  "documentation_url": "https://docs.github.com/rest/reference/git#update-a-reference"
}

* Request took 390.14467ms
HTTP 404: Not Found (https://api.github.com/repos/andrewpollock/osv.dev/git/refs/heads/master)
```@williammartin: Hello everyone, thank you for reporting this bug and providing additional information to help out.

I believe this is a platform change, but I also believe you can resolve it through the CLI. If you aren't interested in the details, jump to the **Resolution** section below.

### Investigation

Firstly, the `404 Not Found` on `https://api.github.com/repos/<org>/<repo>/git/refs/heads/<branch>` is a bit of a red herring. This `404` is a failure to `PATCH`, while attempting to update the branch to point at the commit ref provided in the body. I believe this is failing because the commit does not exist on the targeted repository. The reason it doesn't exist on the provided repository is due to an earlier failure on the `/merge-upstream` endpoint (that is swallowed for historical reasons I don't yet understand). For example, from @andrewpollock's logs:

```
> POST /repos/andrewpollock/osv.dev/merge-upstream HTTP/1.1
> Host: api.github.com
...

{
  "branch": "master"
}

< HTTP/2.0 422 Unprocessable Entity
```

**Edit**: My assumption around the `404` was incorrect, see the following post for details about why it failed and why the `422` and `404` have the same cause.

#### Why is this endpoint failing (intermittently)?

In digging into this, I found a recent change that began requiring [`workflow` scope](https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/scopes-for-oauth-apps#available-scopes) on the token if any commits to be merged from upstream included workflow changes.

I was able to somewhat prove this locally by forking `ocaml/dune`, checking out [@Alizter's commit](https://github.com/ocaml/dune/commit/9ba9901aa8de6a5f67a575a9d1e425774ba34ac7), and seeing `gh repo sync williammartin-test-org/dune` fail because the [next commit included a workflow change](https://github.com/ocaml/dune/commit/e1cd0a436c82ed364d9c46920d4d2e1cb6be2bd9). I then reset to and pushed [`HEAD^1` from `main`](https://github.com/ocaml/dune/commit/847c0eafadd1b91dc1a808b48d31bd8ef9b3bc92) and `sync`ed successfully, seeing that the [next commit didn't have a workflow change.](https://github.com/ocaml/dune/commit/97441450368ca70cdb82ab2dc9161fa9bc6ec0d7)

I believe this theory is further evidenced by:
 * Reports that this occurs intermittently (it would only occur if upstream commits to be merged included a workflow change)
 * Reports that using `sync repo` button on the UI succeeds (which would already have the required privileges)
 * Reports that after using `sync repo` on the UI, that the CLI works again (because the `merge-upstream` no longer fails and the commit exists after `PATCH` occurs)
 * Adding the `workflow` scope to my token allowed me to sync even with workflow changes upstream.

### Resolution

To resolve this you can add the `workflow` scope to your token either on initial login like so:

```
gh auth login -s workflow
```

or via refreshing you existing token like so:

```
gh auth refresh -s workflow
```

#### Caveat

It's possible that this change introduced other failure cases that I don't yet have the necessary knowledge to understand but I believe that this is almost certainly the common case. If after refreshing your token you still experience errors, please leave a comment.

### CLI Changes

From our point of view, we should determine how to handle this error in a more graceful manner, prompting to refresh to include the token. Unfortunately, there are a number of reasons that the API returns `422` and the reasons are opaque to clients, so we may need some changes before we can support this.

In the meantime, hopefully this issue serves as a place for people to find help.@williammartin: ### Further Investigation

Following up on the previous comment I did some further digging into the relationship between the earlier `422` and the later `404`. As it turns out, both occur for the same reason as far as I can tell (missing `workflow` scope).

I created a [commit that changed a workflow that was not associated with a branch](mmit/e22c39e2d13bd8bdc34392133b337e4699357b30) and tried to `PATCH` update the `head/main` ref and received a `404` when using a token without `workflow` scope:

![image](https://github.com/cli/cli/assets/1611510/776041d6-70e0-4ebb-98a9-0e0ffa5bc01a)

It was later successful with `workflow` scope:

<img width="1446" alt="image" src="https://github.com/cli/cli/assets/1611510/a064c648-6254-4128-8c7d-8618cebaa735">


What's happening here is that as described in the [oauth scopes doc](https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/scopes-for-oauth-apps), the `workflow` scope is needed for any workflow file change that isn't in a branch on the repository already. When we attempt to update the `ref`, GitHub is rejecting this as "you don't have permission to point this branch at something that contains a workflow file change" and that's manifesting as a `404`.

I think that ties up the loose ends!

@rjsparks: @williammartin - Thank you for the detailed explanation!@williammartin: The API team have shipped an improved error message for the `/merge-upstream` endpoint:

```
* Request to https://api.github.com/repos/williammartin-test-org/dune/merge-upstream
> POST /repos/williammartin-test-org/dune/merge-upstream HTTP/1.1
> Host: api.github.com
> Accept: application/vnd.github.merge-info-preview+json, application/vnd.github.nebula-preview
> Authorization: token ████████████████████
> Content-Length: 18
> Content-Type: application/json; charset=utf-8
> Time-Zone: Europe/Amsterdam
> User-Agent: GitHub CLI 2.30.0

{
  "branch": "main"
}

⣯< HTTP/2.0 422 Unprocessable Entity
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
< Content-Length: 245
< Content-Security-Policy: default-src 'none'
< Content-Type: application/json; charset=utf-8
< Date: Wed, 21 Jun 2023 11:58:36 GMT
< Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
< Server: GitHub.com
< Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
< Vary: Accept-Encoding, Accept, X-Requested-With
< X-Accepted-Oauth-Scopes:
< X-Content-Type-Options: nosniff
< X-Frame-Options: deny
< X-Github-Api-Version-Selected: 2022-11-28
< X-Github-Media-Type: github.merge-info-preview; param=nebula-preview; format=json
< X-Github-Request-Id: ECA3:30A7:51E59B8:52A51FF:6492E5EC
< X-Oauth-Client-Id: 178c6fc778ccc68e1d6a
< X-Oauth-Scopes: admin:public_key, gist, read:org, repo
< X-Ratelimit-Limit: 15000
< X-Ratelimit-Remaining: 14989
< X-Ratelimit-Reset: 1687351152
< X-Ratelimit-Resource: core
< X-Ratelimit-Used: 11
< X-Xss-Protection: 0

{
  "message": "refusing to allow an OAuth App to create or update workflow `.github/workflows/bench.yml` without `workflow` scope",
  "documentation_url": "https://docs.github.com/rest/branches/branches#sync-a-fork-branch-with-the-upstream-repository"
}
```

However, they have not yet made a change to the `PATCH /repos/<ORG>/<REPO>/git/refs/heads/main` endpoint.
 
Unfortunately, with our current strategy for trying `/merge-upstream` and then falling back to the `ref` update endpoint, we don't expose the original useful error. However, since the missing scope is unrecoverable between attempts, this should give us enough information to bail out early.@spencerschrock: > ### Resolution
> To resolve this you can add the `workflow` scope to your token either on initial login like so:
> 
> ```
> gh auth login -s workflow
> ```
> 
> or via refreshing you existing token like so:
> 
> ```
> gh auth refresh -s workflow
> ```
> 
> #### Caveat
> It's possible that this change introduced other failure cases that I don't yet have the necessary knowledge to understand but I believe that this is almost certainly the common case. If after refreshing your token you still experience errors, please leave a comment.

Is this incompatible with use of `gh` from GitHub Actions? From what I understand, `workflow` is an OAuth app scope, not a permission available to the GITHUB_TOKEN.

I use `gh repo sync` in a workflow to keep a fork in sync with its parent once a day, which started failing intermittently around the same time as this issue.  I tried adding `gh auth refresh -s workflow` to the workflow, but that seems geared towards an interactive login. Instead workflows are encouraged to login a different way:

> To use gh in GitHub Actions, add GH_TOKEN: ${{ github.token }} to "env".

As a test, I bumped the permissions of my workflow to`permissions: write-all` and upstream workflow changes continue to prevent syncs.

EDIT: A fine-grained PAT with `contents` and `workflows` permissions does work, but expires in a year.@samcoe: @spencerschrock Sounds like a platform bug that `permissions: write-all` does not allow syncing. I can ask around internally about it. 

Update, I heard back from the internal API team and they explained that this behavior is a security feature and is not a bug. So unfortunately the only work around is to generate a new token, perhaps with [this](https://github.com/gr2m/github-app-token-action), that has the correct scopes.@fzakaria: I am seeing the same issue after a commit that introduced a workflow file change.
The error still only shows 404 which is perplexing.

Here is the full log with the debug=api
https://pastebin.com/45UBLTUi
@Kamayana: > I am seeing the same issue after a commit that introduced a workflow file change. The error still only shows 404 which is perplexing.
> 
> Here is the full log with the debug=api https://pastebin.com/45UBLTUi

Same here. Should a new issue be made?@williammartin: @fzakaria @kamayana Can you add the workflow scope with `gh auth refresh -s workflow` and tell me it's still an issue?

@fzakaria I can see this in your `GH_DEBUG=api` output:

```
{
  "message": "refusing to allow an OAuth App to create or update workflow `.github/workflows/publishWheelRelease.yml` without `workflow` scope",
  "documentation_url": "https://docs.github.com/rest/branches/branches#sync-a-fork-branch-with-the-upstream-repository"
}
```

Additionally, I can see that you are using and old version of `gh`, and https://github.com/cli/cli/pull/7612 made this error more obvious. I think it went in in 2.32 and you are using 2.30@fzakaria: Yes it worked with that workfow scope. Thank you for looking into it.

I'll update my `gh` version; the discrepancy between the button on the UI and the default on the client still feels like a gap FWIW>@williammartin: > I'll update my gh version; the discrepancy between the button on the UI and the default on the client still feels like a gap FWIW

Yeh it's a tricky balance of asking for the minimal set of scopes required for most operations and providing useful errors when required vs just asking for everything. I'm not sure there is a **right** answer here, but your feedback is useful in helping us figure it out, thanks!@wata727: I also faced this issue. Strangely, even after #7612 was merged, I still get the misleading 404 not found error.

```console
$ gh repo sync terraform-linters/magic-modules
HTTP 404: Not Found (https://api.github.com/repos/GoogleCloudPlatform/magic-modules/git/refs/heads/tflint_provider)
```
https://github.com/terraform-linters/magic-modules/actions/runs/13742985933/job/38434464702

After some digging, I found that the regexp introduced in #7612 doesn't work with GitHub App token error messages. The message looks like this and does not match the "workflow scope" (should be "workflow**s** permission").

```
{
  "message": "refusing to allow a GitHub App to create or update workflow `.github/workflows/teamcity-pr-checks.yml` without `workflows` permission",
  "documentation_url": "https://docs.github.com/rest/branches/branches#sync-a-fork-branch-with-the-upstream-repository",
  "status": "422"
}
```
https://github.com/terraform-linters/magic-modules/actions/runs/13745172118/job/38439197950

@williammartin Would it be ok to open a PR to fix this issue? If so, I can submit a PR with https://github.com/cli/cli/commit/a30afcf239e4016548e28ec4b0c3d511e4d71463 in it.@williammartin: Hey @wata727 thanks for the investigation. Please open a PR against https://github.com/cli/cli/issues/10573.

Maybe you could add some details in the issue about exactly how you obtained your token that was missing the `workflow` permission to make it easy for us to do acceptance on. Thanks.

Solution patch:
diff --git a/pkg/cmd/repo/sync/http.go b/pkg/cmd/repo/sync/http.go
index d5cc0e282..61a1c3317 100644
--- a/pkg/cmd/repo/sync/http.go
+++ b/pkg/cmd/repo/sync/http.go
@@ -6,6 +6,7 @@ import (
 	"errors"
 	"fmt"
 	"net/http"
+	"regexp"
 
 	"github.com/cli/cli/v2/api"
 	"github.com/cli/cli/v2/internal/ghrepo"
@@ -33,6 +34,9 @@ type upstreamMergeErr struct{ error }
 
 var upstreamMergeUnavailableErr = upstreamMergeErr{errors.New("upstream merge API is unavailable")}
 
+var missingWorkflowScopeRE = regexp.MustCompile("refusing to allow.*without `workflow` scope")
+var missingWorkflowScopeErr = errors.New("Upstream commits contain workflow changes, which require the `workflow` scope to merge. To request it, run: gh auth refresh -s workflow")
+
 func triggerUpstreamMerge(client *api.Client, repo ghrepo.Interface, branch string) (string, error) {
 	var payload bytes.Buffer
 	if err := json.NewEncoder(&payload).Encode(map[string]interface{}{
@@ -52,6 +56,9 @@ func triggerUpstreamMerge(client *api.Client, repo ghrepo.Interface, branch stri
 		if errors.As(err, &httpErr) {
 			switch httpErr.StatusCode {
 			case http.StatusUnprocessableEntity, http.StatusConflict:
+				if missingWorkflowScopeRE.MatchString(httpErr.Message) {
+					return "", missingWorkflowScopeErr
+				}
 				return "", upstreamMergeErr{errors.New(httpErr.Message)}
 			case http.StatusNotFound:
 				return "", upstreamMergeUnavailableErr
diff --git a/pkg/httpmock/stub.go b/pkg/httpmock/stub.go
index b26d433fa..9332ac9dd 100644
--- a/pkg/httpmock/stub.go
+++ b/pkg/httpmock/stub.go
@@ -143,7 +143,20 @@ func StatusStringResponse(status int, body string) Responder {
 func JSONResponse(body interface{}) Responder {
 	return func(req *http.Request) (*http.Response, error) {
 		b, _ := json.Marshal(body)
-		return httpResponse(200, req, bytes.NewBuffer(b)), nil
+		header := http.Header{
+			"Content-Type": []string{"application/json"},
+		}
+		return httpResponseWithHeader(200, req, bytes.NewBuffer(b), header), nil
+	}
+}
+
+func StatusJSONResponse(status int, body interface{}) Responder {
+	return func(req *http.Request) (*http.Response, error) {
+		b, _ := json.Marshal(body)
+		header := http.Header{
+			"Content-Type": []string{"application/json"},
+		}
+		return httpResponseWithHeader(status, req, bytes.NewBuffer(b), header), nil
 	}
 }
 
@@ -215,10 +228,14 @@ func ScopesResponder(scopes string) func(*http.Request) (*http.Response, error)
 }
 
 func httpResponse(status int, req *http.Request, body io.Reader) *http.Response {
+	return httpResponseWithHeader(status, req, body, http.Header{})
+}
+
+func httpResponseWithHeader(status int, req *http.Request, body io.Reader, header http.Header) *http.Response {
 	return &http.Response{
 		StatusCode: status,
 		Request:    req,
 		Body:       io.NopCloser(body),
-		Header:     http.Header{},
+		Header:     header,
 	}
 }



Test patch:
diff --git a/pkg/cmd/repo/sync/sync_test.go b/pkg/cmd/repo/sync/sync_test.go
index 90a216607..a532f3992 100644
--- a/pkg/cmd/repo/sync/sync_test.go
+++ b/pkg/cmd/repo/sync/sync_test.go
@@ -457,6 +457,24 @@ func Test_SyncRun(t *testing.T) {
 			wantErr: true,
 			errMsg:  "trunk branch does not exist on OWNER/REPO-FORK repository",
 		},
+		{
+			name: "sync remote fork with missing workflow scope on token",
+			opts: &SyncOptions{
+				DestArg: "FORKOWNER/REPO-FORK",
+			},
+			httpStubs: func(reg *httpmock.Registry) {
+				reg.Register(
+					httpmock.GraphQL(`query RepositoryInfo\b`),
+					httpmock.StringResponse(`{"data":{"repository":{"defaultBranchRef":{"name": "trunk"}}}}`))
+				reg.Register(
+					httpmock.REST("POST", "repos/FORKOWNER/REPO-FORK/merge-upstream"),
+					httpmock.StatusJSONResponse(422, struct {
+						Message string `json:"message"`
+					}{Message: "refusing to allow an OAuth App to create or update workflow `.github/workflows/unimportant.yml` without `workflow` scope"}))
+			},
+			wantErr: true,
+			errMsg:  "Upstream commits contain workflow changes, which require the `workflow` scope to merge. To request it, run: gh auth refresh -s workflow",
+		},
 	}
 	for _, tt := range tests {
 		reg := &httpmock.Registry{}