Repository: qutebrowser/qutebrowser

Issue:
Automate macOS/Windows releases via GitHub Actions
It'd be nice to automatically push releases via ~~Travis/AppVeyor~~ GitHub Actions when a tag is pushed, at least for macOS and Windows. I've historically been uncomfortable with letting the cloud™ push releases - however:

- I currently don't have easy access to a Mac
- Mac releases need to be built on an old macOS version (#3722)
- Windows releases are painful and I probably won't have a valid Windows license in ~1.5 years anymore (not that it'd be a problem, but it's additional work :laughing:)
- Dependencies of qutebrowser are already released that way, so if there's some security issue on Travis/Appveyor, it probably doesn't make a difference either way...
- It'd greatly streamline the release process.
- Those releases currently aren't signed (and won't be), only the source release is. I can still easily do the source release locally.


Discussion:
Will doing this also allow for koshering the app so that it can be run without admin rights?
@gulkily That in itself won't change anything in how releases look. However, it shouldn't need admin rights to run. If it does, please open a separate issue with more information.
Here's a rough draft:

```yaml
name: Upload release
on:
  tags:
    - 'v*'

jobs:
  create-release:
    name: Create Release
    runs-on: ubuntu-latest
    steps:
      - name: Create Release
        id: create-release
        uses: actions/create-release@v1.1.2
        env:
          GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
        with:
          tag_name: "${{ github.ref }}"
          release_name: "${{ github.ref }}"
          body: |
            Check the [changelog](https://github.com/qutebrowser/qutebrowser/blob/master/doc/changelog.asciidoc) for changes in this release.
    outputs:
      upload_url: "${{ steps.create-release.outputs.upload_url }}"

  upload-release:
    runs-on: "${{ matrix.os }}"
    needs: create-release
    strategy:
      matrix:
        include:
          - os: ubuntu-20.04
          - os: macos-10.15
          - os: windows-2019
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: 3.7
      - name: Build release
        run: "python scripts/dev/build_release.py"
      - name: Upload release
        uses: actions/upload-release-asset@v1.0.2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          upload_url: ${{ needs.create-release.outputs.upload_url }}
          asset_path: FIXME
          asset_name: FIXME
          asset_content_type: FIXME
```

One problem I only noticed now: GitHub Actions only supports macOS 10.15 Catalina, which would mean we wouldn't support older macOS versions. That's kind of a deal breaker, unfortunately...
Looks like GitHub recently [shipped workflow restrictions](https://github.blog/2022-03-23-github-actions-secure-self-hosted-runners-specific-workflows/) for self-hosted runners. This would probably make it feasible to use our MacStadium machine as a build agent for releases without running e.g. tests for PRs on it.
Actually, we're most likely fine with dropping 10.14 for qutebrowser 3.0.0, see #7092.
Well, back to square one - GitHub is deprecating macOS 10.15 runners by end of August: https://github.com/actions/virtual-environments/issues/5583 (Qt still seems to support 10.15 and even 10.14 [for Qt 6.4](https://doc-snapshots.qt.io/qt6-6.4/supported-platforms.html)).



We dropped macOS < 11 in #7092 so this should work out fine again. It might lead to us having to drop future macOS versions earlier than we'd want, though... but I suppose it would still be good to have automated building running. If we run into such a situation, we could always revert to hosting a custom runner or manual building if we really want to keep support around for longer.

Another pain point with this is how to handle the PGP signature. Apparently PyPI dropped support for them anyways: [Removing PGP from PyPI - The Python Package Index](https://blog.pypi.org/posts/2023-05-23-removing-pgp/).

I obviously won't upload my private key to GitHub. According to [qutebrowser/qutebrowser - GitHub Release Stats](https://tooomm.github.io/github-release-stats/?username=qutebrowser&repository=qutebrowser), some 10-20% of people downloading the .tar.gz also download the key, so I suppose there is some value in it (especially also for distribution packagers).

I suppose one possibility there would be to generate a new dedicated qutebowser release signing key, and then exposing that as a secret on GitHub CI.

Resolving patch:
diff --git a/scripts/dev/build_release.py b/scripts/dev/build_release.py
--- a/scripts/dev/build_release.py
+++ b/scripts/dev/build_release.py
@@ -400,13 +400,11 @@ def build_windows(
     utils.print_title("Updating VersionInfo file")
     gen_versioninfo.main()
 
-    artifacts = [
-        _build_windows_single(
-            skip_packaging=skip_packaging,
-            debug=debug,
-            qt5=qt5,
-        ),
-    ]
+    artifacts = _build_windows_single(
+        skip_packaging=skip_packaging,
+        debug=debug,
+        qt5=qt5,
+    )
     return artifacts
 
 
@@ -434,6 +432,8 @@ def _package_windows_single(
         name_parts.append('debug')
     if qt5:
         name_parts.append('qt5')
+
+    name_parts.append('amd64')  # FIXME:qt6 temporary until new installer
     name = '-'.join(name_parts) + '.exe'
 
     artifacts.append(Artifact(
@@ -518,9 +518,17 @@ def build_sdist() -> List[Artifact]:
 def test_makefile() -> None:
     """Make sure the Makefile works correctly."""
     utils.print_title("Testing makefile")
+    a2x_path = pathlib.Path(sys.executable).parent / 'a2x'
+    assert a2x_path.exists(), a2x_path
     with tempfile.TemporaryDirectory() as tmpdir:
-        subprocess.run(['make', '-f', 'misc/Makefile',
-                        f'DESTDIR={tmpdir}', 'install'], check=True)
+        subprocess.run(
+            [
+                'make', '-f', 'misc/Makefile',
+                f'DESTDIR={tmpdir}', f'A2X={a2x_path}',
+                'install'
+            ],
+            check=True,
+        )
 
 
 def read_github_token(
@@ -531,6 +539,9 @@ def read_github_token(
     if arg_token is not None:
         return arg_token
 
+    if "GITHUB_TOKEN" in os.environ:
+        return os.environ["GITHUB_TOKEN"]
+
     token_path = pathlib.Path.home() / '.gh_token'
     if not token_path.exists():
         if optional:
@@ -544,13 +555,19 @@ def read_github_token(
     return token
 
 
-def github_upload(artifacts: List[Artifact], tag: str, gh_token: str) -> None:
+def github_upload(
+    artifacts: List[Artifact],
+    tag: str,
+    gh_token: str,
+    experimental: bool,
+) -> None:
     """Upload the given artifacts to GitHub.
 
     Args:
         artifacts: A list of Artifacts to upload.
         tag: The name of the release tag
         gh_token: The GitHub token to use
+        experimental: Upload to the experiments repo
     """
     # pylint: disable=broad-exception-raised
     import github3
@@ -558,14 +575,20 @@ def github_upload(artifacts: List[Artifact], tag: str, gh_token: str) -> None:
     utils.print_title("Uploading to github...")
 
     gh = github3.login(token=gh_token)
-    repo = gh.repository('qutebrowser', 'qutebrowser')
+
+    if experimental:
+        repo = gh.repository('qutebrowser', 'experiments')
+    else:
+        repo = gh.repository('qutebrowser', 'qutebrowser')
 
     release = None  # to satisfy pylint
     for release in repo.releases():
         if release.tag_name == tag:
             break
     else:
-        raise Exception(f"No release found for {tag!r}!")
+        releases = ", ".join(r.tag_name for r in repo.releases())
+        raise Exception(
+            f"No release found for {tag!r} in {repo.full_name}, found: {releases}")
 
     for artifact in artifacts:
         while True:
@@ -575,6 +598,10 @@ def github_upload(artifacts: List[Artifact], tag: str, gh_token: str) -> None:
                       if asset.name == artifact.path.name]
             if assets:
                 print(f"Assets already exist: {assets}")
+
+                if utils.ON_CI:
+                    sys.exit(1)
+
                 print("Press enter to continue anyways or Ctrl-C to abort.")
                 input()
 
@@ -588,8 +615,13 @@ def github_upload(artifacts: List[Artifact], tag: str, gh_token: str) -> None:
                     )
             except github3.exceptions.ConnectionError as e:
                 utils.print_error(f'Failed to upload: {e}')
-                print("Press Enter to retry...", file=sys.stderr)
-                input()
+                if utils.ON_CI:
+                    print("Retrying in 30s...")
+                    time.sleep(30)
+                else:
+                    print("Press Enter to retry...", file=sys.stderr)
+                    input()
+
                 print("Retrying!")
 
                 assets = [asset for asset in release.assets()
@@ -602,10 +634,16 @@ def github_upload(artifacts: List[Artifact], tag: str, gh_token: str) -> None:
                 break
 
 
-def pypi_upload(artifacts: List[Artifact]) -> None:
+def pypi_upload(artifacts: List[Artifact], experimental: bool) -> None:
     """Upload the given artifacts to PyPI using twine."""
+    # https://blog.pypi.org/posts/2023-05-23-removing-pgp/
+    artifacts = [a for a in artifacts if a.mimetype != 'application/pgp-signature']
+
     utils.print_title("Uploading to PyPI...")
-    run_twine('upload', artifacts)
+    if experimental:
+        run_twine('upload', artifacts, "-r", "testpypi")
+    else:
+        run_twine('upload', artifacts)
 
 
 def twine_check(artifacts: List[Artifact]) -> None:
@@ -635,6 +673,8 @@ def main() -> None:
                         help="Build a debug build.")
     parser.add_argument('--qt5', action='store_true', required=False,
                         help="Build against PyQt5")
+    parser.add_argument('--experimental', action='store_true', required=False,
+                        help="Upload to experiments repo and test PyPI")
     args = parser.parse_args()
     utils.change_cwd()
 
@@ -647,6 +687,7 @@ def main() -> None:
         gh_token = read_github_token(args.gh_token)
     else:
         gh_token = read_github_token(args.gh_token, optional=True)
+        assert not args.experimental  # makes no sense without upload
 
     if not misc_checks.check_git():
         utils.print_error("Refusing to do a release with a dirty git tree")
@@ -680,14 +721,15 @@ def main() -> None:
     if args.upload:
         version_tag = f"v{qutebrowser.__version__}"
 
-        if not args.no_confirm:
+        if not args.no_confirm and not utils.ON_CI:
             utils.print_title(f"Press enter to release {version_tag}...")
             input()
 
         assert gh_token is not None
-        github_upload(artifacts, version_tag, gh_token=gh_token)
+        github_upload(
+            artifacts, version_tag, gh_token=gh_token, experimental=args.experimental)
         if upload_to_pypi:
-            pypi_upload(artifacts)
+            pypi_upload(artifacts, experimental=args.experimental)
     else:
         print()
         utils.print_title("Artifacts")
diff --git a/scripts/dev/update_version.py b/scripts/dev/update_version.py
--- a/scripts/dev/update_version.py
+++ b/scripts/dev/update_version.py
@@ -8,6 +8,7 @@
 
 """Update version numbers using bump2version."""
 
+import re
 import sys
 import argparse
 import os.path
@@ -19,6 +20,24 @@
 from scripts import utils
 
 
+class Error(Exception):
+    """Base class for exceptions in this module."""
+
+
+def verify_branch(version_leap):
+    """Check that we're on the correct git branch."""
+    proc = subprocess.run(
+        ['git', 'rev-parse', '--abbrev-ref', 'HEAD'],
+        check=True, capture_output=True, text=True)
+    branch = proc.stdout.strip()
+
+    if (
+        version_leap == 'patch' and not re.fullmatch(r'v\d+\.\d+\.x', branch) or
+        version_leap != 'patch' and branch != 'main'
+    ):
+        raise Error(f"Invalid branch for {version_leap} release: {branch}")
+
+
 def bump_version(version_leap="patch"):
     """Update qutebrowser release version.
 
@@ -31,7 +50,11 @@ def bump_version(version_leap="patch"):
 
 
 def show_commit():
-    subprocess.run(['git', 'show'], check=True)
+    """Show the latest git commit."""
+    git_args = ['git', 'show']
+    if utils.ON_CI:
+        git_args.append("--color")
+    subprocess.run(git_args, check=True)
 
 
 if __name__ == "__main__":
@@ -46,30 +69,39 @@ def show_commit():
     utils.change_cwd()
 
     if not args.commands:
+        verify_branch(args.bump)
         bump_version(args.bump)
         show_commit()
 
     import qutebrowser
     version = qutebrowser.__version__
-    x_version = '.'.join([str(p) for p in qutebrowser.__version_info__[:-1]] +
+    version_x = '.'.join([str(p) for p in qutebrowser.__version_info__[:-1]] +
                          ['x'])
 
-    print("Run the following commands to create a new release:")
-    print("* git push origin; git push origin v{v}".format(v=version))
-    if args.bump == 'patch':
-        print("* git checkout main && git cherry-pick v{v} && "
-              "git push origin".format(v=version))
+    if utils.ON_CI:
+        output_file = os.environ["GITHUB_OUTPUT"]
+        with open(output_file, "w", encoding="ascii") as f:
+            f.write(f"version={version}\n")
+            f.write(f"version_x={version_x}\n")
+
+        print(f"Outputs for {version} written to GitHub Actions output file")
     else:
-        print("* git branch v{x} v{v} && git push --set-upstream origin v{x}"
-              .format(v=version, x=x_version))
-    print("* Create new release via GitHub (required to upload release "
-          "artifacts)")
-    print("* Linux: git fetch && git checkout v{v} && "
-          "tox -e build-release -- --upload"
-          .format(v=version))
-    print("* Windows: git fetch; git checkout v{v}; "
-          "py -3.9 -m tox -e build-release -- --upload"
-          .format(v=version))
-    print("* macOS: git fetch && git checkout v{v} && "
-          "tox -e build-release -- --upload"
-          .format(v=version))
+        print("Run the following commands to create a new release:")
+        print("* git push origin; git push origin v{v}".format(v=version))
+        if args.bump == 'patch':
+            print("* git checkout main && git cherry-pick -x v{v} && "
+                "git push origin".format(v=version))
+        else:
+            print("* git branch v{x} v{v} && git push --set-upstream origin v{x}"
+                .format(v=version, x=version_x))
+        print("* Create new release via GitHub (required to upload release "
+            "artifacts)")
+        print("* Linux: git fetch && git checkout v{v} && "
+            "tox -e build-release -- --upload"
+            .format(v=version))
+        print("* Windows: git fetch; git checkout v{v}; "
+            "py -3.X -m tox -e build-release -- --upload"
+            .format(v=version))
+        print("* macOS: git fetch && git checkout v{v} && "
+            "tox -e build-release -- --upload"
+            .format(v=version))


Test patch: