Repository: GaloisInc/parameterized-utils
Pull request #183: ci: Publish haddocks to GitHub Pages
Author: login: langston-barrett
Created: 2025-03-13T21:38:50Z | Merged: 2025-03-14T16:58:52Z

In between full Hackage releases, it would be nice to be able to read haddocks for the latest commit on the default branch. This PR builds the haddocks in CI, and publishes them to GitHub Pages.

In more detail, this PR breaks the `cabal haddock` step out of the testing job into a separate job for the sake of controlling concurrency and permissions, while reusing the cache from the build step to build the haddocks.

I'd be completetly fine with hearing that we don't think that the maintenance is worth it, I mostly wanted to prototype this for use on our repos that we don't publish to Hackage (e.g., Macaw) and just figured I'd make a PR in case others thought this would be useful.

If we did want to merge this, we would want to temporarily disable the `if` stanza that only deploys on the `master` branch, to try it out.


Diff:
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 88863b2..5f18df8 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -1,9 +1,9 @@
-name: Build and Test
+name: Build, test, and document
 on:
   - push
   - pull_request
 jobs:
-  linux:
+  test:
     name: Testing ${{ matrix.os }} GHC-${{ matrix.ghc }}
     runs-on: ${{ matrix.os }}
     continue-on-error: ${{ matrix.allow-failure }}
@@ -34,6 +34,8 @@ jobs:
           # NB: Each `matrix.os` (e.g., `ubuntu-22.04-arm`) uniquely determines
           # a `runner.arch` (e.g., ARM64), so there is no need to include the
           # latter as part of the cache key
+          #
+          # Any changes to this key should be reflected in the `doc` job below.
           key: cabal-${{ matrix.os }}-ghc${{ matrix.ghc }}
         with:
           path: |
@@ -91,9 +93,6 @@ jobs:
       - name: Test
         shell: bash
         run: cabal test
-      - name: Documentation
-        shell: bash
-        run: cabal haddock
       - uses: actions/cache/save@v4
         name: Save cabal store cache
         if: always()
@@ -102,3 +101,64 @@ jobs:
             ${{ steps.setup-haskell.outputs.cabal-store }}
             dist-newstyle
           key: ${{ steps.cache.outputs.cache-primary-key }}
+
+  doc:
+    name: Publish haddocks to GitHub Pages
+    needs: test
+    env:
+      ghc: "9.10.1"
+      os: "ubuntu-24.04"
+    # For some reason, can't use ${{ env.os }} here, so just be sure to keep
+    # this in sync with the line above.
+    runs-on: "ubuntu-24.04"
+    concurrency:
+      # Only run one of these at a time because they update the global pages.
+      group: pages
+      cancel-in-progress: false
+    permissions:
+      contents: read
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: haskell-actions/setup@v2
+        id: setup-haskell
+        with:
+          ghc-version: ${{ env.ghc }}
+      # Note that this uses (but does not modify) the cache from the `test`
+      # step. They use all the same dependencies, so this is appropriate.
+      - uses: actions/cache/restore@v4
+        name: Restore Cabal cache
+        env:
+          key: cabal-${{ env.os }}-ghc${{ env.ghc }}
+        with:
+          path: |
+            ${{ steps.setup-haskell.outputs.cabal-store }}
+            dist-newstyle
+          key: |
+            ${{ env.key }}-${{ github.ref }}
+          restore-keys: |
+            ${{ env.key }}-
+      - name: Build haddocks
+        run: >
+          cabal
+          haddock
+          --haddock-output-dir=doc
+          --haddock-html-location='https://hackage.haskell.org/package/$pkg-$version/docs'
+        shell: bash
+      - name: Upload artifact with haddocks to GitHub Pages
+        uses: actions/upload-pages-artifact@v3
+        if: github.ref == 'refs/heads/master' &&
+            github.event.pull_request.head.repo.fork == false &&
+            github.repository_owner == 'GaloisInc'
+        with:
+          path: doc/parameterized-utils
+      - name: Deploy haddocks to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4
+        if: github.ref == 'refs/heads/master' &&
+            github.event.pull_request.head.repo.fork == false &&
+            github.repository_owner == 'GaloisInc'
diff --git a/doc/dev.md b/doc/dev.md
index 413b893..31cf3c0 100644
--- a/doc/dev.md
+++ b/doc/dev.md
@@ -15,6 +15,7 @@ When performing such an upgrade, it can be helpful to copy/paste this list into
 - [ ] Run `cabal {build,test,haddock}`, bumping dependency bounds as needed
 - [ ] Fix any new warnings from `-Wdefault`
 - [ ] Add the new GHC version to the matrix in the GitHub Actions configuration
+- [ ] Change the `doc` job to use the new GHC version
 - [ ] Add the new version to the code that sets `GHC_NIXPKGS` in the CI config
 - [ ] Add the new GHC version to the Cabal `tested-with` field
 - [ ] Optionally follow the below steps to remove any old GHC versions


Reviews:
user: login: kquick
body: This overall publishing should be fine in general.  I don't know how fully useful this will be since it will *only* be the docs for this package and so I expect cross references won't work.  Another thought is that if there are space constraints on gh-pages (although I'm not aware of any, and even with our larger projects I suspect the size of generated docs is rather minimal).

This will only provide the latest master version of the docs.

A couple of comments, one geared towards minimizing the support overhead.
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-13T22:00:25Z
user: login: kquick
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-13T22:59:17Z
user: login: kquick
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-13T23:00:14Z
user: login: langston-barrett
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T13:24:42Z
user: login: langston-barrett
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T13:25:15Z
user: login: langston-barrett
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T13:26:24Z
user: login: kquick
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-14T15:01:46Z
user: login: kquick
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-14T15:05:23Z
user: login: langston-barrett
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T15:13:55Z
user: login: langston-barrett
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T15:14:39Z
user: login: langston-barrett
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-14T16:59:13Z

Review comments:
diff_hunk: @@ -102,3 +99,58 @@ jobs:
             ${{ steps.setup-haskell.outputs.cabal-store }}
             dist-newstyle
           key: ${{ steps.cache.outputs.cache-primary-key }}
+
+  doc:
+    name: Publish haddocks to GitHub Pages
+    needs: test
+    env:
+      ghc: "9.10.1"
+      os: "ubuntu-24.04"
+    # For some reason, can't use ${{ env.os }} here, so just be sure to keep
+    # this in sync with the line above.
+    runs-on: "ubuntu-24.04"
+    concurrency:
+      # Only run one of these at a time because they update the global pages.
+      group: haddock
+      cancel-in-progress: false
+    permissions:
+      contents: read
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: haskell-actions/setup@v2
+        id: setup-haskell
+        with:
+          ghc-version: ${{ env.ghc }}
+      - uses: actions/cache/restore@v4
+        name: Restore Cabal cache
+        env:
+          key: cabal-${{ env.os }}-ghc${{ env.ghc }}
path: .github/workflows/test.yml
user: login: kquick
body: This job should probably use a different cache key value than the main build job since the cached objects will be different.
created_at: 2025-03-13T22:59:17Z
author_association: MEMBER
diff_hunk: @@ -102,3 +99,58 @@ jobs:
             ${{ steps.setup-haskell.outputs.cabal-store }}
             dist-newstyle
           key: ${{ steps.cache.outputs.cache-primary-key }}
+
+  doc:
+    name: Publish haddocks to GitHub Pages
+    needs: test
+    env:
+      ghc: "9.10.1"
+      os: "ubuntu-24.04"
+    # For some reason, can't use ${{ env.os }} here, so just be sure to keep
+    # this in sync with the line above.
+    runs-on: "ubuntu-24.04"
path: .github/workflows/test.yml
user: login: kquick
body: Maybe this could just use `ubuntu-latest` (and `ghc-latest` if available) because we're just building docs, not relying on this for actual build/test validation.
created_at: 2025-03-13T23:00:14Z
author_association: MEMBER
diff_hunk: @@ -102,3 +99,58 @@ jobs:
             ${{ steps.setup-haskell.outputs.cabal-store }}
             dist-newstyle
           key: ${{ steps.cache.outputs.cache-primary-key }}
+
+  doc:
+    name: Publish haddocks to GitHub Pages
+    needs: test
+    env:
+      ghc: "9.10.1"
+      os: "ubuntu-24.04"
+    # For some reason, can't use ${{ env.os }} here, so just be sure to keep
+    # this in sync with the line above.
+    runs-on: "ubuntu-24.04"
+    concurrency:
+      # Only run one of these at a time because they update the global pages.
+      group: haddock
+      cancel-in-progress: false
+    permissions:
+      contents: read
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: haskell-actions/setup@v2
+        id: setup-haskell
+        with:
+          ghc-version: ${{ env.ghc }}
+      - uses: actions/cache/restore@v4
+        name: Restore Cabal cache
+        env:
+          key: cabal-${{ env.os }}-ghc${{ env.ghc }}
path: .github/workflows/test.yml
user: login: langston-barrett
body: Ah, this is actually intentional, and for just the reverse reason! I think the cached objects (the dependencies) will be the same! Also note that this step doesn't *modify* the cache, it just reuses the pre-built cache from the `test` job. I can add a comment that makes this all more explicit.
created_at: 2025-03-14T13:24:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -102,3 +99,58 @@ jobs:
             ${{ steps.setup-haskell.outputs.cabal-store }}
             dist-newstyle
           key: ${{ steps.cache.outputs.cache-primary-key }}
+
+  doc:
+    name: Publish haddocks to GitHub Pages
+    needs: test
+    env:
+      ghc: "9.10.1"
+      os: "ubuntu-24.04"
+    # For some reason, can't use ${{ env.os }} here, so just be sure to keep
+    # this in sync with the line above.
+    runs-on: "ubuntu-24.04"
+    concurrency:
+      # Only run one of these at a time because they update the global pages.
+      group: haddock
+      cancel-in-progress: false
+    permissions:
+      contents: read
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: haskell-actions/setup@v2
+        id: setup-haskell
+        with:
+          ghc-version: ${{ env.ghc }}
+      - uses: actions/cache/restore@v4
+        name: Restore Cabal cache
+        env:
+          key: cabal-${{ env.os }}-ghc${{ env.ghc }}
path: .github/workflows/test.yml
user: login: langston-barrett
body: As for whether reusing the cache from `test` is sufficient for speedy doc builds, note that [this one took 18s](https://github.com/GaloisInc/parameterized-utils/actions/runs/13845141506/job/38741914356?pr=183)
created_at: 2025-03-14T13:25:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -102,3 +99,58 @@ jobs:
             ${{ steps.setup-haskell.outputs.cabal-store }}
             dist-newstyle
           key: ${{ steps.cache.outputs.cache-primary-key }}
+
+  doc:
+    name: Publish haddocks to GitHub Pages
+    needs: test
+    env:
+      ghc: "9.10.1"
+      os: "ubuntu-24.04"
+    # For some reason, can't use ${{ env.os }} here, so just be sure to keep
+    # this in sync with the line above.
+    runs-on: "ubuntu-24.04"
path: .github/workflows/test.yml
user: login: langston-barrett
body: I'm a bit hesitant here. I would rather avoid CI failing for reasons unrelated to someone's PR, which can happen when versions silently change in the background. But perhaps more importantly given the dicussion above, this step needs to share the same cache key as `test`, which involves the OS version. I think I'll keep tihs for now.
created_at: 2025-03-14T13:26:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -102,3 +99,58 @@ jobs:
             ${{ steps.setup-haskell.outputs.cabal-store }}
             dist-newstyle
           key: ${{ steps.cache.outputs.cache-primary-key }}
+
+  doc:
+    name: Publish haddocks to GitHub Pages
+    needs: test
+    env:
+      ghc: "9.10.1"
+      os: "ubuntu-24.04"
+    # For some reason, can't use ${{ env.os }} here, so just be sure to keep
+    # this in sync with the line above.
+    runs-on: "ubuntu-24.04"
+    concurrency:
+      # Only run one of these at a time because they update the global pages.
+      group: haddock
+      cancel-in-progress: false
+    permissions:
+      contents: read
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: haskell-actions/setup@v2
+        id: setup-haskell
+        with:
+          ghc-version: ${{ env.ghc }}
+      - uses: actions/cache/restore@v4
+        name: Restore Cabal cache
+        env:
+          key: cabal-${{ env.os }}-ghc${{ env.ghc }}
path: .github/workflows/test.yml
user: login: kquick
body: OK, I missed the fact that you have no cache/save action below, so this will always rebuild the current project, but can take advantage of the cached dependencies.  Please document this, either here or in dev documentation somewhere.
created_at: 2025-03-14T15:01:46Z
author_association: MEMBER
diff_hunk: @@ -102,3 +99,58 @@ jobs:
             ${{ steps.setup-haskell.outputs.cabal-store }}
             dist-newstyle
           key: ${{ steps.cache.outputs.cache-primary-key }}
+
+  doc:
+    name: Publish haddocks to GitHub Pages
+    needs: test
+    env:
+      ghc: "9.10.1"
+      os: "ubuntu-24.04"
+    # For some reason, can't use ${{ env.os }} here, so just be sure to keep
+    # this in sync with the line above.
+    runs-on: "ubuntu-24.04"
path: .github/workflows/test.yml
user: login: kquick
body: I'd be surprised if CI actually failed, given that Haddock is extremely unlikely to change in any non-backward-compatible manner and GHC itself is careful about deprecations and future change warnings... as well as the fact that there's no  tests here.  I was just trying to avoid having to add another bullet item to your dev.md below, but I don't have a strong concern about this being locked if you prefer that route.
created_at: 2025-03-14T15:05:23Z
author_association: MEMBER
diff_hunk: @@ -102,3 +99,58 @@ jobs:
             ${{ steps.setup-haskell.outputs.cabal-store }}
             dist-newstyle
           key: ${{ steps.cache.outputs.cache-primary-key }}
+
+  doc:
+    name: Publish haddocks to GitHub Pages
+    needs: test
+    env:
+      ghc: "9.10.1"
+      os: "ubuntu-24.04"
+    # For some reason, can't use ${{ env.os }} here, so just be sure to keep
+    # this in sync with the line above.
+    runs-on: "ubuntu-24.04"
+    concurrency:
+      # Only run one of these at a time because they update the global pages.
+      group: haddock
+      cancel-in-progress: false
+    permissions:
+      contents: read
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: haskell-actions/setup@v2
+        id: setup-haskell
+        with:
+          ghc-version: ${{ env.ghc }}
+      - uses: actions/cache/restore@v4
+        name: Restore Cabal cache
+        env:
+          key: cabal-${{ env.os }}-ghc${{ env.ghc }}
path: .github/workflows/test.yml
user: login: langston-barrett
body: Is this comment sufficient, or is there more I could add? https://github.com/GaloisInc/parameterized-utils/pull/183/files#diff-faff1af3d8ff408964a57b2e475f69a6b7c7b71c9978cccc8f471798caac2c88R131
created_at: 2025-03-14T15:13:55Z
author_association: CONTRIBUTOR
diff_hunk: @@ -102,3 +99,58 @@ jobs:
             ${{ steps.setup-haskell.outputs.cabal-store }}
             dist-newstyle
           key: ${{ steps.cache.outputs.cache-primary-key }}
+
+  doc:
+    name: Publish haddocks to GitHub Pages
+    needs: test
+    env:
+      ghc: "9.10.1"
+      os: "ubuntu-24.04"
+    # For some reason, can't use ${{ env.os }} here, so just be sure to keep
+    # this in sync with the line above.
+    runs-on: "ubuntu-24.04"
path: .github/workflows/test.yml
user: login: langston-barrett
body: I would also be surprised if it failed... but I don't want to be surprised :smile: 
created_at: 2025-03-14T15:14:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -102,3 +99,58 @@ jobs:
             ${{ steps.setup-haskell.outputs.cabal-store }}
             dist-newstyle
           key: ${{ steps.cache.outputs.cache-primary-key }}
+
+  doc:
+    name: Publish haddocks to GitHub Pages
+    needs: test
+    env:
+      ghc: "9.10.1"
+      os: "ubuntu-24.04"
+    # For some reason, can't use ${{ env.os }} here, so just be sure to keep
+    # this in sync with the line above.
+    runs-on: "ubuntu-24.04"
+    concurrency:
+      # Only run one of these at a time because they update the global pages.
+      group: haddock
+      cancel-in-progress: false
+    permissions:
+      contents: read
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: haskell-actions/setup@v2
+        id: setup-haskell
+        with:
+          ghc-version: ${{ env.ghc }}
+      - uses: actions/cache/restore@v4
+        name: Restore Cabal cache
+        env:
+          key: cabal-${{ env.os }}-ghc${{ env.ghc }}
path: .github/workflows/test.yml
user: login: langston-barrett
body: Happy to add more detail if desired, just let me know.
created_at: 2025-03-14T16:59:13Z
author_association: CONTRIBUTOR

PR comments:
user: login: kquick
created_at: 2025-03-13T22:01:26Z
body: Umm... there were comments, where did they go, Github?!  I'll try to recreate after my meeting.
author_association: MEMBER
user: login: kquick
created_at: 2025-03-13T23:00:32Z
body: OK, comments added.

author_association: MEMBER
user: login: langston-barrett
created_at: 2025-03-14T13:40:05Z
body: Ah, it appears that Github Pages needs to be enabled in the repository settings. @kquick Do you have sufficient permissions to make that happen?
author_association: CONTRIBUTOR
user: login: langston-barrett
created_at: 2025-03-14T15:20:30Z
body: Ah, now it appears that I can't do a test deployment on this branch:

> Branch "lb/ci-haddock" is not allowed to deploy to github-pages due to environment protection rules.

@kquick do you want to try temporarily changing the rules in the settings, or would you rather try merging this? It will *probably* work...
author_association: CONTRIBUTOR
user: login: langston-barrett
created_at: 2025-03-14T15:54:13Z
body: Hmm...
```
Fetching artifact metadata for "github-pages" in this workflow run
Found 2 artifact(s)
Error: Fetching artifact metadata failed. Is githubstatus.com reporting issues with API requests, Pages, or Actions? Please re-run the deployment at a later time.
Error: Error: Multiple artifacts named "github-pages" were unexpectedly found for this workflow run. Artifact count is 2.
    at getArtifactMetadata (/home/runner/work/_actions/actions/deploy-pages/v4/src/internal/api-client.js:89:1)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Deployment.create (/home/runner/work/_actions/actions/deploy-pages/v4/src/internal/deployment.js:66:1)
    at main (/home/runner/work/_actions/actions/deploy-pages/v4/src/index.js:30:1)
Error: Error: Multiple artifacts named "github-pages" were unexpectedly found for this workflow run. Artifact count is 2.
```
FWIW, githubstatus.com is *not* reporting any issues.

I wonder if there is a way to give a more specific name to the artifact, perhaps including the workflow/job run ID?
author_association: CONTRIBUTOR
user: login: langston-barrett
created_at: 2025-03-14T15:56:08Z
body: Oh, this appears to be an issue specifically related to retrying CI builds with these actions:

- https://github.com/actions/upload-pages-artifact/issues/97
- https://github.com/actions/deploy-pages/issues/338

I will try adding a new commit instead.
author_association: CONTRIBUTOR
user: login: langston-barrett
created_at: 2025-03-14T16:10:38Z
body: It works! :rocket: http://galoisinc.github.io/parameterized-utils/
author_association: CONTRIBUTOR
user: login: langston-barrett
created_at: 2025-03-14T16:16:33Z
body: > I don't know how fully useful this will be since it will only be the docs for this package and so I expect cross references won't work.

Current status appears to be that identifiers for e.g. `base` are hyperlinked, but to bad destinations. Example: [`Applicative`](http://galoisinc.github.io/share/doc/ghc-9.10.1/html/libraries/base-4.20.0.0-4014/Control-Applicative.html#t:Applicative). There may be a `cabal haddock` flag we can use to fix this.
author_association: CONTRIBUTOR
user: login: langston-barrett
created_at: 2025-03-14T16:19:09Z
body: > Another thought is that if there are space constraints on gh-pages (although I'm not aware of any, and even with our larger projects I suspect the size of generated docs is rather minimal).

GitHub publishes [some documentation](https://docs.github.com/en/pages/getting-started-with-github-pages/about-github-pages#usage-limits) on usage limits. It's not clear to me from those docs whether or how much our organization is charged for the storage, but as you say, the docs are likely reasonably small. The artifact `.zip` is 265K.

> This will only provide the latest master version of the docs.

Indeed.
author_association: CONTRIBUTOR
user: login: langston-barrett
created_at: 2025-03-14T16:49:53Z
body: Unfortunately, `--haddock-quickjump` and `--haddock-hyperlink-source` both require changes to the build plan (resulting in rebuilding dependencies), so I'll just leave those off for now even though they would be nice to have.
author_association: CONTRIBUTOR
user: login: langston-barrett
created_at: 2025-03-14T16:50:56Z
body: I've got the links to other packages going to hackage.haskell.org now.
author_association: CONTRIBUTOR
user: login: langston-barrett
created_at: 2025-03-14T16:52:36Z
body: FWIW, these deployment errors seen on this PR seem to have to do with competition between the `push` and `pull_request` triggers. It seems we run two CI builds for each push to a branch with an associated PR. This won't be a problem when this is landed, because doc pushes will only happen on `main`.
author_association: CONTRIBUTOR