Repository: python/docsbuild-scripts
Pull request #235: Allow passing multiple branches to build via CLI
Author: login: hugovk
Created: 2024-12-08T19:18:21Z | Merged: 2025-04-11T14:46:23Z

A full set of Python 3.9-3.13 security+bugfix releases were made last week:

https://discuss.python.org/t/python-3-13-1-3-12-8-3-11-11-3-10-16-and-3-9-21-are-now-available/73214

The bugfix branches (3.12 and 3.13) are automatically built on cron.

The security branches (3.9-3.11) need manual rebuilds. We also have the final 3.8 to manually run. It's taken 16 and 13 hours for a full build of the first two so far (3.11 and 3.10; see https://github.com/python/docsbuild-scripts/issues/216).

We will certainly have big omnibus security releases in the future and instead of having to run each one separately:

```sh
/srv/docsbuild/venv/bin/python /srv/docsbuild/scripts/build_docs.py --branch 3.11
/srv/docsbuild/venv/bin/python /srv/docsbuild/scripts/build_docs.py --branch 3.10
/srv/docsbuild/venv/bin/python /srv/docsbuild/scripts/build_docs.py --branch 3.9
/srv/docsbuild/venv/bin/python /srv/docsbuild/scripts/build_docs.py --branch 3.8
```

It would be useful to pass all branch versions, in a similar way we can with `--languages`:

```sh
/srv/docsbuild/venv/bin/python /srv/docsbuild/scripts/build_docs.py --branch 3.11 3.10 3.9 3.8
```

This PR does that.

I added unit tests for the `Version.filter` method I changed, along with coverage. We can use coverage to guide adding more tests in the future.

---

I also added `--branches` as a new name for this argument and kept the old `--branch` so as not to break any other scripts, but we could change it later.

We could also make it's use more explicitly deprecated, like print a warning if `--branch` is used? Maybe it's safe to rename it right away?



Diff:
diff --git a/.coveragerc b/.coveragerc
new file mode 100644
index 0000000..0f12707
--- /dev/null
+++ b/.coveragerc
@@ -0,0 +1,7 @@
+# .coveragerc to control coverage.py
+
+[report]
+# Regexes for lines to exclude from consideration
+exclude_also =
+    # Don't complain if non-runnable code isn't run:
+    if __name__ == .__main__.:
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 2976bae..e272e76 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -33,3 +33,10 @@ jobs:
       - name: Tox tests
         run: |
           uvx --with tox-uv tox -e py
+
+      - name: Upload coverage
+        uses: codecov/codecov-action@v5
+        with:
+          flags: ${{ matrix.os }}
+          name: ${{ matrix.os }} Python ${{ matrix.python-version }}
+          token: ${{ secrets.CODECOV_ORG_TOKEN }}
diff --git a/README.md b/README.md
index 1b76bcd..d78bdb7 100644
--- a/README.md
+++ b/README.md
@@ -1,3 +1,8 @@
+# docsbuild-scripts
+
+[![GitHub Actions status](https://github.com/python/docsbuild-scripts/actions/workflows/test.yml/badge.svg)](https://github.com/python/docsbuild-scripts/actions/workflows/test.yml)
+[![Codecov](https://codecov.io/gh/python/docsbuild-scripts/branch/main/graph/badge.svg)](https://codecov.io/gh/python/docsbuild-scripts)
+
 This repository contains scripts for automatically building the Python
 documentation on [docs.python.org](https://docs.python.org).
 
@@ -12,7 +17,7 @@ python3 ./build_docs.py --quick --build-root ./build_root --www-root ./www --log
 ```
 
 If you don't need to build all translations of all branches, add
-`--language en --branch main`.
+`--languages en --branches main`.
 
 
 ## Check current version
diff --git a/build_docs.py b/build_docs.py
index 08ce611..ba60f53 100755
--- a/build_docs.py
+++ b/build_docs.py
@@ -86,16 +86,17 @@ def from_json(cls, data) -> Versions:
         )
         return cls(versions)
 
-    def filter(self, branch: str = "") -> Sequence[Version]:
+    def filter(self, branches: Sequence[str] = ()) -> Sequence[Version]:
         """Filter the given versions.
 
-        If *branch* is given, only *versions* matching *branch* are returned.
+        If *branches* is given, only *versions* matching *branches* are returned.
 
         Else all live versions are returned (this means no EOL and no
         security-fixes branches).
         """
-        if branch:
-            return [v for v in self if branch in (v.name, v.branch_or_tag)]
+        if branches:
+            branches = frozenset(branches)
+            return [v for v in self if {v.name, v.branch_or_tag} & branches]
         return [v for v in self if v.status not in {"EOL", "security-fixes"}]
 
     @property
@@ -935,9 +936,10 @@ def parse_args():
     )
     parser.add_argument(
         "-b",
-        "--branch",
+        "--branches",
+        nargs="*",
         metavar="3.12",
-        help="Version to build (defaults to all maintained branches).",
+        help="Versions to build (defaults to all maintained branches).",
     )
     parser.add_argument(
         "-r",
@@ -971,7 +973,6 @@ def parse_args():
     )
     parser.add_argument(
         "--languages",
-        "--language",
         nargs="*",
         help="Language translation, as a PEP 545 language tag like"
         " 'fr' or 'pt-br'. "
@@ -1045,10 +1046,10 @@ def build_docs(args: argparse.Namespace) -> bool:
     # This runs languages in config.toml order and versions newest first.
     todo = [
         (version, language)
-        for version in versions.filter(args.branch)
+        for version in versions.filter(args.branches)
         for language in reversed(languages.filter(args.languages))
     ]
-    del args.branch
+    del args.branches
     del args.languages
 
     build_succeeded = set()
diff --git a/tests/test_build_docs_versions.py b/tests/test_build_docs_versions.py
new file mode 100644
index 0000000..5ed9bcb
--- /dev/null
+++ b/tests/test_build_docs_versions.py
@@ -0,0 +1,62 @@
+from build_docs import Versions, Version
+
+
+def test_filter_default() -> None:
+    # Arrange
+    versions = Versions([
+        Version("3.14", status="feature"),
+        Version("3.13", status="bugfix"),
+        Version("3.12", status="bugfix"),
+        Version("3.11", status="security"),
+        Version("3.10", status="security"),
+        Version("3.9", status="security"),
+    ])
+
+    # Act
+    filtered = versions.filter()
+
+    # Assert
+    assert filtered == [
+        Version("3.14", status="feature"),
+        Version("3.13", status="bugfix"),
+        Version("3.12", status="bugfix"),
+    ]
+
+
+def test_filter_one() -> None:
+    # Arrange
+    versions = Versions([
+        Version("3.14", status="feature"),
+        Version("3.13", status="bugfix"),
+        Version("3.12", status="bugfix"),
+        Version("3.11", status="security"),
+        Version("3.10", status="security"),
+        Version("3.9", status="security"),
+    ])
+
+    # Act
+    filtered = versions.filter(["3.13"])
+
+    # Assert
+    assert filtered == [Version("3.13", status="security")]
+
+
+def test_filter_multiple() -> None:
+    # Arrange
+    versions = Versions([
+        Version("3.14", status="feature"),
+        Version("3.13", status="bugfix"),
+        Version("3.12", status="bugfix"),
+        Version("3.11", status="security"),
+        Version("3.10", status="security"),
+        Version("3.9", status="security"),
+    ])
+
+    # Act
+    filtered = versions.filter(["3.13", "3.14"])
+
+    # Assert
+    assert filtered == [
+        Version("3.14", status="feature"),
+        Version("3.13", status="security"),
+    ]
diff --git a/tox.ini b/tox.ini
index 56c6420..12efcdf 100644
--- a/tox.ini
+++ b/tox.ini
@@ -12,8 +12,19 @@ skip_install = true
 deps =
     -r requirements.txt
     pytest
+    pytest-cov
+pass_env =
+    FORCE_COLOR
+set_env =
+    COVERAGE_CORE = sysmon
 commands =
-    {envpython} -m pytest {posargs}
+    {envpython} -m pytest \
+      --cov . \
+      --cov tests \
+      --cov-report html \
+      --cov-report term \
+      --cov-report xml \
+      {posargs}
 
 [testenv:lint]
 skip_install = true


Reviews:
user: login: m-aciek
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2024-12-08T21:02:46Z
user: login: hugovk
state: COMMENTED
author_association: MEMBER
submitted_at: 2024-12-08T21:46:35Z
user: login: hugovk
state: COMMENTED
author_association: MEMBER
submitted_at: 2024-12-08T21:46:42Z
user: login: AA-Turner
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-11T14:23:28Z
user: login: AA-Turner
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-11T14:45:59Z

Review comments:
diff_hunk: @@ -12,7 +17,7 @@ python3 ./build_docs.py --quick --build-root ./build_root --www-root ./www --log
 ```
 
 If you don't need to build all translations of all branches, add
-`--language en --branch main`.
+`--language en --branches main`.
path: README.md
user: login: m-aciek
body: By the way, it looks like the first flag name should be `languages`.
created_at: 2024-12-08T20:51:51Z
author_association: CONTRIBUTOR
diff_hunk: @@ -12,7 +17,7 @@ python3 ./build_docs.py --quick --build-root ./build_root --www-root ./www --log
 ```
 
 If you don't need to build all translations of all branches, add
-`--language en --branch main`.
+`--language en --branches main`.
path: README.md
user: login: hugovk
body: Well spotted, will fix :)
created_at: 2024-12-08T21:46:35Z
author_association: MEMBER
diff_hunk: @@ -12,7 +17,7 @@ python3 ./build_docs.py --quick --build-root ./build_root --www-root ./www --log
 ```
 
 If you don't need to build all translations of all branches, add
-`--language en --branch main`.
+`--language en --branches main`.
path: README.md
user: login: hugovk
body: ```suggestion
`--languages en --branches main`.
```
created_at: 2024-12-08T21:46:41Z
author_association: MEMBER
diff_hunk: @@ -86,16 +86,18 @@ def from_json(cls, data) -> Versions:
         )
         return cls(versions)
 
-    def filter(self, branch: str = "") -> Sequence[Version]:
+    def filter(self, branches: list[str] = None) -> Sequence[Version]:
path: build_docs.py
user: login: AA-Turner
body: ```suggestion
    def filter(self, branches: Sequence[str] = ()) -> Sequence[Version]:
```
created_at: 2025-04-11T14:21:29Z
author_association: MEMBER
diff_hunk: @@ -86,16 +86,18 @@ def from_json(cls, data) -> Versions:
         )
         return cls(versions)
 
-    def filter(self, branch: str = "") -> Sequence[Version]:
+    def filter(self, branches: list[str] = None) -> Sequence[Version]:
         """Filter the given versions.
 
-        If *branch* is given, only *versions* matching *branch* are returned.
+        If *branches* is given, only *versions* matching *branches* are returned.
 
         Else all live versions are returned (this means no EOL and no
         security-fixes branches).
         """
-        if branch:
-            return [v for v in self if branch in (v.name, v.branch_or_tag)]
+        if branches:
+            return [
+                v for v in self if v.name in branches or v.branch_or_tag in branches
+            ]
path: build_docs.py
user: login: AA-Turner
body: ```suggestion
        if branches:
            branches = frozenset(branches)
            return [
                v for v in self if {v.name, v.branch_or_tag} & branches
            ]
```
created_at: 2025-04-11T14:22:36Z
author_association: MEMBER
diff_hunk: @@ -994,6 +996,9 @@ def parse_args():
         version_info()
         sys.exit(0)
     del args.version
+    if args.branch and not args.branches:
+        args.branches = args.branch
+        del args.branch
path: build_docs.py
user: login: AA-Turner
body: ```suggestion
```
created_at: 2025-04-11T14:23:02Z
author_association: MEMBER

PR comments:
user: login: hugovk
created_at: 2025-04-11T11:35:17Z
body: Merge conflict fixed following refactor in https://github.com/python/docsbuild-scripts/pull/254.

Added https://github.com/python/docsbuild-scripts/pull/235/commits/f7e617d0da4c48d3cae52cc7d4ef950b9808d6e1 to use https://docs.python.org/3/library/argparse.html#deprecated, added in 3.13, to make the deprecation more visible.
author_association: MEMBER
user: login: AA-Turner
created_at: 2025-04-11T13:52:43Z
body: We should just remove the singular variants of the flags. `build_docs.py` is only invoked by `psf-salt`, tests in `python-docs-theme`, and users with SSH access to the docs server. I added ``--language`` as a hot-fix in 6d537969095c21f34406ff954535020b690f0d3d as the incorrect spelling of `--language` had spread, but was permitted due to `allow_abbrev` defaulting to `True` in `ArgumentParser`. See https://github.com/python/psf-salt/pull/560 and https://github.com/python/python-docs-theme/pull/228 to update usage to ``--languages``.

For `--version`, it's not used externally, so we can just remove it.

A
author_association: MEMBER