Repository: mne-tools/mne-python

Issue:
doc build error on windows
@CarinaFo ran into this error when building the docs locally on windows

```
The chromedriver version (99.0.4844.51) detected in PATH at C:\Program Files\chromedriver\chromedriver.exe might not be compatible with the detected chrome version (117.0.5938.134); currently, chromedriver 117.0.5938.149 is recommended for chrome 117., so it is advised to delete the driver in PATH and retry
Exception ignored in: <_io.FileIO name='nul' mode='wb' closefd=True>
Traceback (most recent call last):
  File "C:\Users\Carina\Documents\mne_howtocontribute_workshop\mne-python\doc\sphinxext\contrib_avatars.py", line 21, in generate_contrib_avatars
    driver = webdriver.Firefox(options=options)
    ^^^^^^
ResourceWarning: unclosed file <_io.BufferedWriter name='nul'>
C:\Users\Carina\mambaforge\envs\mnedev\Lib\site-packages\sphinx\events.py:97: ResourceWarning: unclosed file <_io.BufferedWriter name='nul'>
  results.append(listener.handler(self.app,args))
ResourceWarning: Enable tracemalloc to get the object allocation traceback
```

this did not kill the doc build process, but the following later error did:

```
Traceback (most recent call last):
  File "C:\Users\Carina\mambaforge\envs\mnedev\Lib\site-packages\sphinx\events.py", line 97, in emit
    results.append(listener.handler(self.app, *args))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Carina\mambaforge\envs\mnedev\Lib\site-packages\numpydoc\numpydoc.py", line 214, in mangle_docstrings
    report = validate(doc)
             ^^^^^^^^^^^^^
  File "C:\Users\Carina\mambaforge\envs\mnedev\Lib\site-packages\numpydoc\validate.py", line 617, in validate
    ignore_validation_comments = extract_ignore_validation_comments(
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Carina\mambaforge\envs\mnedev\Lib\site-packages\numpydoc\validate.py", line 145, in extract_ignore_validation_comments
    for token in tokenize.generate_tokens(file.readline):
  File "C:\Users\Carina\mambaforge\envs\mnedev\Lib\tokenize.py", line 454, in _tokenize
    line = readline()
           ^^^^^^^^^^
  File "C:\Users\Carina\mambaforge\envs\mnedev\Lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 3922: character maps to <undefined>
```

The first error is related to selenium building the contributor avatars, and will need a bit of debugging (it's also popping open the browser, which it shouldn't --- supposed to be headless).

The second error can be avoided by doing `export PYTHONUTF8=1` in gitbash before running the doc build `make` command. After that, we get a different error:

```
Traceback (most recent call last):
"C:\Users\Carina\mambaforge\envs\mnedev\Lib\site-packages\sphinx\events.py", line 97, in emit
    results.append(listener.handler(self.app, *args))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Carina\Documents\mne_howtocontribute_workshop\mne-python\doc\conf.py", line 1783, in make_redirects
    if fr_path.split("/")[-2] in (
       ~~~~~~^^^^
IndexError: list index out of range
```


Discussion:
`tokenize` is almost certainly a NumpyDoc bug here:

https://github.com/numpy/numpydoc/blob/b4135ce76e722088e2975ec01ca61d7b69b4bad2/numpydoc/validate.py#L138-L140

We should use `encoding="utf-8"` probably to unify Unix + Windows. Can you open a numpydoc issue for that part?
@larsoner Should we add PYTHONUTF8=1 to the conda environment deployed on Windows through our installers?
> Can you open a numpydoc issue for that part?

https://github.com/numpy/numpydoc/pull/510
> @larsoner Should we add PYTHONUTF8=1 to the conda environment deployed on Windows through our installers?

My initial reaction is "no" because doing more magic in our installers just delays confusion -- code will work in our installers but not in regular conda, venvs, etc. Better if people (and packages like `numpydoc`) fixe their code. But as @drammock pointed out in the numpydoc issue this will become standard in a couple of years in 3.15 so maybe it's okay in a future-proofing sense. 🤷 
just a note that the first traceback (the ResourceWarning) is already fixed upstream by https://github.com/SeleniumHQ/selenium/pull/12637.  Since it doesn't actually halt the docbuild I think there's no action needed except to wait for a new selenium release.

Resolving patch:
diff --git a/doc/conf.py b/doc/conf.py
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -9,6 +9,7 @@
 import gc
 from importlib.metadata import metadata
 import os
+from pathlib import Path
 import subprocess
 import sys
 import time
@@ -60,8 +61,8 @@
 # (Sphinx looks at variable changes and rewrites all files if some change)
 copyright = (
     f'2012–{td.year}, MNE Developers. Last updated <time datetime="{td.isoformat()}" class="localized">{td.strftime("%Y-%m-%d %H:%M %Z")}</time>\n'  # noqa: E501
-    '<script type="text/javascript">$(function () { $("time.localized").each(function () { var el = $(this); el.text(new Date(el.attr("datetime")).toLocaleString([], {dateStyle: "medium", timeStyle: "long"})); }); } )</script>'
-)  # noqa: E501
+    '<script type="text/javascript">$(function () { $("time.localized").each(function () { var el = $(this); el.text(new Date(el.attr("datetime")).toLocaleString([], {dateStyle: "medium", timeStyle: "long"})); }); } )</script>'  # noqa: E501
+)
 if os.getenv("MNE_FULL_DATE", "false").lower() != "true":
     copyright = f"2012–{td.year}, MNE Developers. Last updated locally."
 
@@ -1636,87 +1637,70 @@ def reset_warnings(gallery_conf, fname):
 custom_redirects = {
     # Custom redirects (one HTML path to another, relative to outdir)
     # can be added here as fr->to key->value mappings
-    "install/contributing.html": "development/contributing.html",
-    "overview/roadmap.html": "development/roadmap.html",
-    "whats_new.html": "development/whats_new.html",
-    f"{tu}/evoked/plot_eeg_erp.html": f"{tu}/evoked/30_eeg_erp.html",
-    f"{tu}/evoked/plot_whitened.html": f"{tu}/evoked/40_whitened.html",
-    f"{tu}/misc/plot_modifying_data_inplace.html": f"{tu}/intro/15_inplace.html",
-    f"{tu}/misc/plot_report.html": f"{tu}/intro/70_report.html",
-    f"{tu}/misc/plot_seeg.html": f"{tu}/clinical/20_seeg.html",
-    f"{tu}/misc/plot_ecog.html": f"{tu}/clinical/30_ecog.html",
-    f"{tu}/{ml}/plot_receptive_field.html": f"{tu}/{ml}/30_strf.html",
-    f"{tu}/{ml}/plot_sensors_decoding.html": f"{tu}/{ml}/50_decoding.html",
-    f"{tu}/{sm}/plot_background_freesurfer.html": f"{tu}/{fw}/10_background_freesurfer.html",  # noqa E501
-    f"{tu}/{sm}/plot_source_alignment.html": f"{tu}/{fw}/20_source_alignment.html",
-    f"{tu}/{sm}/plot_forward.html": f"{tu}/{fw}/30_forward.html",
-    f"{tu}/{sm}/plot_eeg_no_mri.html": f"{tu}/{fw}/35_eeg_no_mri.html",
-    f"{tu}/{sm}/plot_background_freesurfer_mne.html": f"{tu}/{fw}/50_background_freesurfer_mne.html",  # noqa E501
-    f"{tu}/{sm}/plot_fix_bem_in_blender.html": f"{tu}/{fw}/80_fix_bem_in_blender.html",
-    f"{tu}/{sm}/plot_compute_covariance.html": f"{tu}/{fw}/90_compute_covariance.html",
-    f"{tu}/{sm}/plot_object_source_estimate.html": f"{tu}/{nv}/10_stc_class.html",
-    f"{tu}/{sm}/plot_dipole_fit.html": f"{tu}/{nv}/20_dipole_fit.html",
-    f"{tu}/{sm}/plot_mne_dspm_source_localization.html": f"{tu}/{nv}/30_mne_dspm_loreta.html",  # noqa E501
-    f"{tu}/{sm}/plot_dipole_orientations.html": f"{tu}/{nv}/35_dipole_orientations.html",  # noqa E501
-    f"{tu}/{sm}/plot_mne_solutions.html": f"{tu}/{nv}/40_mne_fixed_free.html",
-    f"{tu}/{sm}/plot_beamformer_lcmv.html": f"{tu}/{nv}/50_beamformer_lcmv.html",
-    f"{tu}/{sm}/plot_visualize_stc.html": f"{tu}/{nv}/60_visualize_stc.html",
-    f"{tu}/{sm}/plot_eeg_mri_coords.html": f"{tu}/{nv}/70_eeg_mri_coords.html",
-    f"{tu}/{sd}/plot_brainstorm_phantom_elekta.html": f"{tu}/{nv}/80_brainstorm_phantom_elekta.html",  # noqa E501
-    f"{tu}/{sd}/plot_brainstorm_phantom_ctf.html": f"{tu}/{nv}/85_brainstorm_phantom_ctf.html",  # noqa E501
-    f"{tu}/{sd}/plot_phantom_4DBTi.html": f"{tu}/{nv}/90_phantom_4DBTi.html",
-    f"{tu}/{sd}/plot_brainstorm_auditory.html": f"{tu}/io/60_ctf_bst_auditory.html",
-    f"{tu}/{sd}/plot_sleep.html": f"{tu}/clinical/60_sleep.html",
-    f"{tu}/{di}/plot_background_filtering.html": f"{tu}/{pr}/25_background_filtering.html",  # noqa E501
-    f"{tu}/{di}/plot_background_statistics.html": f"{tu}/{sn}/10_background_stats.html",
-    f"{tu}/{sn}/plot_stats_cluster_erp.html": f"{tu}/{sn}/20_erp_stats.html",
-    f"{tu}/{sn}/plot_stats_cluster_1samp_test_time_frequency.html": f"{tu}/{sn}/40_cluster_1samp_time_freq.html",  # noqa E501
-    f"{tu}/{sn}/plot_stats_cluster_time_frequency.html": f"{tu}/{sn}/50_cluster_between_time_freq.html",  # noqa E501
-    f"{tu}/{sn}/plot_stats_spatio_temporal_cluster_sensors.html": f"{tu}/{sn}/75_cluster_ftest_spatiotemporal.html",  # noqa E501
-    f"{tu}/{sr}/plot_stats_cluster_spatio_temporal.html": f"{tu}/{sr}/20_cluster_1samp_spatiotemporal.html",  # noqa E501
-    f"{tu}/{sr}/plot_stats_cluster_spatio_temporal_2samp.html": f"{tu}/{sr}/30_cluster_ftest_spatiotemporal.html",  # noqa E501
-    f"{tu}/{sr}/plot_stats_cluster_spatio_temporal_repeated_measures_anova.html": f"{tu}/{sr}/60_cluster_rmANOVA_spatiotemporal.html",  # noqa E501
-    f"{tu}/{sr}/plot_stats_cluster_time_frequency_repeated_measures_anova.html": f"{tu}/{sn}/70_cluster_rmANOVA_time_freq.html",  # noqa E501
-    f"{tu}/{tf}/plot_sensors_time_frequency.html": f"{tu}/{tf}/20_sensors_time_frequency.html",  # noqa E501
-    f"{tu}/{tf}/plot_ssvep.html": f"{tu}/{tf}/50_ssvep.html",
-    f"{tu}/{si}/plot_creating_data_structures.html": f"{tu}/{si}/10_array_objs.html",
-    f"{tu}/{si}/plot_point_spread.html": f"{tu}/{si}/70_point_spread.html",
-    f"{tu}/{si}/plot_dics.html": f"{tu}/{si}/80_dics.html",
-    f"{tu}/{tf}/plot_eyetracking.html": f"{tu}/{pr}/90_eyetracking_data.html",
-    f"{ex}/{co}/mne_inverse_label_connectivity.html": f"{mne_conn}/{ex}/mne_inverse_label_connectivity.html",  # noqa E501
-    f"{ex}/{co}/cwt_sensor_connectivity.html": f"{mne_conn}/{ex}/cwt_sensor_connectivity.html",  # noqa E501
-    f"{ex}/{co}/mixed_source_space_connectivity.html": f"{mne_conn}/{ex}/mixed_source_space_connectivity.html",  # noqa E501
-    f"{ex}/{co}/mne_inverse_coherence_epochs.html": f"{mne_conn}/{ex}/mne_inverse_coherence_epochs.html",  # noqa E501
-    f"{ex}/{co}/mne_inverse_connectivity_spectrum.html": f"{mne_conn}/{ex}/mne_inverse_connectivity_spectrum.html",  # noqa E501
-    f"{ex}/{co}/mne_inverse_envelope_correlation_volume.html": f"{mne_conn}/{ex}/mne_inverse_envelope_correlation_volume.html",  # noqa E501
-    f"{ex}/{co}/mne_inverse_envelope_correlation.html": f"{mne_conn}/{ex}/mne_inverse_envelope_correlation.html",  # noqa E501
-    f"{ex}/{co}/mne_inverse_psi_visual.html": f"{mne_conn}/{ex}/mne_inverse_psi_visual.html",  # noqa E501
-    f"{ex}/{co}/sensor_connectivity.html": f"{mne_conn}/{ex}/sensor_connectivity.html",
-    f"{ex}/{vi}/publication_figure.html": f"{tu}/{vi}/10_publication_figure.html",
-    f"{ex}/{vi}/sensor_noise_level.html": f"{tu}/{pr}/50_artifact_correction_ssp.html",
+    "install/contributing": "development/contributing",
+    "overview/cite": "documentation/cite",
+    "overview/get_help": "help/index",
+    "overview/roadmap": "development/roadmap",
+    "whats_new": "development/whats_new",
+    f"{tu}/evoked/plot_eeg_erp": f"{tu}/evoked/30_eeg_erp",
+    f"{tu}/evoked/plot_whitened": f"{tu}/evoked/40_whitened",
+    f"{tu}/misc/plot_modifying_data_inplace": f"{tu}/intro/15_inplace",
+    f"{tu}/misc/plot_report": f"{tu}/intro/70_report",
+    f"{tu}/misc/plot_seeg": f"{tu}/clinical/20_seeg",
+    f"{tu}/misc/plot_ecog": f"{tu}/clinical/30_ecog",
+    f"{tu}/{ml}/plot_receptive_field": f"{tu}/{ml}/30_strf",
+    f"{tu}/{ml}/plot_sensors_decoding": f"{tu}/{ml}/50_decoding",
+    f"{tu}/{sm}/plot_background_freesurfer": f"{tu}/{fw}/10_background_freesurfer",
+    f"{tu}/{sm}/plot_source_alignment": f"{tu}/{fw}/20_source_alignment",
+    f"{tu}/{sm}/plot_forward": f"{tu}/{fw}/30_forward",
+    f"{tu}/{sm}/plot_eeg_no_mri": f"{tu}/{fw}/35_eeg_no_mri",
+    f"{tu}/{sm}/plot_background_freesurfer_mne": f"{tu}/{fw}/50_background_freesurfer_mne",  # noqa E501
+    f"{tu}/{sm}/plot_fix_bem_in_blender": f"{tu}/{fw}/80_fix_bem_in_blender",
+    f"{tu}/{sm}/plot_compute_covariance": f"{tu}/{fw}/90_compute_covariance",
+    f"{tu}/{sm}/plot_object_source_estimate": f"{tu}/{nv}/10_stc_class",
+    f"{tu}/{sm}/plot_dipole_fit": f"{tu}/{nv}/20_dipole_fit",
+    f"{tu}/{sm}/plot_mne_dspm_source_localization": f"{tu}/{nv}/30_mne_dspm_loreta",
+    f"{tu}/{sm}/plot_dipole_orientations": f"{tu}/{nv}/35_dipole_orientations",
+    f"{tu}/{sm}/plot_mne_solutions": f"{tu}/{nv}/40_mne_fixed_free",
+    f"{tu}/{sm}/plot_beamformer_lcmv": f"{tu}/{nv}/50_beamformer_lcmv",
+    f"{tu}/{sm}/plot_visualize_stc": f"{tu}/{nv}/60_visualize_stc",
+    f"{tu}/{sm}/plot_eeg_mri_coords": f"{tu}/{nv}/70_eeg_mri_coords",
+    f"{tu}/{sd}/plot_brainstorm_phantom_elekta": f"{tu}/{nv}/80_brainstorm_phantom_elekta",  # noqa E501
+    f"{tu}/{sd}/plot_brainstorm_phantom_ctf": f"{tu}/{nv}/85_brainstorm_phantom_ctf",
+    f"{tu}/{sd}/plot_phantom_4DBTi": f"{tu}/{nv}/90_phantom_4DBTi",
+    f"{tu}/{sd}/plot_brainstorm_auditory": f"{tu}/io/60_ctf_bst_auditory",
+    f"{tu}/{sd}/plot_sleep": f"{tu}/clinical/60_sleep",
+    f"{tu}/{di}/plot_background_filtering": f"{tu}/{pr}/25_background_filtering",
+    f"{tu}/{di}/plot_background_statistics": f"{tu}/{sn}/10_background_stats",
+    f"{tu}/{sn}/plot_stats_cluster_erp": f"{tu}/{sn}/20_erp_stats",
+    f"{tu}/{sn}/plot_stats_cluster_1samp_test_time_frequency": f"{tu}/{sn}/40_cluster_1samp_time_freq",  # noqa E501
+    f"{tu}/{sn}/plot_stats_cluster_time_frequency": f"{tu}/{sn}/50_cluster_between_time_freq",  # noqa E501
+    f"{tu}/{sn}/plot_stats_spatio_temporal_cluster_sensors": f"{tu}/{sn}/75_cluster_ftest_spatiotemporal",  # noqa E501
+    f"{tu}/{sr}/plot_stats_cluster_spatio_temporal": f"{tu}/{sr}/20_cluster_1samp_spatiotemporal",  # noqa E501
+    f"{tu}/{sr}/plot_stats_cluster_spatio_temporal_2samp": f"{tu}/{sr}/30_cluster_ftest_spatiotemporal",  # noqa E501
+    f"{tu}/{sr}/plot_stats_cluster_spatio_temporal_repeated_measures_anova": f"{tu}/{sr}/60_cluster_rmANOVA_spatiotemporal",  # noqa E501
+    f"{tu}/{sr}/plot_stats_cluster_time_frequency_repeated_measures_anova": f"{tu}/{sn}/70_cluster_rmANOVA_time_freq",  # noqa E501
+    f"{tu}/{tf}/plot_sensors_time_frequency": f"{tu}/{tf}/20_sensors_time_frequency",
+    f"{tu}/{tf}/plot_ssvep": f"{tu}/{tf}/50_ssvep",
+    f"{tu}/{si}/plot_creating_data_structures": f"{tu}/{si}/10_array_objs",
+    f"{tu}/{si}/plot_point_spread": f"{tu}/{si}/70_point_spread",
+    f"{tu}/{si}/plot_dics": f"{tu}/{si}/80_dics",
+    f"{tu}/{tf}/plot_eyetracking": f"{tu}/{pr}/90_eyetracking_data",
+    f"{ex}/{co}/mne_inverse_label_connectivity": f"{mne_conn}/{ex}/mne_inverse_label_connectivity",  # noqa E501
+    f"{ex}/{co}/cwt_sensor_connectivity": f"{mne_conn}/{ex}/cwt_sensor_connectivity",
+    f"{ex}/{co}/mixed_source_space_connectivity": f"{mne_conn}/{ex}/mixed_source_space_connectivity",  # noqa E501
+    f"{ex}/{co}/mne_inverse_coherence_epochs": f"{mne_conn}/{ex}/mne_inverse_coherence_epochs",  # noqa E501
+    f"{ex}/{co}/mne_inverse_connectivity_spectrum": f"{mne_conn}/{ex}/mne_inverse_connectivity_spectrum",  # noqa E501
+    f"{ex}/{co}/mne_inverse_envelope_correlation_volume": f"{mne_conn}/{ex}/mne_inverse_envelope_correlation_volume",  # noqa E501
+    f"{ex}/{co}/mne_inverse_envelope_correlation": f"{mne_conn}/{ex}/mne_inverse_envelope_correlation",  # noqa E501
+    f"{ex}/{co}/mne_inverse_psi_visual": f"{mne_conn}/{ex}/mne_inverse_psi_visual",
+    f"{ex}/{co}/sensor_connectivity": f"{mne_conn}/{ex}/sensor_connectivity",
+    f"{ex}/{vi}/publication_figure": f"{tu}/{vi}/10_publication_figure",
+    f"{ex}/{vi}/sensor_noise_level": f"{tu}/{pr}/50_artifact_correction_ssp",
 }
 
-
-def check_existing_redirect(path):
-    """Make sure existing HTML files are redirects, before overwriting."""
-    if os.path.isfile(path):
-        with open(path, "r") as fid:
-            for _ in range(8):
-                next(fid)
-            line = fid.readline()
-            assert "Page Redirection" in line, line
-
-
-def make_redirects(app, exception):
-    """Make HTML redirects."""
-    # https://www.sphinx-doc.org/en/master/extdev/appapi.html
-    # Adapted from sphinxcontrib/redirects (BSD-2-Clause)
-    if not (
-        isinstance(app.builder, sphinx.builders.html.StandaloneHTMLBuilder)
-        and exception is None
-    ):
-        return
-    TEMPLATE = """\
+# Adapted from sphinxcontrib/redirects (BSD-2-Clause)
+REDIRECT_TEMPLATE = """\
 <!DOCTYPE HTML>
 <html lang="en-US">
     <head>
@@ -1730,66 +1714,104 @@ def make_redirects(app, exception):
     <body>
         If you are not redirected automatically, follow this <a href='{to}'>link</a>.
     </body>
-</html>"""  # noqa: E501
-    sphinx_gallery_conf = app.config["sphinx_gallery_conf"]
-    for src_dir, out_dir in zip(
-        sphinx_gallery_conf["examples_dirs"], sphinx_gallery_conf["gallery_dirs"]
-    ):
-        root = os.path.abspath(os.path.join(app.srcdir, src_dir))
+</html>"""
+
+
+def check_existing_redirect(path):
+    """Make sure existing HTML files are redirects, before overwriting."""
+    if path.is_file():
+        with open(path, "r") as fid:
+            for _ in range(8):
+                next(fid)
+            line = fid.readline()
+            if "Page Redirection" not in line:
+                raise RuntimeError(
+                    "Attempted overwrite of HTML file with a redirect, where the "
+                    "original file was not already a redirect."
+                )
+
+
+def _check_valid_builder(app, exception):
+    valid_builder = isinstance(app.builder, sphinx.builders.html.StandaloneHTMLBuilder)
+    return valid_builder and exception is None
+
+
+def make_gallery_redirects(app, exception):
+    """Make HTML redirects for our sphinx gallery pages."""
+    if not _check_valid_builder(app, exception):
+        return
+    sg_conf = app.config["sphinx_gallery_conf"]
+    for src_dir, out_dir in zip(sg_conf["examples_dirs"], sg_conf["gallery_dirs"]):
+        root = (Path(app.srcdir) / src_dir).resolve()
         fnames = [
-            os.path.join(os.path.relpath(dirpath, root), fname)
-            for dirpath, _, fnames in os.walk(root)
-            for fname in fnames
-            if fname in needed_plot_redirects
+            pyfile.relative_to(root)
+            for pyfile in root.rglob(r"**/*.py")
+            if pyfile.name in needed_plot_redirects
         ]
         # plot_ redirects
         for fname in fnames:
-            dirname = os.path.join(app.outdir, out_dir, os.path.dirname(fname))
-            to_fname = os.path.splitext(os.path.basename(fname))[0] + ".html"
+            dirname = Path(app.outdir) / out_dir / fname.parent
+            to_fname = fname.with_suffix(".html").name
             fr_fname = f"plot_{to_fname}"
-            to_path = os.path.join(dirname, to_fname)
-            fr_path = os.path.join(dirname, fr_fname)
-            assert os.path.isfile(to_path), (fname, to_path)
+            to_path = dirname / to_fname
+            fr_path = dirname / fr_fname
+            assert to_path.is_file(), (fname, to_path)
             with open(fr_path, "w") as fid:
-                fid.write(TEMPLATE.format(to=to_fname))
+                fid.write(REDIRECT_TEMPLATE.format(to=to_fname))
         sphinx_logger.info(
             f"Added {len(fnames):3d} HTML plot_* redirects for {out_dir}"
         )
-    # API redirects
+
+
+def make_api_redirects(app, exception):
+    """Make HTML redirects for our API pages."""
+    if not _check_valid_builder(app, exception):
+        return
+
     for page in api_redirects:
         fname = f"{page}.html"
-        fr_path = os.path.join(app.outdir, fname)
-        to_path = os.path.join(app.outdir, "api", fname)
+        fr_path = Path(app.outdir) / fname
+        to_path = Path(app.outdir) / "api" / fname
         # allow overwrite if existing file is just a redirect
         check_existing_redirect(fr_path)
         with open(fr_path, "w") as fid:
-            fid.write(TEMPLATE.format(to=to_path))
+            fid.write(REDIRECT_TEMPLATE.format(to=to_path))
     sphinx_logger.info(f"Added {len(api_redirects):3d} HTML API redirects")
-    # custom redirects
-    for fr, to in custom_redirects.items():
-        if not to.startswith("http"):
-            assert os.path.isfile(os.path.join(app.outdir, to)), to
-            # handle links to sibling folders
-            path_parts = to.split("/")
-            if tu in path_parts:
-                path_parts = [".."] + path_parts[(path_parts.index(tu) + 1) :]
-            to = os.path.join(*path_parts)
-        assert to.endswith("html"), to
-        fr_path = os.path.join(app.outdir, fr)
-        assert fr_path.endswith("html"), fr_path
-        # allow overwrite if existing file is just a redirect
+
+
+def make_custom_redirects(app, exception):
+    """Make HTML redirects for miscellaneous pages."""
+    if not _check_valid_builder(app, exception):
+        return
+
+    for _fr, _to in custom_redirects.items():
+        fr = f"{_fr}.html"
+        to = f"{_to}.html"
+        fr_path = Path(app.outdir) / fr
         check_existing_redirect(fr_path)
-        # handle folders that no longer exist
-        if fr_path.split("/")[-2] in (
+        if to.startswith("http"):
+            to_path = to
+        else:
+            to_path = Path(app.outdir) / to
+            assert to_path.is_file(), to_path
+        # recreate folders that no longer exist
+        defunct_gallery_folders = (
             "misc",
             "discussions",
             "source-modeling",
             "sample-datasets",
             "connectivity",
+        )
+        parts = fr_path.relative_to(Path(app.outdir)).parts
+        if (
+            len(parts) > 1  # whats_new violates this
+            and parts[1] in defunct_gallery_folders
+            and not fr_path.parent.exists()
         ):
-            os.makedirs(os.path.dirname(fr_path), exist_ok=True)
+            os.makedirs(fr_path.parent, exist_ok=True)
+        # write the redirect
         with open(fr_path, "w") as fid:
-            fid.write(TEMPLATE.format(to=to))
+            fid.write(REDIRECT_TEMPLATE.format(to=to_path))
     sphinx_logger.info(f"Added {len(custom_redirects):3d} HTML custom redirects")
 
 
@@ -1818,5 +1840,7 @@ def setup(app):
     app.connect("autodoc-process-docstring", append_attr_meth_examples)
     report_scraper.app = app
     app.connect("builder-inited", report_scraper.copyfiles)
-    app.connect("build-finished", make_redirects)
+    app.connect("build-finished", make_gallery_redirects)
+    app.connect("build-finished", make_api_redirects)
+    app.connect("build-finished", make_custom_redirects)
     app.connect("build-finished", make_version)


Test patch: