Repository: mne-tools/mne-python

Issue:
RFC: doc/ folder reorg
The organization of files in our `doc/` folder is out of control; every time I need to edit a page in our docs it is a challenge to find the correct `.rst` source file. Below (hidden under "details") I lay out a proposal to have subfolders within `doc/` mirror the top-level sections of our site.  Given that there are a few outstanding issues with doc content and organization:

- #10707 
- #11370 
- #11609  

...I feel like now is the time to tackle this.  

Note that a couple years ago there was some discussion of reorganizing the website pages (I think around the time of #8757 or #9292, when the API pages were split up?) about "not breaking people's browser bookmarks" (I think @agramfort was the one who raised this objection?).  Nowadays we already track a bunch of custom redirects in `conf.py` so IMO that objection is not nearly as strong today as it was then.

<details>

```
.
├── auto_examples
│   └── ... 16 items
│
├── auto_tutorials
│   └── ... 20 items
│
├── bibliography.rst               →  (orphan) linked from overview/cite.rst
│
├── _build
│   └── ... 3 items
│
├── changes
│   └── ... 31 items
│
├── cited.rst                      →  move to `doc/documentation/`
├── conf.py
├── conftest.py
├── connectivity.rst               →  move to `doc/api/`
├── contributing.html              →  (redirect) to install/contributing.rst
├── covariance.rst                 →  move to `doc/api/`
├── creating_from_arrays.rst       →  move to `doc/api/`
├── datasets.rst                   →  move to `doc/api/`
├── decoding.rst                   →  move to `doc/api/`
├── documentation.html             →  (redirect) to overview/index.rst
├── events.rst                     →  move to `doc/api/`
├── export.rst                     →  move to `doc/api/`
├── file_io.rst                    →  move to `doc/api/`
├── forward.rst                    →  move to `doc/api/`
├── funding.rst                    →  (orphan) linked from homepage
│
├── generated
│   └── ... 2491 items
│
├── getting_started.html           →  redirect to install/index.rst (1 of 2)
├── glossary.rst                   →  move to `doc/documentation/`
│
├── _includes
│   └── ... 13 items               →  these can be left alone for now, but worth revisiting
│                                     which ones are still needed (or only needed in one place,
│                                     hence no longer necessary to abstract into an `.. include::`
│                                     directive)
│
├── index.rst                      →  (main homepage)
│
├── install
│   ├── advanced.rst               →  ok
│   ├── check_installation.rst     →  ok
│   ├── contributing.rst           →  move to `doc/development/`
│   ├── freesurfer.rst             →  ok
│   ├── index.rst                  →  ok
│   ├── installers.rst             →  ok
│   ├── manual_install_python.rst  →  (orphan, cruft?) linked from `install/contributing`,
│   │                                 `install/manual_install`, and `overview/get_help`. Maybe
│   │                                 cruft because it recommends Anaconda (instead of mambaforge)
│   │
│   ├── manual_install.rst         →  ok
│   ├── mne_c.rst                  →  ok
│   ├── mne_tools_suite.rst        →  ok
│   └── updating.rst               →  ok
│
├── install_mne_python.html        →  (redirect) to install/index.rst (2 of 2)
├── inverse.rst                    →  move to `doc/api/`
├── links.inc
├── logging.rst                    →  move to `doc/api/`
├── Makefile
├── martinos.rst                   →  (orphan) not linked from anywhere (is this still valid/needed?)
├── mne_cpp.rst                    →  (orphan) linked from `doc/install/mne_tools_suite.rst` (still needed?)
├── most_used_classes.rst          →  move to `doc/api/`
├── mri.rst                        →  move to `doc/api/`
│
├── overview
│   ├── cite.rst                   →  move to `doc/documentation/` (maybe as `how-to-cite.rst` to differentiate from `cited.rst`?)
│   ├── cookbook.rst               →  (split?) move to `doc/documentation/`
│   ├── datasets_index.rst         →  move to `doc/documentation/datasets.rst`
│   ├── design_philosophy.rst      →  (cruft? split?) move to `doc/documentation/`
│   ├── development.rst            →  move to `doc/development/index.rst`
│   ├── faq.rst                    →  (needs overhaul) move to `doc/help/`
│   ├── get_help.rst               →  move to `doc/help/index.rst`
│   ├── governance.rst             →  move to `doc/development/`
│   ├── images
│   │   ├── Digitizer-example.png  →  (cruft)
│   │   └── flow_diagram.svg       →  move to `doc/_static/`
│   ├── implementation.rst         →  (split?) move to `doc/documentation/`
│   ├── index.rst                  →  move to `doc/documentation/index.rst`
│   ├── learn_python.rst           →  move to `doc/help/`
│   ├── matlab.rst                 →  (cruft?) move to `doc/documentation/`
│   ├── migrating.rst              →  move to `doc/help/migrating.rst`
│   ├── people.rst                 →  (orphan) linked from `doc/overview/governance.rst`
│   └── roadmap.rst                →  move to `doc/development/`
│
├── preprocessing.rst              →  move to `doc/api/`
├── python_reference.rst           →  move to `doc/api/`
├── reading_raw_data.rst           →  move to `doc/api/`
├── realtime.rst                   →  move to `doc/api/`
├── references.bib
├── report.rst                     →  move to `doc/api/`
├── sensor_space.rst               →  move to `doc/api/`
├── simulation.rst                 →  move to `doc/api/`
├── source_space.rst               →  move to `doc/api/`
│
├── sphinxext
│   └── ... 8 items
│
├── _static
│   └── ... 16 items
│
├── statistics.rst                 →  move to `doc/api/`
│
├── _templates
│   └── ... 5 items
│
├── time_frequency.rst             →  move to `doc/api/`
├── visualization.rst              →  move to `doc/api/`
└── whats_new.rst                  →  move to `doc/development/`
```

</details>

Eager to hear what @mne-tools/mne-python-steering-committee thinks about this.


Discussion:
sounds like a wise idea !

> Message ID: ***@***.***>
>

I feel the same pain and having the organization mirror the docs would be great!
I would also find such a re-organization beneficial!
+1
> every time I need to edit a page in our docs it is a challenge to find the correct `.rst` source file

Interesting 🤔 The way I usually do it is that I look at the URL in the browser, copy the filename and then in VS Code I use Cmd-P and paste the base of the filename, which usually takes me right to the relevant RST file.

That said, I like organization, so +1 from my side on this endeavor 👍 
Sounds good to me! +1
+1 from me too!

Resolving patch:
diff --git a/doc/conf.py b/doc/conf.py
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -1591,6 +1591,32 @@ def reset_warnings(gallery_conf, fname):
     "xdawn_denoising.py",
     "xhemi.py",
 }
+api_redirects = {
+    "connectivity",
+    "covariance",
+    "creating_from_arrays",
+    "datasets",
+    "decoding",
+    "events",
+    "export",
+    "file_io",
+    "forward",
+    "inverse",
+    "logging",
+    "most_used_classes",
+    "mri",
+    "preprocessing",
+    "python_reference",
+    "reading_raw_data",
+    "realtime",
+    "report",
+    "sensor_space",
+    "simulation",
+    "source_space",
+    "statistics",
+    "time_frequency",
+    "visualization",
+}
 ex = "auto_examples"
 co = "connectivity"
 mne_conn = "https://mne.tools/mne-connectivity/stable"
@@ -1609,36 +1635,39 @@ def reset_warnings(gallery_conf, fname):
 custom_redirects = {
     # Custom redirects (one HTML path to another, relative to outdir)
     # can be added here as fr->to key->value mappings
+    "install/contributing.html": "development/contributing.html",
+    "overview/roadmap.html": "development/roadmap.html",
+    "whats_new.html": "development/whats_new.html",
     f"{tu}/evoked/plot_eeg_erp.html": f"{tu}/evoked/30_eeg_erp.html",
     f"{tu}/evoked/plot_whitened.html": f"{tu}/evoked/40_whitened.html",
-    f"{tu}/misc/plot_modifying_data_inplace.html": f"{tu}/intro/15_inplace.html",  # noqa E501
+    f"{tu}/misc/plot_modifying_data_inplace.html": f"{tu}/intro/15_inplace.html",
     f"{tu}/misc/plot_report.html": f"{tu}/intro/70_report.html",
     f"{tu}/misc/plot_seeg.html": f"{tu}/clinical/20_seeg.html",
     f"{tu}/misc/plot_ecog.html": f"{tu}/clinical/30_ecog.html",
     f"{tu}/{ml}/plot_receptive_field.html": f"{tu}/{ml}/30_strf.html",
     f"{tu}/{ml}/plot_sensors_decoding.html": f"{tu}/{ml}/50_decoding.html",
     f"{tu}/{sm}/plot_background_freesurfer.html": f"{tu}/{fw}/10_background_freesurfer.html",  # noqa E501
-    f"{tu}/{sm}/plot_source_alignment.html": f"{tu}/{fw}/20_source_alignment.html",  # noqa E501
+    f"{tu}/{sm}/plot_source_alignment.html": f"{tu}/{fw}/20_source_alignment.html",
     f"{tu}/{sm}/plot_forward.html": f"{tu}/{fw}/30_forward.html",
     f"{tu}/{sm}/plot_eeg_no_mri.html": f"{tu}/{fw}/35_eeg_no_mri.html",
     f"{tu}/{sm}/plot_background_freesurfer_mne.html": f"{tu}/{fw}/50_background_freesurfer_mne.html",  # noqa E501
-    f"{tu}/{sm}/plot_fix_bem_in_blender.html": f"{tu}/{fw}/80_fix_bem_in_blender.html",  # noqa E501
-    f"{tu}/{sm}/plot_compute_covariance.html": f"{tu}/{fw}/90_compute_covariance.html",  # noqa E501
-    f"{tu}/{sm}/plot_object_source_estimate.html": f"{tu}/{nv}/10_stc_class.html",  # noqa E501
+    f"{tu}/{sm}/plot_fix_bem_in_blender.html": f"{tu}/{fw}/80_fix_bem_in_blender.html",
+    f"{tu}/{sm}/plot_compute_covariance.html": f"{tu}/{fw}/90_compute_covariance.html",
+    f"{tu}/{sm}/plot_object_source_estimate.html": f"{tu}/{nv}/10_stc_class.html",
     f"{tu}/{sm}/plot_dipole_fit.html": f"{tu}/{nv}/20_dipole_fit.html",
     f"{tu}/{sm}/plot_mne_dspm_source_localization.html": f"{tu}/{nv}/30_mne_dspm_loreta.html",  # noqa E501
     f"{tu}/{sm}/plot_dipole_orientations.html": f"{tu}/{nv}/35_dipole_orientations.html",  # noqa E501
     f"{tu}/{sm}/plot_mne_solutions.html": f"{tu}/{nv}/40_mne_fixed_free.html",
-    f"{tu}/{sm}/plot_beamformer_lcmv.html": f"{tu}/{nv}/50_beamformer_lcmv.html",  # noqa E501
+    f"{tu}/{sm}/plot_beamformer_lcmv.html": f"{tu}/{nv}/50_beamformer_lcmv.html",
     f"{tu}/{sm}/plot_visualize_stc.html": f"{tu}/{nv}/60_visualize_stc.html",
     f"{tu}/{sm}/plot_eeg_mri_coords.html": f"{tu}/{nv}/70_eeg_mri_coords.html",
     f"{tu}/{sd}/plot_brainstorm_phantom_elekta.html": f"{tu}/{nv}/80_brainstorm_phantom_elekta.html",  # noqa E501
     f"{tu}/{sd}/plot_brainstorm_phantom_ctf.html": f"{tu}/{nv}/85_brainstorm_phantom_ctf.html",  # noqa E501
     f"{tu}/{sd}/plot_phantom_4DBTi.html": f"{tu}/{nv}/90_phantom_4DBTi.html",
-    f"{tu}/{sd}/plot_brainstorm_auditory.html": f"{tu}/io/60_ctf_bst_auditory.html",  # noqa E501
+    f"{tu}/{sd}/plot_brainstorm_auditory.html": f"{tu}/io/60_ctf_bst_auditory.html",
     f"{tu}/{sd}/plot_sleep.html": f"{tu}/clinical/60_sleep.html",
     f"{tu}/{di}/plot_background_filtering.html": f"{tu}/preprocessing/25_background_filtering.html",  # noqa E501
-    f"{tu}/{di}/plot_background_statistics.html": f"{tu}/{sn}/10_background_stats.html",  # noqa E501
+    f"{tu}/{di}/plot_background_statistics.html": f"{tu}/{sn}/10_background_stats.html",
     f"{tu}/{sn}/plot_stats_cluster_erp.html": f"{tu}/{sn}/20_erp_stats.html",
     f"{tu}/{sn}/plot_stats_cluster_1samp_test_time_frequency.html": f"{tu}/{sn}/40_cluster_1samp_time_freq.html",  # noqa E501
     f"{tu}/{sn}/plot_stats_cluster_time_frequency.html": f"{tu}/{sn}/50_cluster_between_time_freq.html",  # noqa E501
@@ -1649,10 +1678,10 @@ def reset_warnings(gallery_conf, fname):
     f"{tu}/{sr}/plot_stats_cluster_time_frequency_repeated_measures_anova.html": f"{tu}/{sn}/70_cluster_rmANOVA_time_freq.html",  # noqa E501
     f"{tu}/{tf}/plot_sensors_time_frequency.html": f"{tu}/{tf}/20_sensors_time_frequency.html",  # noqa E501
     f"{tu}/{tf}/plot_ssvep.html": f"{tu}/{tf}/50_ssvep.html",
-    f"{tu}/{si}/plot_creating_data_structures.html": f"{tu}/{si}/10_array_objs.html",  # noqa E501
+    f"{tu}/{si}/plot_creating_data_structures.html": f"{tu}/{si}/10_array_objs.html",
     f"{tu}/{si}/plot_point_spread.html": f"{tu}/{si}/70_point_spread.html",
     f"{tu}/{si}/plot_dics.html": f"{tu}/{si}/80_dics.html",
-    f"{tu}/{tf}/plot_eyetracking.html": f"{tu}/preprocessing/90_eyetracking_data.html",  # noqa E501
+    f"{tu}/{tf}/plot_eyetracking.html": f"{tu}/preprocessing/90_eyetracking_data.html",
     f"{ex}/{co}/mne_inverse_label_connectivity.html": f"{mne_conn}/{ex}/mne_inverse_label_connectivity.html",  # noqa E501
     f"{ex}/{co}/cwt_sensor_connectivity.html": f"{mne_conn}/{ex}/cwt_sensor_connectivity.html",  # noqa E501
     f"{ex}/{co}/mixed_source_space_connectivity.html": f"{mne_conn}/{ex}/mixed_source_space_connectivity.html",  # noqa E501
@@ -1661,11 +1690,21 @@ def reset_warnings(gallery_conf, fname):
     f"{ex}/{co}/mne_inverse_envelope_correlation_volume.html": f"{mne_conn}/{ex}/mne_inverse_envelope_correlation_volume.html",  # noqa E501
     f"{ex}/{co}/mne_inverse_envelope_correlation.html": f"{mne_conn}/{ex}/mne_inverse_envelope_correlation.html",  # noqa E501
     f"{ex}/{co}/mne_inverse_psi_visual.html": f"{mne_conn}/{ex}/mne_inverse_psi_visual.html",  # noqa E501
-    f"{ex}/{co}/sensor_connectivity.html": f"{mne_conn}/{ex}/sensor_connectivity.html",  # noqa E501
-    f"{ex}/{vi}/publication_figure.html": f"{tu}/{vi}/10_publication_figure.html",  # noqa E501
+    f"{ex}/{co}/sensor_connectivity.html": f"{mne_conn}/{ex}/sensor_connectivity.html",
+    f"{ex}/{vi}/publication_figure.html": f"{tu}/{vi}/10_publication_figure.html",
 }
 
 
+def check_existing_redirect(path):
+    """Make sure existing HTML files are redirects, before overwriting."""
+    if os.path.isfile(path):
+        with open(path, "r") as fid:
+            for _ in range(8):
+                next(fid)
+            line = fid.readline()
+            assert "Page Redirection" in line, line
+
+
 def make_redirects(app, exception):
     """Make HTML redirects."""
     # https://www.sphinx-doc.org/en/master/extdev/appapi.html
@@ -1675,7 +1714,6 @@ def make_redirects(app, exception):
         and exception is None
     ):
         return
-    logger = sphinx.util.logging.getLogger("mne")
     TEMPLATE = """\
 <!DOCTYPE HTML>
 <html lang="en-US">
@@ -1715,25 +1753,30 @@ def make_redirects(app, exception):
         sphinx_logger.info(
             f"Added {len(fnames):3d} HTML plot_* redirects for {out_dir}"
         )
+    # API redirects
+    for page in api_redirects:
+        fname = f"{page}.html"
+        fr_path = os.path.join(app.outdir, fname)
+        to_path = os.path.join(app.outdir, "api", fname)
+        # allow overwrite if existing file is just a redirect
+        check_existing_redirect(fr_path)
+        with open(fr_path, "w") as fid:
+            fid.write(TEMPLATE.format(to=to_path))
+    sphinx_logger.info(f"Added {len(api_redirects):3d} HTML API redirects")
     # custom redirects
     for fr, to in custom_redirects.items():
         if not to.startswith("http"):
             assert os.path.isfile(os.path.join(app.outdir, to)), to
             # handle links to sibling folders
             path_parts = to.split("/")
-            assert tu in path_parts, path_parts  # need to refactor otherwise
-            path_parts = [".."] + path_parts[(path_parts.index(tu) + 1) :]
+            if tu in path_parts:
+                path_parts = [".."] + path_parts[(path_parts.index(tu) + 1) :]
             to = os.path.join(*path_parts)
         assert to.endswith("html"), to
         fr_path = os.path.join(app.outdir, fr)
         assert fr_path.endswith("html"), fr_path
         # allow overwrite if existing file is just a redirect
-        if os.path.isfile(fr_path):
-            with open(fr_path, "r") as fid:
-                for _ in range(8):
-                    next(fid)
-                line = fid.readline()
-                assert "Page Redirection" in line, line
+        check_existing_redirect(fr_path)
         # handle folders that no longer exist
         if fr_path.split("/")[-2] in (
             "misc",
@@ -1755,7 +1798,6 @@ def make_version(app, exception):
         and exception is None
     ):
         return
-    logger = sphinx.util.logging.getLogger("mne")
     try:
         stdout, _ = run_subprocess(["git", "rev-parse", "HEAD"], verbose=False)
     except Exception as exc:
diff --git a/tutorials/io/70_reading_eyetracking_data.py b/tutorials/io/70_reading_eyetracking_data.py
--- a/tutorials/io/70_reading_eyetracking_data.py
+++ b/tutorials/io/70_reading_eyetracking_data.py
@@ -23,8 +23,7 @@
 
 .. seealso:: Some MNE functions may not be available to eyetracking and other
              physiological data, because MNE does not consider them to be data
-             channels. See the :doc:`glossary </glossary>` for more
-             information.
+             channels. See the :ref:`glossary` for more information.
 
 .. _import-eyelink_asc:
 


Test patch: