Repository: pymc-devs/pymc

Issue #5282: Review and update API docs
Created: 2021-12-22T14:14:21Z
We need to make sure all our public api is documented and that they are documented at the recommended import path. #5266 updated the distributions subsection (but I did not check there were no distributions missing) and can be used as template.
Note: if there are functions/classes that are not (or should not be) part of the public api but you think it would be helpful (even if only to you) to have a rendered version of their docstring on the website list them here too. I am considering adding a private api subsection within the contributing section

Duplicate issue #6423: DOC: Unable to successfully build docs: `AttributeError: module 'pymc' has no attribute ...`
Created: 2023-01-01T22:34:54Z
Issue with current documentation:
I'm trying to build the documentation.
What I do

Clone PyMC
Check out the commit tagged with v5.0.1
Install deps:
 $ python -m pip install --requirement requirements-dev.txt
 $ python -m pip install numpyro
 $ python -m pip install --editable .

Run the make doc commands:
$ make clean
$ make html


This is all ran via GH actions, and the past setup has worked perfectly for previous versions of PyMC,
it's the upgrade to 5.0.0+ that has given me trouble.
What I get
Problem seems to happen when dimensionality.ipynb is executed.
I get a lot of ModuleNotFoundError errors like this for a lot of different modules:
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/api/vi.rst:23: WARNING: autosummary: failed to import NormalizingFlow.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.NormalizingFlow'
* AttributeError: module 'pymc' has no attribute 'NormalizingFlow'
* ModuleNotFoundError: No module named 'NormalizingFlow'
* KeyError: 'NormalizingFlow'
* ValueError: not enough values to unpack (expected 2, got 1)
Full logs are below—but it seem like these attributes are missing, some examples:

pymc.align_minibatches
pymc.SymbolicRandomVariable
pymc.SimulatorRV
pymc.KullbackLiebler
pymc.AR1
pymc.ordered_univariate


Full run logs
Run nox --tags build
nox > Running session clone
nox > gh repo clone pymc-devs/pymc
Cloning into 'pymc'...
nox > cd pymc
nox > gh api '--header=Accept: application/vnd.github+json' /repos/pymc-devs/pymc/releases/latest --jq=.tag_name
nox > git checkout tags/v5.0.1 -b v5.0.1
Switched to a new branch 'v5.0.1'
nox > Session clone was successful.
nox > Running session docs
nox > Creating virtual environment (virtualenv) using python3.10 in .nox/docs
nox > cd pymc
nox > python -m pip install --requirement requirements-dev.txt
nox > python -m pip install numpyro
nox > python -m pip install --editable .
nox > make clean
rm -rf docs/_build/*
rm -rf docs/source/api/generated
rm -rf docs/source/api/**/generated
rm -rf docs/source/api/**/classmethods
rm -rf docs/source/contributing/private_api/generated
rm -rf docs/source/contributing/private_api/**/generated
rm -rf docs/source/contributing/private_api/**/classmethods
rm -rf docs/jupyter_execute
nox > make html
sphinx-build docs/source docs/_build -b html
Running Sphinx v5.3.0
WARNING: Invalid configuration value found: 'language = None'. Update your configuration to a valid language code. Falling back to 'en' (English).
making output directory... done
[autosummary] generating autosummary for: 404.md, PyMC_and_PyTensor.rst, api.rst, api/backends.rst, api/data.rst, api/distributions.rst, api/distributions/censored.rst, api/distributions/continuous.rst, api/distributions/discrete.rst, api/distributions/mixture.rst, ..., learn/consulting.md, learn/core_notebooks/GLM_linear.ipynb, learn/core_notebooks/dimensionality.ipynb, learn/core_notebooks/index.md, learn/core_notebooks/model_comparison.ipynb, learn/core_notebooks/posterior_predictive.ipynb, learn/core_notebooks/pymc_overview.ipynb, learn/core_notebooks/pymc_pytensor.ipynb, learn/usage_overview.rst, learn/videos_and_podcasts.md
WARNING: [autosummary] failed to import pymc.AR1.
Possible hints:
* AttributeError: module 'pymc' has no attribute 'AR1'
* ImportError: 
* ModuleNotFoundError: No module named 'pymc.AR1'
WARNING: [autosummary] failed to import pymc.KullbackLiebler.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.KullbackLiebler'
* AttributeError: module 'pymc' has no attribute 'KullbackLiebler'
* ImportError: 
WARNING: [autosummary] failed to import pymc.NFVI.
Possible hints:
* AttributeError: module 'pymc' has no attribute 'NFVI'
* ModuleNotFoundError: No module named 'pymc.NFVI'
* ImportError: 
WARNING: [autosummary] failed to import pymc.NormalizingFlow.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.NormalizingFlow'
* AttributeError: module 'pymc' has no attribute 'NormalizingFlow'
* ImportError: 
WARNING: [autosummary] failed to import pymc.SimulatorRV.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.SimulatorRV'
* AttributeError: module 'pymc' has no attribute 'SimulatorRV'
* ImportError: 
WARNING: [autosummary] failed to import pymc.align_minibatches.
Possible hints:
* AttributeError: module 'pymc' has no attribute 'align_minibatches'
* ModuleNotFoundError: No module named 'pymc.align_minibatches'
* ImportError: 
WARNING: [autosummary] failed to import pymc.at_rng.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.at_rng'
* ImportError: 
* AttributeError: module 'pymc' has no attribute 'at_rng'
WARNING: [autosummary] failed to import pymc.distributions.transforms.ordered_multivariate.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.distributions.transforms.ordered_multivariate'; 'pymc.distributions.transforms' is not a package
* ImportError: 
* AttributeError: module 'pymc.distributions.transforms' has no attribute 'ordered_multivariate'
WARNING: [autosummary] failed to import pymc.distributions.transforms.ordered_univariate.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.distributions.transforms.ordered_univariate'; 'pymc.distributions.transforms' is not a package
* ImportError: 
* AttributeError: module 'pymc.distributions.transforms' has no attribute 'ordered_univariate'
WARNING: [autosummary] failed to import pymc.math.abs_.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.math.abs_'; 'pymc.math' is not a package
* AttributeError: module 'pymc.math' has no attribute 'abs_'
* ImportError: 
WARNING: [autosummary] failed to import pymc.pandas_to_array.
Possible hints:
* ImportError: 
* ModuleNotFoundError: No module named 'pymc.pandas_to_array'
* AttributeError: module 'pymc' has no attribute 'pandas_to_array'
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/pymc/sampling/jax.py:39: UserWarning: This module is experimental.
  warnings.warn("This module is experimental.")
WARNING: [autosummary] failed to import pymc.set_at_rng.
* ModuleNotFoundError: No module named 'set_at_rng'
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/api/pytensorf.rst:6: WARNING: autosummary: failed to import at_rng.
Possible hints:
* ModuleNotFoundError: No module named 'at_rng'
* AttributeError: module 'pymc' has no attribute 'at_rng'
* ValueError: not enough values to unpack (expected 2, got 1)
* KeyError: 'at_rng'
* ModuleNotFoundError: No module named 'pymc.at_rng'
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/api/pytensorf.rst:6: WARNING: autosummary: failed to import pandas_to_array.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.pandas_to_array'
* ValueError: not enough values to unpack (expected 2, got 1)
* ModuleNotFoundError: No module named 'pandas_to_array'
* KeyError: 'pandas_to_array'
* AttributeError: module 'pymc' has no attribute 'pandas_to_array'
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/api/vi.rst:7: WARNING: autosummary: failed to import NFVI.
Possible hints:
* AttributeError: module 'pymc' has no attribute 'NFVI'
* ValueError: not enough values to unpack (expected 2, got 1)
* ModuleNotFoundError: No module named 'NFVI'
* ModuleNotFoundError: No module named 'pymc.NFVI'
* KeyError: 'NFVI'
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/api/vi.rst:23: WARNING: autosummary: failed to import NormalizingFlow.
Possible hints:
* ModuleNotFoundError: No module named 'pymc.NormalizingFlow'
* AttributeError: module 'pymc' has no attribute 'NormalizingFlow'
* ModuleNotFoundError: No module named 'NormalizingFlow'
* KeyError: 'NormalizingFlow'
* ValueError: not enough values to unpack (expected 2, got 1)
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/glossary.md:18: WARNING: glossary term must be preceded by empty line
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/glossary.md:2: WARNING: duplicate label glossary, other instance in /home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/glossary.md
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/guides/Gaussian_Processes.rst:8: WARNING: duplicate label gaussian processes, other instance in /home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/api/gp.rst
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/installation.md:2: WARNING: duplicate label installation, other instance in /home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/installation.md
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/learn/books.md:2: WARNING: duplicate label books, other instance in /home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/learn/books.md
/home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/learn/consulting.md:2: WARNING: duplicate label consulting, other instance in /home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/learn/consulting.md

Exception occurred:
  File "/home/runner/work/pymc-dash-docset/pymc-dash-docset/.nox/docs/lib/python3.10/site-packages/myst_nb/core/execute/direct.py", line 50, in start_client
    raise ExecutionError(str(self.path)) from result.err
myst_nb.core.execute.base.ExecutionError: /home/runner/work/pymc-dash-docset/pymc-dash-docset/pymc/docs/source/learn/core_notebooks/dimensionality.ipynb
The full traceback has been saved in /tmp/sphinx-err-48k_zbw6.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
nox > Command make html failed with exit code 2
nox > Session docs failed.
nox > Ran multiple sessions:
nox > * clone: success
nox > * docs: failed
make: *** [Makefile:64: html] Error 2
Error: Process completed with exit code 1.

Context
I maintain the Dash docset for PyMC, and try to keep up with the new releases!
Idea or request for content:
No response

Closing comment (2023-01-10T13:11:31Z): None of the ModuleNotFound errors are the reason for the failed doc build. They are logger errors, not actual exceptions, they also happen in our readthedocs build (and have been for quite a while, these are tracked in #5282). From the full logs, it looks like the error is that the dimensionality notebook can't be executed.
We re-execute the notebooks in https://www.pymc.io/projects/docs/en/stable/learn/core_notebooks/index.html with every single commit to main, but we recently modified the sphinx config so that execution errors in notebooks fail the sphinx build, before that there would be errors laying around in the rendered docs which wasn't ideal. See dim notebook for 4.4 release for example (also see how in 5 and 5.0.1 the whole notebook runs without errors though).
My guess therefore is that there is an installation issue when using pip on gh-actions (we use conda on both gh actions and readthedocs) which prevents the notebook from running and it now fails the build.
Potential fixes that come to mind would be

using conda for the pymc install (you can take a look at https://github.com/pymc-devs/pymc/blob/main/.github/workflows/tests.yml for the config)
or override the notebook execution error -> sphinx build failure config variable. You could change the nb_execution_allow_errors but it might be a better idea if you do that to directly skip executing notebooks to build faster and to avoid the errors making their way into the built docs. I think you should use the D flag like this: sphinx-build ... -D nb_execution_mode=off. Ref: https://www.sphinx-doc.org/en/master/man/sphinx-build.html#cmdoption-sphinx-build-D