Repository: mitmproxy/pdoc
Pull request #784: add descriptive error message if re.split fails in numpy
Author: login: tobiscode
Created: 2025-03-14T08:58:10Z | Merged: 2025-03-18T11:02:38Z

Hi,
I ran into the same issue as reported in https://github.com/mitmproxy/pdoc/issues/758, and have created a fork to implement the solution proposed by @dhermangh. In its current form, it prints the affected content and the header it's under, but since the individual docstring conversion function does not know which function/class etc. it is operating on, it can't give that info (or a line number in the original file, for that matter). If there is a way to know the latter, I'm happy to add it as well, but for now, it at least suggest to the user that a type descriptor might be missing.
Does this look good to everyone? Happy to include more fixes.
Cheers,
Tobias


Diff:
diff --git a/pdoc/docstrings.py b/pdoc/docstrings.py
index b9588287..5c9dc682 100644
--- a/pdoc/docstrings.py
+++ b/pdoc/docstrings.py
@@ -24,6 +24,13 @@
 from textwrap import indent
 import warnings
 
+AnyException = (SystemExit, GeneratorExit, Exception)
+"""BaseException, but excluding KeyboardInterrupt.
+
+Modules may raise SystemExit on import (which we want to catch),
+but we don't want to catch a user's KeyboardInterrupt.
+"""
+
 
 @cache
 def convert(docstring: str, docformat: str, source_file: Path | None) -> str:
@@ -32,17 +39,25 @@ def convert(docstring: str, docformat: str, source_file: Path | None) -> str:
     """
     docformat = docformat.lower()
 
-    if any(x in docformat for x in ["google", "numpy", "restructuredtext"]):
-        docstring = rst(docstring, source_file)
+    try:
+        if any(x in docformat for x in ["google", "numpy", "restructuredtext"]):
+            docstring = rst(docstring, source_file)
+
+        if "google" in docformat:
+            docstring = google(docstring)
 
-    if "google" in docformat:
-        docstring = google(docstring)
+        if "numpy" in docformat:
+            docstring = numpy(docstring)
 
-    if "numpy" in docformat:
-        docstring = numpy(docstring)
+        if source_file is not None and os.environ.get("PDOC_EMBED_IMAGES") != "0":
+            docstring = embed_images(docstring, source_file)
 
-    if source_file is not None and os.environ.get("PDOC_EMBED_IMAGES") != "0":
-        docstring = embed_images(docstring, source_file)
+    except AnyException as e:
+        raise RuntimeError(
+            'Docstring processing failed for docstring=\n"""\n'
+            + docstring
+            + f'\n"""\n{source_file=}\n{docformat=}'
+        ) from e
 
     return docstring
 
diff --git a/pdoc/extract.py b/pdoc/extract.py
index fa9f2524..255f6ecc 100644
--- a/pdoc/extract.py
+++ b/pdoc/extract.py
@@ -61,7 +61,7 @@ def walk_specs(specs: Sequence[Path | str]) -> list[str]:
                 modspec = importlib.util.find_spec(modname)
                 if modspec is None:
                     raise ModuleNotFoundError(modname)
-        except AnyException:
+        except pdoc.docstrings.AnyException:
             warnings.warn(
                 f"Cannot find spec for {modname} (from {spec}):\n{traceback.format_exc()}",
                 stacklevel=2,
@@ -110,7 +110,7 @@ def parse_spec(spec: Path | str) -> str:
                 modspec = importlib.util.find_spec(spec)
                 if modspec is None:
                     raise ModuleNotFoundError
-        except AnyException:
+        except pdoc.docstrings.AnyException:
             # Module does not exist, use local file.
             spec = pspec
         else:
@@ -218,18 +218,10 @@ def load_module(module: str) -> types.ModuleType:
     Returns the imported module."""
     try:
         return importlib.import_module(module)
-    except AnyException as e:
+    except pdoc.docstrings.AnyException as e:
         raise RuntimeError(f"Error importing {module}") from e
 
 
-AnyException = (SystemExit, GeneratorExit, Exception)
-"""BaseException, but excluding KeyboardInterrupt.
-
-Modules may raise SystemExit on import (which we want to catch),
-but we don't want to catch a user's KeyboardInterrupt.
-"""
-
-
 def iter_modules2(module: types.ModuleType) -> dict[str, pkgutil.ModuleInfo]:
     """
     Returns all direct child modules of a given module.
@@ -315,7 +307,7 @@ def module_mtime(modulename: str) -> float | None:
     try:
         with mock_some_common_side_effects():
             spec = importlib.util.find_spec(modulename)
-    except AnyException:
+    except pdoc.docstrings.AnyException:
         pass
     else:
         if spec is not None and spec.origin is not None:
@@ -365,7 +357,7 @@ def invalidate_caches(module_name: str) -> None:
                 continue  # some funky stuff going on - one example is typing.io, which is a class.
             with mock_some_common_side_effects():
                 importlib.reload(sys.modules[modname])
-        except AnyException:
+        except pdoc.docstrings.AnyException:
             warnings.warn(
                 f"Error reloading {modname}:\n{traceback.format_exc()}",
                 stacklevel=2,
diff --git a/test/test_docstrings.py b/test/test_docstrings.py
index 8a172538..658d4d0d 100644
--- a/test/test_docstrings.py
+++ b/test/test_docstrings.py
@@ -37,6 +37,19 @@ def test_rst_extract_options_fuzz(s):
     assert not s or content or options
 
 
+def test_convert_exception(monkeypatch):
+    def raise_(*_):
+        raise Exception
+
+    monkeypatch.setattr(docstrings, "rst", raise_)
+    with pytest.raises(RuntimeError, match="Docstring processing failed"):
+        docstrings.convert(
+            "Valid minimal docstring without in- or output.",
+            "numpy",
+            None,
+        )
+
+
 def test_rst_extract_options():
     content = (
         ":alpha: beta\n"
diff --git a/test/test_smoke.py b/test/test_smoke.py
index f8c67c49..d2b215d7 100644
--- a/test/test_smoke.py
+++ b/test/test_smoke.py
@@ -22,7 +22,7 @@ def test_smoke(module):
     try:
         with pdoc.extract.mock_some_common_side_effects():
             importlib.import_module(module)
-    except pdoc.extract.AnyException:
+    except pdoc.docstrings.AnyException:
         pass
     else:
         try:


Reviews:
user: login: mhils
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T09:09:37Z
user: login: tobiscode
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T09:24:14Z
user: login: tobiscode
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T09:41:53Z
user: login: mhils
body: Thanks! 🍰 😃 
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-18T11:02:32Z

Review comments:
diff_hunk: @@ -37,6 +37,15 @@ def test_rst_extract_options_fuzz(s):
     assert not s or content or options
 
 
+def test_convert_exception():
path: test/test_docstrings.py
user: login: mhils
body: This is not a useful test because it tests unwanted behavior (we don't want any known cases of raising). Let's use pytest's monkeypatch to replace `rst` with a method that raises an exception, and then test with that as you currently do right now (`with pytest.raises()` is great).
created_at: 2025-03-18T09:09:26Z
author_association: MEMBER
diff_hunk: @@ -37,6 +37,15 @@ def test_rst_extract_options_fuzz(s):
     assert not s or content or options
 
 
+def test_convert_exception():
path: test/test_docstrings.py
user: login: tobiscode
body: Ahhh I see, that makes more sense.
created_at: 2025-03-18T09:24:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -37,6 +37,15 @@ def test_rst_extract_options_fuzz(s):
     assert not s or content or options
 
 
+def test_convert_exception():
path: test/test_docstrings.py
user: login: tobiscode
body: Does the new commit look good now?
created_at: 2025-03-18T09:41:53Z
author_association: CONTRIBUTOR

PR comments:
user: login: mhils
created_at: 2025-03-14T09:07:45Z
body: Thanks! I'd suggest we generically catch all exceptions in `convert`, and then do something like this:

```python3
except AnyException:
    raise RuntimeError(f"Docstring processing failed:\n{content=}\n{source_file=}\n{docformat=}") from e
```
(see pdoc.extract for AnyException - we can move it somewhere else)
author_association: MEMBER
user: login: tobiscode
created_at: 2025-03-17T08:33:34Z
body: Hi,

got it - do you want that in addition to the `numpy`-specific catch block, or instead of? The advantage of keeping both would be that the specific block can point exactly to the part of the docstring that is problematic, which a catch block in `convert` by itself could not. It might become too verbose for the `numpy` case, but if I define a `NumPyParseException` or somthing similar, I could also handle that properly.

Cheers
author_association: CONTRIBUTOR
user: login: mhils
created_at: 2025-03-17T08:52:26Z
body: Instead of. The point of that is to help debugging and fixing pdoc. 

I'm happy to merge a second PR that fixes the actual bug in the numpy docstring conversion.
author_association: MEMBER
user: login: tobiscode
created_at: 2025-03-17T09:37:26Z
body: >  that fixes the actual bug in the numpy docstring conversion.

I'm not sure it is a bug - the [NumPy docstring convention](https://numpydoc.readthedocs.io/en/stable/format.html#parameters) requires there to be a type for both in- and output parameters. Sphinx has an [extension](https://github.com/tox-dev/sphinx-autodoc-typehints) that takes advantage of python type hints but I think it's just for convenience rather than a newer docstring convention. How would you have suggested to fix the parsing bug otherwise?
author_association: CONTRIBUTOR
user: login: mhils
created_at: 2025-03-17T14:41:30Z
body: `pdoc.docstrings.convert` is not supposed to raise an exception under any circumstances. It may produce a warning, but it must not error.
author_association: MEMBER
user: login: tobiscode
created_at: 2025-03-17T15:43:00Z
body: > `pdoc.docstrings.convert` is not supposed to raise an exception under any circumstances. It may produce a warning, but it must not error.

Now I'm even more confused, since in your first comment you suggested to catch `AnyException` in `pdoc.docstrings.convert` and then turn it into a `RuntimeError`. Don't get me wrong, I'm happy to implement whatever solution you prefer, it's just unclear to me what that is.

If you meant `pdoc.docstrings.**numpy**` is not supposed to raise an exception, then I could modify the `re.split` operation such that it throws a warning instead. In that case, I would need to know what the code is supposed to do for that section - just ignore it (in which case the Returns section would be empty) or add a dummy variable name (maybe `_` in keeping with some Python standards, this would allow the description to still appear in the generated documentation).


author_association: CONTRIBUTOR
user: login: mhils
created_at: 2025-03-17T16:07:36Z
body: > Now I'm even more confused, since in your first comment you suggested to catch `AnyException` in `pdoc.docstrings.convert` and then turn it into a `RuntimeError`. Don't get me wrong, I'm happy to implement whatever solution you prefer, it's just unclear to me what that is.

It's not meant to ever throw an exception (but if it does, it should come with useful context for debugging). Phrased differently: If the code has a bug, `except AnyException` is supposed to attach the full docstring to the traceback for debuggability and then reraise.

> If you meant pdoc.docstrings.**numpy** is not supposed to raise an exception,

Correct, it's not supposed to. If it does, it's a bug. Generally speaking, if something does not parse as a particular format, we should leave the text as-is. I haven't looked super close into the specific numpy case. In either case, that's a separate PR with a test case for more clarity.

Thanks!
author_association: MEMBER
user: login: tobiscode
created_at: 2025-03-17T16:12:28Z
body: Okay now I think I get it, thanks for the clarification. What I'll do then is:

1. move the try/except block from the `numpy` subfunction to the `convert` function, and update this PR with that commit, and
2. make a minimal test case that for the NumPy-specific case mentioned in https://github.com/mitmproxy/pdoc/issues/758 produces different test outputs. I'll write that to the issue first, and then after deciding how to handle it, I'll make a new PR.


author_association: CONTRIBUTOR
user: login: mhils
created_at: 2025-03-17T16:15:46Z
body: Perfect, thanks!
author_association: MEMBER
user: login: tobiscode
created_at: 2025-03-17T16:44:59Z
body: Okay so it seems the test coverage action fails on these lines:

```python
    except AnyException as e:
        raise RuntimeError(
```

How should I address this? By making a test case in `test_docstrings.py` which will raise that exception?
author_association: CONTRIBUTOR
user: login: mhils
created_at: 2025-03-17T17:12:51Z
body: Yes please. The test needs to mock one of the subfunctions to raise. pytest's monkeypatch is great for that.
author_association: MEMBER
user: login: tobiscode
created_at: 2025-03-18T09:01:19Z
body: Okay, so since I wasn't familiar with monkeypatch I imitated how the `AnyException` catches were tested in `test_extract.py` and did something similar in `test_docstrings.py`. I hope that's okay, otherwise I can try to do it the monkeypatch way. With that, all tests pass, and the branch could be merged.

There is still a test warning appearing in the logs, however, which seems unrelated to my changes:

```
 test/test_doc_types.py::test_eval_fail2
  D:\a\pdoc\pdoc\pdoc\doc_types.py:148: UserWarning: Error parsing type annotation xyz for a. Import of xyz failed: name 'xyz' is not defined
    warnings.warn(
```

It looks to me in `doc_types.py` that this is one of the possible outcomes, but not the one it's testing for. I don't know how to fix this since I don't understand the logic flow in this test, but happy to make some changes if someone explains it to me. (And probably make a new PR for that.)
author_association: CONTRIBUTOR