Repository: HypothesisWorks/hypothesis

Issue:
Can we consider optionally disabling automatic repr-printing and jsonification when `TESTCASE_CALLBACKS` are present?
Thanks for Hypothesis, it's awesome!

I have a REPL set up for continuously testing the program I'm working on using some hypothesis tests. The inputs are fairly large dataclasses that are unfortunately self-referential.

This works well in general, but when I attempt to tap result generation via a callable in TESTCASE_CALLBACKS, things slow down or crash due to blowing the stack. I have my own pretty-printers that I'd like to call from the repl experience, so I don't have any use for the jsonable or pprinted inputs. I checked over the code to see if I could find a flag to override the behavior (or return my own value for either) but I couldn't find any reasonable solution.

Would you be willing to consider making that behavior optionable via a flag or bump the `to_json` check earlier in the `to_jsonable` resolution chain?

I'd be happy to patch it to save you the time if a solution jumps out at you as a good one.

Thanks!
Josh

[to_jsonable](https://github.com/HypothesisWorks/hypothesis/blob/56319cb80f2b1df9d0776aa73a0696ff6db19df7/hypothesis-python/src/hypothesis/strategies/_internal/utils.py#L155)

[only attempts user-configurable to_json if all else fails](https://github.com/HypothesisWorks/hypothesis/blob/56319cb80f2b1df9d0776aa73a0696ff6db19df7/hypothesis-python/src/hypothesis/strategies/_internal/utils.py#L193)

[impossible to prevent jsonification when TESTCASE_CALLBACKS are present](https://github.com/HypothesisWorks/hypothesis/blob/56319cb80f2b1df9d0776aa73a0696ff6db19df7/hypothesis-python/src/hypothesis/core.py#L914)


Discussion:
I'd prefer not to add another config option, but would be happy to shift the `.to_json()` check to before the attrs/dataclasses/pydantic block in `to_jsonable()`.  PR welcome for that!

You can also improve the interaction with pretty-printing downstream; we support [the IPython protocol](https://ipython.readthedocs.io/en/stable/api/generated/IPython.lib.pretty.html) and/or you can [shove a callback directly into the registry](https://github.com/HypothesisWorks/hypothesis/blob/56319cb80f2b1df9d0776aa73a0696ff6db19df7/hypothesis-python/src/hypothesis/vendor/pretty.py#L735-L769) if you'd prefer.
> I'd prefer not to add another config option

Understandable!

> happy to shift the .to_json() check to before the attrs/dataclasses/pydantic block in to_jsonable(). PR welcome for that!

Sure thing, I'll get one cooking today!

> You can also improve the interaction with pretty-printing downstream; we support [the IPython protocol](https://ipython.readthedocs.io/en/stable/api/generated/IPython.lib.pretty.html) and/or you can [shove a callback directly into the registry](https://github.com/HypothesisWorks/hypothesis/blob/56319cb80f2b1df9d0776aa73a0696ff6db19df7/hypothesis-python/src/hypothesis/vendor/pretty.py#L735-L769) if you'd prefer.

That's a solid idea, thanks!

Resolving patch:
diff --git a/hypothesis-python/src/hypothesis/strategies/_internal/utils.py b/hypothesis-python/src/hypothesis/strategies/_internal/utils.py
--- a/hypothesis-python/src/hypothesis/strategies/_internal/utils.py
+++ b/hypothesis-python/src/hypothesis/strategies/_internal/utils.py
@@ -176,6 +176,14 @@ def to_jsonable(obj: object) -> object:
             for k, v in obj.items()
         }
 
+    # Hey, might as well try calling a .to_json() method - it works for Pandas!
+    # We try this before the below general-purpose handlers to give folks a
+    # chance to control this behavior on their custom classes.
+    try:
+        return to_jsonable(obj.to_json())  # type: ignore
+    except Exception:
+        pass
+
     # Special handling for dataclasses, attrs, and pydantic classes
     if (
         (dcs := sys.modules.get("dataclasses"))
@@ -188,11 +196,5 @@ def to_jsonable(obj: object) -> object:
     if (pyd := sys.modules.get("pydantic")) and isinstance(obj, pyd.BaseModel):
         return to_jsonable(obj.model_dump())
 
-    # Hey, might as well try calling a .to_json() method - it works for Pandas!
-    try:
-        return to_jsonable(obj.to_json())  # type: ignore
-    except Exception:
-        pass
-
     # If all else fails, we'll just pretty-print as a string.
     return pretty(obj)


Test patch:
diff --git a/hypothesis-python/tests/cover/test_searchstrategy.py b/hypothesis-python/tests/cover/test_searchstrategy.py
--- a/hypothesis-python/tests/cover/test_searchstrategy.py
+++ b/hypothesis-python/tests/cover/test_searchstrategy.py
@@ -160,3 +160,16 @@ def test_jsonable_very_large_ints():
     # previously caused OverflowError when casting to float.
     n = 2**1024
     assert to_jsonable(n) == sys.float_info.max
+
+
+@dataclasses.dataclass()
+class HasCustomJsonFormat:
+    x: str
+
+    def to_json(self):
+        return "surprise!"
+
+
+def test_jsonable_override():
+    obj = HasCustomJsonFormat("expected")
+    assert to_jsonable(obj) == "surprise!"