Repository: falconry/falcon

Issue:
Detect the deserialization error class in JSONHandler
Normally, we just assume it is a subclass of `ValueError`. However, it is not true for `msgspec`...

So maybe we can run a simple test and record the exception type?

Alternatively, we could always catch `Exception`, but that sounds unnecessarily broad... 🤔


Discussion:

Patch:
diff --git a/docs/_newsfragments/2476.newandimproved.rst b/docs/_newsfragments/2476.newandimproved.rst
new file mode 100644
index 000000000..a589ca007
--- /dev/null
+++ b/docs/_newsfragments/2476.newandimproved.rst
@@ -0,0 +1,8 @@
+:class:`~falcon.media.JSONHandler` can now detect a non-standard
+(not a subclass of :class:`ValueError`) deserialization error type for a custom
+`loads` function.
+
+(Normally, :func:`json.loads` and third party alternatives do raise a subclass
+of :class:`ValueError` on invalid input data, however, this is not the case
+for, e.g., the popular :ref:`msgspec <msgspec_recipe>` library
+at the time of writing.)
diff --git a/docs/user/recipes/msgspec-integration.rst b/docs/user/recipes/msgspec-integration.rst
index 67928bdfe..e3f707c94 100644
--- a/docs/user/recipes/msgspec-integration.rst
+++ b/docs/user/recipes/msgspec-integration.rst
@@ -23,12 +23,6 @@ Media handlers
     ``msgspec.json.decode()`` and ``msgspec.json.encode()``, respectively.
     This optimization is left as an exercise for the reader.
 
-.. attention::
-    As seen above, plugging ``msgspec`` into :class:`~falcon.media.JSONHandler`
-    may look extremely straightforward. We have, however, omitted an important
-    caveat when it comes to error handling -- see a more detailed discussion in
-    the below chapter: :ref:`msgspec_error_handling`.
-
 Using ``msgspec`` for handling MessagePack (or YAML) media is slightly more
 involved, as we need to implement the :class:`~falcon.media.BaseHandler`
 interface:
@@ -36,6 +30,11 @@ interface:
 .. literalinclude:: ../../../examples/recipes/msgspec_msgpack_handler.py
     :language: python
 
+.. attention::
+    In contrast to :class:`~falcon.media.JSONHandler`, we would also need to
+    implement error handling for invalid MessagePack payload. See more in the
+    below chapter: :ref:`msgspec_error_handling`.
+
 We can now use these handlers for request and response media
 (see also: :ref:`custom_media_handlers`).
 
@@ -90,17 +89,20 @@ case is represented by ``msgspec.ValidationError``. We could either create an
 exception as :class:`~falcon.MediaValidationError`, or just use a
 ``try.. except`` clause, and reraise directly inside middleware.
 
-Our customized :class:`~falcon.media.JSONHandler` has another issue: unlike the
-stdlib's :mod:`json` or the majority of other JSON libraries,
-``msgspec.DecodeError`` is not a subclass of :class:`ValueError`:
+``msgspec.json.decode()`` has another peculiarity: unlike the stdlib's
+:mod:`json` or the majority of other JSON libraries, ``msgspec.DecodeError`` is
+not a subclass of :class:`ValueError`:
 
 >>> import msgspec
 >>> issubclass(msgspec.DecodeError, ValueError)
 False
 
-This discrepancy can be worked around using a wrapper around
-``msgspec.json.decode()`` that reraises a :class:`ValueError` from
-``msgspec.DecodeError``. Furthermore, the problem has been reported upstream,
+Falcon's :class:`~falcon.media.JSONHandler` works around this discrepancy by
+detecting the actual deserialization exception type at the time of
+initialization, but you may still encounter the issue when using the library
+manually, or when using it for other media types such as MessagePack
+(see above).
+Furthermore, the problem has been reported upstream,
 and received positive feedback from the maintainer, so hopefully it could get
 resolved in the near future.
 
diff --git a/examples/recipes/msgspec_main.py b/examples/recipes/msgspec_main.py
index b676d80fd..af605b4a8 100644
--- a/examples/recipes/msgspec_main.py
+++ b/examples/recipes/msgspec_main.py
@@ -30,7 +30,7 @@ class NoteResource:
 
     def __init__(self) -> None:
         # NOTE: In a real-world app, you would want to use persistent storage.
-        self._store = {}
+        self._store: dict[str, Note] = {}
 
     def on_get_note(self, req: Request, resp: Response, noteid: uuid.UUID) -> None:
         resp.media = self._store.get(str(noteid))
@@ -62,13 +62,6 @@ def process_resource(
             params[param] = msgspec.convert(req.get_media(), schema)
 
 
-def _msgspec_loads(content: str) -> Any:
-    try:
-        return msgspec.json.decode(content)
-    except msgspec.DecodeError as ex:
-        raise falcon.MediaMalformedError(falcon.MEDIA_JSON) from ex
-
-
 def _handle_validation_error(
     req: Request, resp: Response, ex: msgspec.ValidationError, params: dict[str, Any]
 ) -> None:
@@ -81,7 +74,7 @@ def create_app() -> falcon.App:
 
     json_handler = JSONHandler(
         dumps=msgspec.json.encode,
-        loads=_msgspec_loads,
+        loads=msgspec.json.decode,
     )
     app.req_options.media_handlers[falcon.MEDIA_JSON] = json_handler
     app.resp_options.media_handlers[falcon.MEDIA_JSON] = json_handler
diff --git a/falcon/media/json.py b/falcon/media/json.py
index 44f06c72b..92cd6d4dc 100644
--- a/falcon/media/json.py
+++ b/falcon/media/json.py
@@ -2,7 +2,7 @@
 
 from functools import partial
 import json
-from typing import Any, Callable, Optional, Union
+from typing import Any, Callable, Optional, Tuple, Type, Union
 
 from falcon import errors
 from falcon import http_error
@@ -162,6 +162,7 @@ def __init__(
     ) -> None:
         self._dumps = dumps or partial(json.dumps, ensure_ascii=False)
         self._loads = loads or json.loads
+        self._deserialization_errors: Tuple[Type[Exception], ...] = (ValueError,)
 
         # PERF(kgriffs): Test dumps once up front so we can set the
         #     proper serialize implementation.
@@ -173,6 +174,16 @@ def __init__(
             self.serialize = self._serialize_b  # type: ignore[method-assign]
             self.serialize_async = self._serialize_async_b  # type: ignore[method-assign]
 
+        # PERF(vytas): Try to detect nonstandard deserialization exception
+        #   classes (that do not subclass ValueError) up front, and add them to
+        #   the tuple of known deserialization errors.
+        try:
+            self._loads('frozenset({1})')
+        except Exception as ex:
+            error_cls = type(ex)
+            if not issubclass(error_cls, ValueError):
+                self._deserialization_errors += (error_cls,)
+
         # NOTE(kgriffs): To be safe, only enable the optimized protocol when
         #   not subclassed.
         if type(self) is JSONHandler:
@@ -184,8 +195,8 @@ def _deserialize(self, data: bytes) -> Any:
             raise errors.MediaNotFoundError('JSON')
         try:
             return self._loads(data.decode())
-        except ValueError as err:
-            raise errors.MediaMalformedError('JSON') from err
+        except self._deserialization_errors as ex:
+            raise errors.MediaMalformedError('JSON') from ex
 
     def deserialize(
         self,


Test patch:
diff --git a/tests/test_media_handlers.py b/tests/test_media_handlers.py
index e8faff624..fa5b48d82 100644
--- a/tests/test_media_handlers.py
+++ b/tests/test_media_handlers.py
@@ -291,6 +291,42 @@ async def on_post(self, req, resp):
         testing.simulate_post(app, '/', json={})
 
 
+def test_nonstandard_deserialization_error(asgi, util):
+    # NOTE(vytas): At the time of writing, this is also tested by msgspec.
+    #   However, we add an additional generic test here if msgspec restructures
+    #   its class hierarchy to inherit from ValueError, or if it is unavailable
+    #   in the environment in question.
+
+    class SuchException(RuntimeError):
+        def __init__(self):
+            super().__init__('wow such exception')
+
+    def fancy_loads(data):
+        try:
+            return json.loads(data)
+        except ValueError:
+            raise SuchException()
+
+    class DevNull:
+        def on_post(self, req, resp):
+            req.get_media()
+
+        async def on_post_async(self, req, resp):
+            await req.get_media()
+
+    app = util.create_app(asgi)
+    app.add_route('/dev/null', DevNull(), suffix=('async' if asgi else None))
+    app.req_options.media_handlers[falcon.MEDIA_JSON] = media.JSONHandler(
+        loads=fancy_loads
+    )
+
+    resp = testing.simulate_post(
+        app, '/dev/null', body=b'Hello: world', content_type=falcon.MEDIA_JSON
+    )
+    assert resp.status_code == 400
+    assert resp.json.get('title') == 'Invalid JSON'
+
+
 def test_sync_methods_not_overridden(asgi, util):
     app = util.create_app(asgi)
 
diff --git a/tests/test_recipes.py b/tests/test_recipes.py
index e15f012f8..739aa05ca 100644
--- a/tests/test_recipes.py
+++ b/tests/test_recipes.py
@@ -249,6 +249,11 @@ async def on_post_async(self, req, resp):
         suffix = 'async' if asgi else None
         app.add_route('/media', MediaResource(), suffix=suffix)
 
+        resp0 = client.simulate_post(
+            '/media', body=b'Hello: world', content_type=falcon.MEDIA_JSON
+        )
+        assert resp0.status_code == 400
+
         resp1 = client.simulate_post('/media', json=[1, 3, 3, 7])
         assert resp1.status_code == 200
         assert resp1.text == '[1, 3, 3, 7]'