Repository: falconry/falcon

Issue:
Clarify that `TestClient` does not use the app's JSON handlers
As I'm writing pytest tests for my ASGI app, I'm using the TestClient as a fixture as described in the docs:

```python
@pytest.fixture
def falcon_test_client():
    return TestClient(get_app())
```

`get_app()` is a function that creates and configures my app including some custom Json handlers (which as you can see below I also use for `application/x-www-form-urlencoded`):
```python
def get_app():
    app = App()

    json_handler = JSONHandler(
        dumps=partial(json.dumps, cls=CustomJsonEncoder),
        loads=partial(json.loads, cls=CustomJsonEncoder),
    )

    extra_handlers = {
        falcon.MEDIA_JSON: json_handler,
        falcon.MEDIA_URLENCODED: json_handler,
    }

    app.req_options.media_handlers.update(extra_handlers)
    app.resp_options.media_handlers.update(extra_handlers)

    # ... routes ...

    return app
```

`CustomJsonEncoder` is an encoder that can handle `UUID` and `datetime` objects; testing it with `json.dumps()` works so I know the problem is not there.

The problem is that when I try to give some data that includes a `UUID` to `simulate_post()`, it fails with `TypeError: Object of type UUID is not JSON serializable`, and I can see from pytest's traceback that at the moment of the exception, `self` is `json.encoder.JSONEncoder`, namely not the custom encoder I defined.

This is the simple test I'm trying this with:

```python
def test_(falcon_test_client):
    response = falcon_test_client.simulate_post(
        "/my/endpoint",
        json={
            "data": {},  # Just a dict with str keys and a UUID as value
        },
    )
```

I'm either forgetting something or I got something wrong, but I'm not sure what and how.

Also, I was rather confused when I was looking for ways to simulate and receive `x-www-form-urlencoded` data: if I set the `content_type` accordingly in something like `simulate_post()` and put the data in `body` as a dict¹, `req.get_media()` in the route handler seems to receive it as plain text.  
If I put the data in the `json` parameter, as stated in the docs, the `content_type` is forced to `application/json`, making my test subtly different from reality, and since there doesn't seem to be a way to avoid this behaviour, I'd rather use an alternative that keeps the content type intact. How should I handle this?

¹ I know that that is basically JSON, but my app needs to receive webhooks from an app which sends JSON-like data as `x-www-form-urlencoded`.


Discussion:
Hi @berzi,
Maybe we should clarify this in the documentation, but the [`simulate_*()`](https://falcon.readthedocs.io/en/stable/api/testing.html#simulating-requests) methods do not use any custom media handlers from the app by design. `TestClient` works roughly like the popular `requests` library, it pretends to have no specific knowledge of the app in question. In fact, with some luck it works with ASGI and WSGI apps written in other frameworks too, not just Falcon apps.


Hm, that's not at all what I expected. What is the reason for this?

Regardless, what else should I use to test my application with its configuration?
Not sure if we have ever thought of this, or received a feature request along these lines :)
But as said, these `simulate_*()` methods are meant to send requests to your app without going via the network, but they otherwise simulate how an app would be used by clients. If your app's user doesn't have access to its source code, or even doesn't use Python, they would obviously be unable to reuse your custom encoder either.

For now, the probably easiest way to test your application is to simply reuse the same methods manually to dump `body` for `simulate_post()`:
```python
import datetime
import functools
import json

import falcon.asgi
import falcon.media
import falcon.testing


class DatetimeEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            return obj.isoformat()
        return super().default(obj)


class Hello:
    async def on_post(self, req, resp):
        req_media = await req.get_media()
        resp.media = {
            'message': 'Hello!',
            'now': datetime.datetime.utcnow(),
            'req.media': req_media,
        }


def get_app():
    app = falcon.asgi.App()

    json_handler = falcon.media.JSONHandler(
        dumps=functools.partial(json.dumps, cls=DatetimeEncoder)
    )

    extra_handlers = {falcon.MEDIA_JSON: json_handler}
    app.req_options.media_handlers.update(extra_handlers)
    app.resp_options.media_handlers.update(extra_handlers)

    app.add_route('/', Hello())

    return app


def test_app():
    client = falcon.testing.TestClient(get_app())
    resp = client.simulate_post(
        '/',
        body=json.dumps(
            {'epoch': datetime.datetime.utcfromtimestamp(0)},
            cls=DatetimeEncoder,
        ),
        headers={'Content-Type': 'application/json'},
    )
    assert resp.json == {'msg': 'Hello'}
```

Now, my test fails, but everything seems to be wired as expected:
```
E       AssertionError: assert {'message': 'Hello!',\n 'now': '2022-08-28T07:55:09.015297',\n 'req.media': {'epoch': '1970-01-01T00:00:00'}} == {'msg': 'Hello'}
E         Left contains 3 more items:
E         {'message': 'Hello!',
E          'now': '2022-08-28T07:55:09.015297',
E          'req.media': {'epoch': '1970-01-01T00:00:00'}}
E         Right contains 1 more item:
E         {'msg': 'Hello'}
E         Full diff:
E           {
E         -  'msg': 'Hello',
E         +  'message': 'Hello!',
E         ?    + ++ +         +
E         +  'now': '2022-08-28T07:55:09.015297',
E         +  'req.media': {'epoch': '1970-01-01T00:00:00'},
E           }
```
Oh I get it now; for some reason I thought the decoder was the problem, not the encoder. It does make sense that encoding for incoming requests wouldn't happen the way my app defines it.

So am I supposed to use `body` and `application/json` even though I know the client will send `x-www-form-urlencoded`?
As to the `application/x-www-form-urlencoded` part -- no, you should use `application/x-www-form-urlencoded` then!
You should not add a JSON handler for `falcon.MEDIA_URLENCODED` unless you know that your clients are going to send JSON masquerading as `application/x-www-form-urlencoded`... (That would be pretty odd, but I have seen someone having exactly this situation.)

Again, you need to supply the `body` and `Content-Type` to `simulate_post()`:
```python
import datetime
import functools
import json

import falcon.asgi
import falcon.media
import falcon.testing


class DatetimeEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            return obj.isoformat()
        return super().default(obj)


class Hello:
    async def on_post(self, req, resp):
        req_media = await req.get_media()
        resp.media = {
            'message': 'Hello!',
            'now': datetime.datetime.utcnow(),
            'req.media': req_media,
        }


def get_app():
    app = falcon.asgi.App()

    json_handler = falcon.media.JSONHandler(
        dumps=functools.partial(json.dumps, cls=DatetimeEncoder)
    )

    extra_handlers = {falcon.MEDIA_JSON: json_handler}
    app.req_options.media_handlers.update(extra_handlers)
    app.resp_options.media_handlers.update(extra_handlers)

    app.add_route('/', Hello())

    return app


def test_app():
    client = falcon.testing.TestClient(get_app())
    resp = client.simulate_post(
        '/',
        body=falcon.to_query_str(
            {'app': 'webhook', 'value': 3.1}, prefix=False
        ),
        headers={'Content-Type': 'application/x-www-form-urlencoded'},
    )
    assert resp.json == {'msg': 'Hello'}
```

This works out of the box, because Falcon (as of 3.0+) installs [`URLEncodedFormHandler`](https://falcon.readthedocs.io/en/stable/api/media.html#falcon.media.URLEncodedFormHandler) for you by default (you can of course remove or customize it as well).

NB: note that `value` becomes `'3.1'` because URL-encoded form can only encode strings.
Yes, that is exactly the situation, unfortunately.

Thank you. I'll close this as I don't think my mistake was really due to any lack of documentation.
Reopening this, the work remaining here is to clarify in the docs that the test client is (almost) completely decoupled from the app, and it doesn't use the app's JSON handlers.

Other questions seem to duplicate #1636.

Resolving patch:
diff --git a/falcon/media/json.py b/falcon/media/json.py
--- a/falcon/media/json.py
+++ b/falcon/media/json.py
@@ -133,6 +133,15 @@ def default(self, obj):
         app.req_options.media_handlers.update(extra_handlers)
         app.resp_options.media_handlers.update(extra_handlers)
 
+    .. note:: When testing an application employing a custom JSON encoder, bear
+        in mind that :class:`~.testing.TestClient` is decoupled from the app,
+        and it simulates requests as if they were performed by a third-party
+        client (just sans network). Therefore, passing the **json** parameter
+        to :ref:`simulate_* <testing_standalone_methods>` methods will
+        effectively use the stdlib's :func:`json.dumps`. If you want to
+        serialize custom objects for testing, you will need to dump them into a
+        string yourself, and pass it using the **body** parameter instead
+        (accompanied by the ``application/json`` content type header).
 
     Keyword Arguments:
         dumps (func): Function to use when serializing JSON responses.


Test patch:
diff --git a/docs/api/testing.rst b/docs/api/testing.rst
--- a/docs/api/testing.rst
+++ b/docs/api/testing.rst
@@ -29,6 +29,8 @@ Main Interface
 .. autoclass:: Cookie
    :members:
 
+.. _testing_standalone_methods:
+
 Standalone Methods
 ~~~~~~~~~~~~~~~~~~