Repository: litestar-org/litestar

Issue:
Responses with a non `text/*` media type get rendered as JSON
### Summary
I'm using v2 from main.

The rendering (everything not `bytes`) of a response only depends on the provided media type and is defaulted to JSON. The logic for rendering can be a bit surprising when you want to return text with a non `text/*` media type.

### Basic Example

```python
from starlite import Response, Starlite, get


@get("/resource")
def retrieve_resource() -> Response[str]:
    return Response(content="<xml><msg>Hello World!</msg></xml>", media_type="application/xml")

app = Starlite(route_handlers=[retrieve_resource])
```
This will return `"<xml><msg>Hello World!</msg></xml>"` because it is rendered as JSON.

### Drawbacks and Impact

The media type is defaulted to JSON in
https://github.com/starlite-api/starlite/blob/c8aa65d1419590d7ccb738911783857ca92ddcc1/starlite/response/base.py#L60
This is ok and stated in the documentation.

The render logic in
https://github.com/starlite-api/starlite/blob/c8aa65d1419590d7ccb738911783857ca92ddcc1/starlite/response/base.py#L233-L254
only checks for `text/*`. Obviously starlite can't know every media type in existance but I think the default handling of an unknown media type could be better.

Explicitly check for known (starlite has special handling for it) media types (MediaType.JSON, MediaType.MESSAGEPACK).  Existing code still returns JSON because that's the default.

Handle unknown media types by some documented heuristic. I would propose checking if the type is `str` and then do the text encoding, otherwise throw an exception. Returning text and having it properly encoded is common enough to warrant handling in the framework. If a user needs custom rendering of his data, he can always do it himself and return `bytes`. 

### Unresolved questions

_No response_


Discussion:
Have you tried:

```python
from starlite import Response, Starlite, get, MediaType

@get("/resource", media_type=MediaType.XML)
def retrieve_resource() -> str:
    return "<xml><msg>Hello World!</msg></xml>"

app = Starlite(route_handlers=[retrieve_resource])
```

You can do this as well for any media type you want:

```python
from starlite import Response, Starlite, get

@get("/resource", media_type="application/my-favourite-media-type")
def retrieve_resource() -> str:
    return "Hello World!</msg>"

app = Starlite(route_handlers=[retrieve_resource])
```
Both will also return the string rendered as JSON (with quotes). Isn't it internally converted to `Response[str]`?

```console
$ curl http://127.0.0.1:8001/resource
"<xml><msg>Hello World!</msg></xml>"
```
If you  change `str` to `bytes` and return a normal string then you get the same result because of the default json encoding. 
As an addition to the render logic: Should this throw an error if the stated return type is different from the actual type? Starlite cares about annotating the return type and throws an error if you don't, should it also check that you return the correct type?

If you  change `str` to `bytes` and return a bytes string then you get:
```console
$ curl http://127.0.0.1:8001/resource
<xml><msg>Hello World!</msg></xml>
```
Sorry that I missed it again. I'm using v2 from main.
> Both will also return the string rendered as JSON (with quotes)

That would be a bug then. The example I provided is supposed to work. Would this behaviour (if working as intended) address your issue?
https://github.com/starlite-api/starlite/blob/c8aa65d1419590d7ccb738911783857ca92ddcc1/tests/response/test_serialization.py#L67-L70

We are missing the test case for XML and arbitrary types here.
> That would be a bug then. The example I provided is supposed to work. Would this behaviour (if working as intended) address your issue?

Yes that should do it. My expectation (guided by the documentation) is:

1. no `media_type` given or `media_type=MediaType.JSON`: render the stated return type as JSON.
2. `media_type=MediaType.MESSAGEPACK`: render the stated type as MESSAGEPACK.
3. `media_type=MediaType.TEXT`: render the encoded string.
4. `media_type=MediaType.HTML`: render the encoded string.

My expectation (not stated in the documentation):

5. `media_type=<anything>` and stated type `bytes`. Do nothing and just repond with the bytes.
6. `media_type=<anything else>` and stated type `str`. render the encoded string.
7. `media_type=<anything else>` and stated type `!= str` is ambigious (currently behaves like 1).

Regarding 7., for clarity I would like to get an exception. This would show me that starlite can't render the given type for the stated media_type (because it doesn't know what that media_type means) and that I should do it myself (return `bytes`, 5.).

Adding documentation for 5.-7. would help users that want to return more exotic reponses.

If you want to keep 7. as-is, maybe add a paragraph to the documentation stating that for media_types not in the [list](https://docs.starliteproject.dev/latest/usage/responses.html#media-type) the output will always be JSON and a user should use `bytes` to get the output he wants.


> Regarding 7., for clarity I would like to get an exception. This would show me that starlite can't render the given type for the stated media_type (because it doesn't know what that media_type means) and that I should do it myself

That's a very reasonable approach I think. @starlite-api/maintainers?

Do you want to add a PR, since you've apparently given it a good of amount of consideration already? 
im good with this, thanks @NomAnor for your efforts
All sounds good to me. I'm in favour of raising for 7 also, as long as the check is made during startup and we can have a nice exception message that details the error and the alternatives for the user.

> thanks @NomAnor for your efforts

Seconded!
Is this issue resolved @NomAnor ? should we close this issue or is there a PR incoming?

Resolving patch:
diff --git a/litestar/response/base.py b/litestar/response/base.py
--- a/litestar/response/base.py
+++ b/litestar/response/base.py
@@ -92,7 +92,7 @@ def __init__(
             dict(headers) if isinstance(headers, Mapping) else {h.name: h.value for h in headers or {}}
         )
         self.is_head_response = is_head_response
-        self.media_type = get_enum_string_value(media_type)
+        self.media_type = get_enum_string_value(media_type) or MediaType.JSON
         self.status_allows_body = not (
             status_code in {HTTP_204_NO_CONTENT, HTTP_304_NOT_MODIFIED} or status_code < HTTP_200_OK
         )
@@ -240,7 +240,7 @@ def render(self, content: Any) -> bytes:
             An encoded bytes string
         """
         try:
-            if self.media_type.startswith("text/"):
+            if self.media_type.startswith("text/") or isinstance(content, str):
                 if not content:
                     return b""
 
@@ -249,7 +249,10 @@ def render(self, content: Any) -> bytes:
             if self.media_type == MediaType.MESSAGEPACK:
                 return encode_msgpack(content, self._enc_hook)
 
-            return encode_json(content, self._enc_hook)
+            if self.media_type.startswith("application/json"):
+                return encode_json(content, self._enc_hook)
+
+            raise ImproperlyConfiguredException(f"unsupported media_type {self.media_type} for content {content!r}")
         except (AttributeError, ValueError, TypeError) as e:
             raise ImproperlyConfiguredException("Unable to serialize response content") from e
 


Test patch:
diff --git a/tests/contrib/htmx/test_htmx_request.py b/tests/contrib/htmx/test_htmx_request.py
--- a/tests/contrib/htmx/test_htmx_request.py
+++ b/tests/contrib/htmx/test_htmx_request.py
@@ -85,7 +85,7 @@ def handler(request: HTMXRequest) -> Response:
 
     with create_test_client(route_handlers=[handler], request_class=HTMXRequest) as client:
         response = client.get("/", headers={HTMXHeaders.CURRENT_URL.value: "https://example.com"})
-        assert response.text == '"https://example.com"'
+        assert response.text == "https://example.com"
 
 
 def test_current_url_set_url_encoded() -> None:
@@ -102,7 +102,7 @@ def handler(request: HTMXRequest) -> Response:
                 HTMXHeaders.CURRENT_URL.value + "-URI-AutoEncoded": "true",
             },
         )
-        assert response.text == '"https://example.com/?"'
+        assert response.text == "https://example.com/?"
 
 
 def test_current_url_abs_path_default() -> None:
@@ -126,7 +126,7 @@ def handler(request: HTMXRequest) -> Response:
         response = client.get(
             "/", headers={HTMXHeaders.CURRENT_URL.value: "http://testserver.local/duck/?quack=true#h2"}
         )
-        assert response.text == '"/duck/?quack=true#h2"'
+        assert response.text == "/duck/?quack=true#h2"
 
 
 def test_current_url_abs_path_set_other_domain() -> None:
@@ -181,7 +181,7 @@ def handler(request: HTMXRequest) -> Response:
 
     with create_test_client(route_handlers=[handler], request_class=HTMXRequest) as client:
         response = client.get("/", headers={HTMXHeaders.PROMPT.value: "Yes"})
-        assert response.text == '"Yes"'
+        assert response.text == "Yes"
 
 
 def test_target_default() -> None:
@@ -203,7 +203,7 @@ def handler(request: HTMXRequest) -> Response:
 
     with create_test_client(route_handlers=[handler], request_class=HTMXRequest) as client:
         response = client.get("/", headers={HTMXHeaders.TARGET.value: "#element"})
-        assert response.text == '"#element"'
+        assert response.text == "#element"
 
 
 def test_trigger_default() -> None:
@@ -225,7 +225,7 @@ def handler(request: HTMXRequest) -> Response:
 
     with create_test_client(route_handlers=[handler], request_class=HTMXRequest) as client:
         response = client.get("/", headers={HTMXHeaders.TRIGGER_ID.value: "#element"})
-        assert response.text == '"#element"'
+        assert response.text == "#element"
 
 
 def test_trigger_name_default() -> None:
@@ -247,7 +247,7 @@ def handler(request: HTMXRequest) -> Response:
 
     with create_test_client(route_handlers=[handler], request_class=HTMXRequest) as client:
         response = client.get("/", headers={HTMXHeaders.TRIGGER_NAME.value: "name_of_element"})
-        assert response.text == '"name_of_element"'
+        assert response.text == "name_of_element"
 
 
 def test_triggering_event_none() -> None:
diff --git a/tests/contrib/htmx/test_htmx_response.py b/tests/contrib/htmx/test_htmx_response.py
--- a/tests/contrib/htmx/test_htmx_response.py
+++ b/tests/contrib/htmx/test_htmx_response.py
@@ -77,7 +77,7 @@ def handler() -> PushUrl:
     with create_test_client(route_handlers=[handler], request_class=HTMXRequest) as client:
         response = client.get("/")
         assert response.status_code == HTTP_200_OK
-        assert response.text == '"Success!"'
+        assert response.text == "Success!"
         assert response.headers[HTMXHeaders.PUSH_URL] == "/index.html"
 
 
@@ -100,7 +100,7 @@ def handler() -> ReplaceUrl:
     with create_test_client(route_handlers=[handler], request_class=HTMXRequest) as client:
         response = client.get("/")
         assert response.status_code == HTTP_200_OK
-        assert response.text == '"Success!"'
+        assert response.text == "Success!"
         assert response.headers[HTMXHeaders.REPLACE_URL] == "/index.html"
 
 
@@ -112,7 +112,7 @@ def handler() -> Reswap:
     with create_test_client(route_handlers=[handler], request_class=HTMXRequest) as client:
         response = client.get("/")
         assert response.status_code == HTTP_200_OK
-        assert response.text == '"Success!"'
+        assert response.text == "Success!"
         assert response.headers[HTMXHeaders.RE_SWAP] == "beforebegin"
 
 
@@ -124,7 +124,7 @@ def handler() -> Retarget:
     with create_test_client(route_handlers=[handler], request_class=HTMXRequest) as client:
         response = client.get("/")
         assert response.status_code == HTTP_200_OK
-        assert response.text == '"Success!"'
+        assert response.text == "Success!"
         assert response.headers[HTMXHeaders.RE_TARGET] == "#element"
 
 
@@ -138,7 +138,7 @@ def handler() -> TriggerEvent:
     with create_test_client(route_handlers=[handler], request_class=HTMXRequest) as client:
         response = client.get("/")
         assert response.status_code == HTTP_200_OK
-        assert response.text == '"Success!"'
+        assert response.text == "Success!"
         assert response.headers[HTMXHeaders.TRIGGER_EVENT] == '{"alert":{"warning":"Confirm your choice!"}}'
 
 
@@ -151,7 +151,7 @@ def handler() -> TriggerEvent:
         response = client.get("/")
 
         assert response.status_code == HTTP_200_OK
-        assert response.text == '"Success!"'
+        assert response.text == "Success!"
         assert response.headers[HTMXHeaders.TRIGGER_EVENT] == '{"alert":{}}'
 
 
@@ -165,7 +165,7 @@ def handler() -> TriggerEvent:
     with create_test_client(route_handlers=[handler], request_class=HTMXRequest) as client:
         response = client.get("/")
         assert response.status_code == HTTP_200_OK
-        assert response.text == '"Success!"'
+        assert response.text == "Success!"
         assert response.headers[HTMXHeaders.TRIGGER_AFTER_SETTLE] == '{"alert":{"warning":"Confirm your choice!"}}'
 
 
@@ -177,7 +177,7 @@ def handler() -> TriggerEvent:
     with create_test_client(route_handlers=[handler], request_class=HTMXRequest) as client:
         response = client.get("/")
         assert response.status_code == HTTP_200_OK
-        assert response.text == '"Success!"'
+        assert response.text == "Success!"
         assert response.headers[HTMXHeaders.TRIGGER_AFTER_SWAP] == '{"alert":{"warning":"Confirm your choice!"}}'
 
 
diff --git a/tests/response/test_serialization.py b/tests/response/test_serialization.py
--- a/tests/response/test_serialization.py
+++ b/tests/response/test_serialization.py
@@ -8,6 +8,7 @@
 from pydantic import SecretStr
 
 from litestar import MediaType, Response
+from litestar.exceptions import ImproperlyConfiguredException
 from litestar.status_codes import HTTP_200_OK
 from tests import (
     MsgSpecStructPerson,
@@ -70,3 +71,23 @@ def test_response_serialization_structured_types(content: Any, response_type: An
 def test_response_serialization_text_types(content: Any, response_type: Any, media_type: MediaType) -> None:
     response = Response[response_type](content, media_type=media_type, status_code=HTTP_200_OK)
     assert response.body == content.encode("utf-8")
+
+
+@pytest.mark.parametrize(
+    "content, response_type, media_type, should_raise",
+    [
+        ["abcdefg", str, "text/custom", False],
+        ["<xml/>", str, "application/unknown", False],
+        [b"<xml/>", bytes, "application/unknown", False],
+        [{"key": "value"}, dict, "application/unknown", True],
+    ],
+)
+def test_response_validation_of_unknown_media_types(
+    content: Any, response_type: Any, media_type: MediaType, should_raise: bool
+) -> None:
+    if should_raise:
+        with pytest.raises(ImproperlyConfiguredException):
+            Response[response_type](content, media_type=media_type, status_code=HTTP_200_OK)
+    else:
+        response = Response[response_type](content, media_type=media_type, status_code=HTTP_200_OK)
+        assert response.body == (content.encode("utf-8") if not isinstance(content, bytes) else content)