nebius/SWE-rebench-V2-PRs huggingface dataset:
Repository: encode/httpx
Pull request #2669 (2023-04-18 18:53:35)

Fix exception suppression in asgi transport
Bug description:

If httpx client uses asgi transport with `raise_app_exceptions=False` and response is not complete, exception is still raised.

This is mostly come up in tests where you want to check how your application behave in error states. But currently it works only with non-streaming responses, which is quite limiting.

I propose simple fix for such non-complete responses to be able to comply with the `raise_app_exceptions=False`.

Test case included.

tomchristie: Okay, let's also add this to the `CHANGELOG`.
What's the best user-facing description we can add here?

> Return `500` error response instead of exceptions when `raise_app_exceptions=False` is set on `ASGITransport`.

Perhaps?

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 71f5b02..d19b2ab 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,6 +8,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ### Fixed
 
+* Return `500` error response instead of exceptions when `raise_app_exceptions=False` is set on `ASGITransport`. (#2669)
 * Ensure all WSGITransport environs have a SERVER_PROTOCOL. (#2708)
 
 ## 0.24.1 (17th May, 2023)
diff --git a/httpx/_transports/asgi.py b/httpx/_transports/asgi.py
index bdf7f7a..f67f0fb 100644
--- a/httpx/_transports/asgi.py
+++ b/httpx/_transports/asgi.py
@@ -161,9 +161,15 @@ class ASGITransport(AsyncBaseTransport):
         try:
             await self.app(scope, receive, send)
         except Exception:  # noqa: PIE-786
-            if self.raise_app_exceptions or not response_complete.is_set():
+            if self.raise_app_exceptions:
                 raise
 
+            response_complete.set()
+            if status_code is None:
+                status_code = 500
+            if response_headers is None:
+                response_headers = {}
+
         assert response_complete.is_set()
         assert status_code is not None
         assert response_headers is not None


Test patch:
diff --git a/tests/test_asgi.py b/tests/test_asgi.py
index 17c8d5e..14d6df6 100644
--- a/tests/test_asgi.py
+++ b/tests/test_asgi.py
@@ -3,6 +3,7 @@ import json
 import pytest
 
 import httpx
+from httpx import ASGITransport
 
 
 async def hello_world(scope, receive, send):
@@ -191,3 +192,12 @@ async def test_asgi_disconnect_after_response_complete():
 
     assert response.status_code == 200
     assert disconnect
+
+
+@pytest.mark.anyio
+async def test_asgi_exc_no_raise():
+    transport = ASGITransport(app=raise_exc, raise_app_exceptions=False)
+    async with httpx.AsyncClient(transport=transport) as client:
+        response = await client.get("http://www.example.org/")
+
+        assert response.status_code == 500
Repository: encode/httpx
Pull request #2708 (2023-05-18 20:48:50)

Ensure all WSGITransport environs have a SERVER_PROTOCOL
<!-- Thanks for contributing to HTTPX! 💚
Given this is a project maintained by volunteers, please read this template to not waste your time, or ours! 😁 -->

# Summary

- I noticed that WSGI environs sent to applications with `httpx.Client(app=app)` did not contain a SERVER_PROTOCOL
- [PEP-3333](https://peps.python.org/pep-3333/#environ-variables) indicates that this key must be present in the WSGI environ, and popular WSGI frameworks (such as [webob](https://github.com/Pylons/webob)) assume it will be as well
- It looks like httpx defaults to using `HTTP/1.1` for a nearly identical case in the ASGITransport, so I did the same here
- I'm more than happy to discuss any of this further!

# Checklist

- [x] I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
- [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
- [x] I've updated the documentation accordingly.
  - _to my knowledge, this does not require a documentation update_


lazorchakp: Great, thanks! I'll give it a day or so before merging to see if anyone has other thoughts
tomchristie: valid.

Patch:
diff --git a/httpx/_transports/wsgi.py b/httpx/_transports/wsgi.py
index 33035ce..a23d42c 100644
--- a/httpx/_transports/wsgi.py
+++ b/httpx/_transports/wsgi.py
@@ -102,6 +102,7 @@ class WSGITransport(BaseTransport):
             "QUERY_STRING": request.url.query.decode("ascii"),
             "SERVER_NAME": request.url.host,
             "SERVER_PORT": str(port),
+            "SERVER_PROTOCOL": "HTTP/1.1",
             "REMOTE_ADDR": self.remote_addr,
         }
         for header_key, header_value in request.headers.raw:


Test patch:
diff --git a/tests/test_wsgi.py b/tests/test_wsgi.py
index ff3f24e..a952da6 100644
--- a/tests/test_wsgi.py
+++ b/tests/test_wsgi.py
@@ -173,3 +173,20 @@ def test_wsgi_server_port(url: str, expected_server_port: str) -> None:
     assert response.status_code == 200
     assert response.text == "Hello, World!"
     assert server_port == expected_server_port
+
+
+def test_wsgi_server_protocol():
+    server_protocol = None
+
+    def app(environ, start_response):
+        nonlocal server_protocol
+        server_protocol = environ["SERVER_PROTOCOL"]
+        start_response("200 OK", [("Content-Type", "text/plain")])
+        return [b"success"]
+
+    with httpx.Client(app=app, base_url="http://testserver") as client:
+        response = client.get("/")
+
+    assert response.status_code == 200
+    assert response.text == "success"
+    assert server_protocol == "HTTP/1.1"
Repository: encode/httpx
Pull request #2852 (2023-09-17 00:53:51)

Raise ValueError on `Response.encoding` being set after `Response.text` has been accessed
Discussed in #2041

<!-- Thanks for contributing to HTTPX! 💚
Given this is a project maintained by volunteers, please read this template to not waste your time, or ours! 😁 -->

# Summary

After `Response.text` is accessed, if you try to change `Response.encoding`, nothing happens. So just raise an Exception on change to `encoding` after `text` is accessed.

<!-- Write a small summary about what is happening here. -->

# Checklist

- [x] I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
- [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
- [ ] I've updated the documentation accordingly.


tomchristie: Okay yes, as discussed this is probably a marginal improvement in behaviour. Thanks.
tomchristie: What's a more precise title for this PR?
xzmeng: Modified. I'm not proficient in English, please change it if it's not appropriate.

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 66d2b08..4842dff 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
+## Unreleased
+
+### Fixed
+
+* Raise `ValueError` on `Response.encoding` being set after `Response.text` has been accessed. (#2852)
+
 ## 0.25.0 (11th Sep, 2023)
 
 ### Removed
diff --git a/httpx/_models.py b/httpx/_models.py
index e1e45cf..8a6bda0 100644
--- a/httpx/_models.py
+++ b/httpx/_models.py
@@ -603,6 +603,16 @@ class Response:
 
     @encoding.setter
     def encoding(self, value: str) -> None:
+        """
+        Set the encoding to use for decoding the byte content into text.
+
+        If the `text` attribute has been accessed, attempting to set the
+        encoding will throw a ValueError.
+        """
+        if hasattr(self, "_text"):
+            raise ValueError(
+                "Setting encoding after `text` has been accessed is not allowed."
+            )
         self._encoding = value
 
     @property


Test patch:
diff --git a/tests/models/test_responses.py b/tests/models/test_responses.py
index 9e65de8..9177773 100644
--- a/tests/models/test_responses.py
+++ b/tests/models/test_responses.py
@@ -298,6 +298,23 @@ def test_response_force_encoding():
     assert response.encoding == "iso-8859-1"
 
 
+def test_response_force_encoding_after_text_accessed():
+    response = httpx.Response(
+        200,
+        content=b"Hello, world!",
+    )
+    assert response.status_code == 200
+    assert response.reason_phrase == "OK"
+    assert response.text == "Hello, world!"
+    assert response.encoding == "utf-8"
+
+    with pytest.raises(ValueError):
+        response.encoding = "UTF8"
+
+    with pytest.raises(ValueError):
+        response.encoding = "iso-8859-1"
+
+
 def test_read():
     response = httpx.Response(
         200,
Repository: encode/httpx
Pull request #2886 (2023-10-10 08:45:53)

Fix encode host
# Summary

The ``encode_host`` method used the regular expression in the variable IPv4_STYLE_HOSTNAME. This was before:
``r"^[0-9]+.[0-9]+.[0-9]+.[0-9]+$"`` which wrongly allows hostnames like ``023b76x43144`` to pas as a ipv4 address. This is now fixed by changed the url to ``r"^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$"``

# Checklist

- [x] I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
- [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
- [ ] I've updated the documentation accordingly.: Not applicable IMHO



Patch:
diff --git a/httpx/_urlparse.py b/httpx/_urlparse.py
index e1ba8dc..8e06042 100644
--- a/httpx/_urlparse.py
+++ b/httpx/_urlparse.py
@@ -87,7 +87,7 @@ COMPONENT_REGEX = {
 
 # We use these simple regexs as a first pass before handing off to
 # the stdlib 'ipaddress' module for IP address validation.
-IPv4_STYLE_HOSTNAME = re.compile(r"^[0-9]+.[0-9]+.[0-9]+.[0-9]+$")
+IPv4_STYLE_HOSTNAME = re.compile(r"^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$")
 IPv6_STYLE_HOSTNAME = re.compile(r"^\[.*\]$")
 
 


Test patch:
diff --git a/tests/test_urlparse.py b/tests/test_urlparse.py
index 3ae9b04..b03291b 100644
--- a/tests/test_urlparse.py
+++ b/tests/test_urlparse.py
@@ -45,6 +45,12 @@ def test_urlparse_normalized_host():
     assert url.host == "example.com"
 
 
+def test_urlparse_ipv4_like_host():
+    """rare host names used to quality as IPv4"""
+    url = httpx.URL("https://023b76x43144/")
+    assert url.host == "023b76x43144"
+
+
 def test_urlparse_valid_ipv4():
     url = httpx.URL("https://1.2.3.4/")
     assert url.host == "1.2.3.4"
Repository: encode/httpx
Pull request #3116 (2024-02-23 21:33:35)

Fix client.send() timeout when used with a new Request instance
# Summary

Discussion here https://github.com/encode/httpx/discussions/3110

On `httpx.client.send()`, now the timeout of the request is set if not set already.
This is for cases in which `httpx.client.send()` is used with `httpx.Request()` instead of `client.build_request()`

I expected timeout tests for the non-async client too. But since there are only async tests I didn't create a non-async test.

I didn't think a documentation change was needed since this is (for me) the expect behavior.

# Checklist

- [X] I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
- [X] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
- [ ] I've updated the documentation accordingly.



Patch:
diff --git a/httpx/_client.py b/httpx/_client.py
index cf3b306..ba5decf 100644
--- a/httpx/_client.py
+++ b/httpx/_client.py
@@ -562,6 +562,15 @@ class BaseClient:
 
         return request.stream
 
+    def _set_timeout(self, request: Request) -> None:
+        if "timeout" not in request.extensions:
+            timeout = (
+                self.timeout
+                if isinstance(self.timeout, UseClientDefault)
+                else Timeout(self.timeout)
+            )
+            request.extensions = dict(**request.extensions, timeout=timeout.as_dict())
+
 
 class Client(BaseClient):
     """
@@ -911,6 +920,8 @@ class Client(BaseClient):
             else follow_redirects
         )
 
+        self._set_timeout(request)
+
         auth = self._build_request_auth(request, auth)
 
         response = self._send_handling_auth(
@@ -1658,6 +1669,8 @@ class AsyncClient(BaseClient):
             else follow_redirects
         )
 
+        self._set_timeout(request)
+
         auth = self._build_request_auth(request, auth)
 
         response = await self._send_handling_auth(


Test patch:
diff --git a/tests/test_timeouts.py b/tests/test_timeouts.py
index 09b2516..666cc8e 100644
--- a/tests/test_timeouts.py
+++ b/tests/test_timeouts.py
@@ -42,3 +42,14 @@ async def test_pool_timeout(server):
         with pytest.raises(httpx.PoolTimeout):
             async with client.stream("GET", server.url):
                 await client.get(server.url)
+
+
+@pytest.mark.anyio
+async def test_async_client_new_request_send_timeout(server):
+    timeout = httpx.Timeout(1e-6)
+
+    async with httpx.AsyncClient(timeout=timeout) as client:
+        with pytest.raises(httpx.TimeoutException):
+            await client.send(
+                httpx.Request("GET", server.url.copy_with(path="/slow_response"))
+            )
Repository: encode/httpx
Pull request #3312 (2024-09-23 18:12:25)

Clarified error when header value is None
Previously, when the header value was None, a misleading error was thrown: "AttributeError: Object 'NoneType' does not have attribute 'encode'." Now it looks better.

![image](https://github.com/user-attachments/assets/ec12330f-aa7e-4084-8c81-c89aef8fdea9)



Patch:
diff --git a/httpx/_utils.py b/httpx/_utils.py
index a9ece19..46a4d63 100644
--- a/httpx/_utils.py
+++ b/httpx/_utils.py
@@ -50,6 +50,8 @@ def normalize_header_value(value: str | bytes, encoding: str | None = None) -> b
     """
     if isinstance(value, bytes):
         return value
+    if not isinstance(value, str):
+        raise TypeError(f"Header value must be str or bytes, not {type(value)}")
     return value.encode(encoding or "ascii")
 
 


Test patch:
diff --git a/tests/client/test_headers.py b/tests/client/test_headers.py
index c51e40c..b8d2976 100755
--- a/tests/client/test_headers.py
+++ b/tests/client/test_headers.py
@@ -177,6 +177,14 @@ def test_header_does_not_exist():
         del headers["baz"]
 
 
+def test_header_with_incorrect_value():
+    with pytest.raises(
+        TypeError,
+        match=f"Header value must be str or bytes, not {type(None)}",
+    ):
+        httpx.Headers({"foo": None})  # type: ignore
+
+
 def test_host_with_auth_and_port_in_url():
     """
     The Host header should only include the hostname, or hostname:port
Repository: encode/httpx
Pull request #3376 (2024-10-28 17:34:46)

Just use default `safe=...` characters for `urlescape`
Don't try to be clever about which `safe=...` characters to use when calling `urlescape()`.



Patch:
diff --git a/httpx/_urls.py b/httpx/_urls.py
index bfc0e9e..7976cb1 100644
--- a/httpx/_urls.py
+++ b/httpx/_urls.py
@@ -12,20 +12,6 @@ from ._utils import primitive_value_to_str
 __all__ = ["URL", "QueryParams"]
 
 
-# To urlencode query parameters, we use the whatwg query percent-encode set
-# and additionally escape U+0025 (%), U+0026 (&), U+002B (+) and U+003D (=).
-
-# https://url.spec.whatwg.org/#percent-encoded-bytes
-
-URLENCODE_SAFE = "".join(
-    [
-        chr(i)
-        for i in range(0x20, 0x7F)
-        if i not in (0x20, 0x22, 0x23, 0x25, 0x26, 0x2B, 0x3C, 0x3D, 0x3E)
-    ]
-)
-
-
 class URL:
     """
     url = httpx.URL("HTTPS://jo%40email.com:a%20secret@müller.de:1234/pa%20th?search=ab#anchorlink")
@@ -619,7 +605,7 @@ class QueryParams(typing.Mapping[str, str]):
         return sorted(self.multi_items()) == sorted(other.multi_items())
 
     def __str__(self) -> str:
-        return urlencode(self.multi_items(), safe=URLENCODE_SAFE)
+        return urlencode(self.multi_items())
 
     def __repr__(self) -> str:
         class_name = self.__class__.__name__


Test patch:
diff --git a/tests/models/test_url.py b/tests/models/test_url.py
index d32ed52..03072e8 100644
--- a/tests/models/test_url.py
+++ b/tests/models/test_url.py
@@ -148,7 +148,7 @@ def test_url_query_encoding():
     assert url.raw_path == b"/?a=b+c&d=e/f"
 
     url = httpx.URL("https://www.example.com/", params={"a": "b c", "d": "e/f"})
-    assert url.raw_path == b"/?a=b+c&d=e/f"
+    assert url.raw_path == b"/?a=b+c&d=e%2Ff"
 
 
 def test_url_params():
@@ -309,7 +309,7 @@ def test_param_with_existing_escape_requires_encoding():
     # even if they include a valid escape sequence.
     # We want to match browser form behaviour here.
     url = httpx.URL("http://webservice", params={"u": "http://example.com?q=foo%2Fa"})
-    assert str(url) == "http://webservice?u=http://example.com?q%3Dfoo%252Fa"
+    assert str(url) == "http://webservice?u=http%3A%2F%2Fexample.com%3Fq%3Dfoo%252Fa"
 
 
 # Tests for query parameter percent encoding.
Repository: encode/httpx
Pull request #696 (2019-12-30 15:06:54)

Update link to timeout fine tuning in quickstart


Patch:
diff --git a/docs/quickstart.md b/docs/quickstart.md
index faa3495..15defb1 100644
--- a/docs/quickstart.md
+++ b/docs/quickstart.md
@@ -433,7 +433,7 @@ You can also disable the timeout behavior completely...
 >>> await httpx.get('https://github.com/', timeout=None)
 ```
 
-For advanced timeout management, see [Timeout fine-tuning](https://www.encode.io/httpx/advanced/#timeout-fine-tuning).
+For advanced timeout management, see [Timeout fine-tuning](https://www.encode.io/httpx/advanced/#fine-tuning-the-configuration).
 
 ## Authentication
 
diff --git a/httpx/content_streams.py b/httpx/content_streams.py
index 62d150b..73ef59e 100644
--- a/httpx/content_streams.py
+++ b/httpx/content_streams.py
@@ -7,6 +7,7 @@ from json import dumps as json_dumps
 from pathlib import Path
 from urllib.parse import urlencode
 
+from .exceptions import StreamConsumed
 from .utils import format_form_param
 
 RequestData = typing.Union[dict, str, bytes, typing.AsyncIterator[bytes]]
@@ -81,6 +82,7 @@ class AsyncIteratorStream(ContentStream):
     ) -> None:
         self.aiterator = aiterator
         self.close_func = close_func
+        self.is_stream_consumed = False
 
     def can_replay(self) -> bool:
         return False
@@ -89,6 +91,9 @@ class AsyncIteratorStream(ContentStream):
         return {"Transfer-Encoding": "chunked"}
 
     async def __aiter__(self) -> typing.AsyncIterator[bytes]:
+        if self.is_stream_consumed:
+            raise StreamConsumed()
+        self.is_stream_consumed = True
         async for part in self.aiterator:
             yield part
 


Test patch:
diff --git a/tests/test_content_streams.py b/tests/test_content_streams.py
index 64146ed..764e00e 100644
--- a/tests/test_content_streams.py
+++ b/tests/test_content_streams.py
@@ -3,6 +3,7 @@ import io
 import pytest
 
 from httpx.content_streams import encode
+from httpx.exceptions import StreamConsumed
 
 
 @pytest.mark.asyncio
@@ -39,6 +40,21 @@ async def test_aiterator_content():
     assert content == b"Hello, world!"
 
 
+@pytest.mark.asyncio
+async def test_aiterator_is_stream_consumed():
+    async def hello_world():
+        yield b"Hello, "
+        yield b"world!"
+
+    stream = encode(data=hello_world())
+    b"".join([part async for part in stream])
+
+    assert stream.is_stream_consumed
+
+    with pytest.raises(StreamConsumed) as _:
+        b"".join([part async for part in stream])
+
+
 @pytest.mark.asyncio
 async def test_json_content():
     stream = encode(json={"Hello": "world!"})
Repository: encode/httpx
Pull request #974 (2020-05-21 14:14:56)

Fix bytes support in multipart uploads
The `files` argument allows `str/bytes` along with file objects, but we were not handling the `bytes` branch yet (incorrectly treating that as file objects).

I've provided a test, but also have a follow-up PR coming to tighten up the type checking in the multipart implementation (which was too lose for mypy to catch this).


Patch:
diff --git a/httpx/_content_streams.py b/httpx/_content_streams.py
index 72c5c52..2484aa4 100644
--- a/httpx/_content_streams.py
+++ b/httpx/_content_streams.py
@@ -247,7 +247,7 @@ class MultipartStream(ContentStream):
         def get_length(self) -> int:
             headers = self.render_headers()
 
-            if isinstance(self.file, str):
+            if isinstance(self.file, (str, bytes)):
                 return len(headers) + len(self.file)
 
             # Let's do our best not to read `file` into memory.
@@ -279,7 +279,7 @@ class MultipartStream(ContentStream):
             return self._headers
 
         def render_data(self) -> typing.Iterator[bytes]:
-            if isinstance(self.file, str):
+            if isinstance(self.file, (str, bytes)):
                 yield to_bytes(self.file)
                 return
 
@@ -297,7 +297,7 @@ class MultipartStream(ContentStream):
                 self.file.seek(0)
 
         def can_replay(self) -> bool:
-            return True if isinstance(self.file, str) else self.file.seekable()
+            return True if isinstance(self.file, (str, bytes)) else self.file.seekable()
 
         def render(self) -> typing.Iterator[bytes]:
             yield self.render_headers()


Test patch:
diff --git a/tests/test_multipart.py b/tests/test_multipart.py
index 67f6a30..aeb287a 100644
--- a/tests/test_multipart.py
+++ b/tests/test_multipart.py
@@ -181,8 +181,14 @@ def test_multipart_encode_files_guesses_correct_content_type(
         )
 
 
-def test_multipart_encode_files_allows_str_content() -> None:
-    files = {"file": ("test.txt", "<string content>", "text/plain")}
+@pytest.mark.parametrize(
+    "value, output",
+    ((b"<bytes content>", "<bytes content>"), ("<string content>", "<string content>")),
+)
+def test_multipart_encode_files_allows_bytes_or_str_content(
+    value: typing.Union[str, bytes], output: str
+) -> None:
+    files = {"file": ("test.txt", value, "text/plain")}
     with mock.patch("os.urandom", return_value=os.urandom(16)):
         boundary = binascii.hexlify(os.urandom(16)).decode("ascii")
 
@@ -193,9 +199,9 @@ def test_multipart_encode_files_allows_str_content() -> None:
         content = (
             '--{0}\r\nContent-Disposition: form-data; name="file"; '
             'filename="test.txt"\r\n'
-            "Content-Type: text/plain\r\n\r\n<string content>\r\n"
+            "Content-Type: text/plain\r\n\r\n{1}\r\n"
             "--{0}--\r\n"
-            "".format(boundary).encode("ascii")
+            "".format(boundary, output).encode("ascii")
         )
         assert stream.get_headers()["Content-Length"] == str(len(content))
         assert b"".join(stream) == content