internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: encode/httpx

Issue:
Removed lowercased environment variables test cases from the proxies tests.
as per comment in #402 , separated PR for lowercased environment variables


Discussion:
These environment variables are standard to also accept the lowercase versions. See curl and requests. I think we should maintain the behavior. 
Given `curl` supports the lowercase variant I'm okay with us having both, yeah.
The only slight possible behavioral difference is that I'd also okay with us *underspecifying* a precedence. (Ie. If you're specifying both then we're not going to give you any definitive answer as to which you'd get.) The tests currently expect lowercase>uppercase, which I don't think is something we need to assert for.
So, what's the status on this? Should we re-add a test case that verifies lowercased environment variables are supported? (But not re-introduce the old tests which we asserting on which case is preferred.)

Resolving patch:
diff --git a/httpx/utils.py b/httpx/utils.py
--- a/httpx/utils.py
+++ b/httpx/utils.py
@@ -219,30 +219,13 @@ def get_environment_proxies() -> typing.Dict[str, str]:
     # Registry and Config for proxies on Windows and macOS.
     # We don't want to propagate non-HTTP proxies into
     # our configuration such as 'TRAVIS_APT_PROXY'.
-    proxies = {
+    supported_proxy_schemes = ("http", "https", "all")
+    return {
         key: val
         for key, val in getproxies().items()
-        if ("://" in key or key in ("http", "https"))
+        if ("://" in key or key in supported_proxy_schemes)
     }
 
-    # Favor lowercase environment variables over uppercase.
-    all_proxy = get_environ_lower_and_upper("ALL_PROXY")
-    if all_proxy is not None:
-        proxies["all"] = all_proxy
-
-    return proxies
-
-
-def get_environ_lower_and_upper(key: str) -> typing.Optional[str]:
-    """Gets a value from os.environ with both the lowercase and uppercase
-    environment variable. Prioritizes the lowercase environment variable.
-    """
-    for key in (key.lower(), key.upper()):
-        value = os.environ.get(key, None)
-        if value is not None and isinstance(value, str):
-            return value
-    return None
-
 
 def to_bytes(value: typing.Union[str, bytes], encoding: str = "utf-8") -> bytes:
     return value.encode(encoding) if isinstance(value, str) else value


Test patch:
diff --git a/tests/test_utils.py b/tests/test_utils.py
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -178,14 +178,7 @@ async def test_elapsed_timer():
             {"https_proxy": "http://127.0.0.1", "HTTP_PROXY": "https://127.0.0.1"},
             {"https": "http://127.0.0.1", "http": "https://127.0.0.1"},
         ),
-        (
-            {"all_proxy": "http://127.0.0.1", "ALL_PROXY": "https://1.1.1.1"},
-            {"all": "http://127.0.0.1"},
-        ),
-        (
-            {"https_proxy": "http://127.0.0.1", "HTTPS_PROXY": "https://1.1.1.1"},
-            {"https": "http://127.0.0.1"},
-        ),
+        ({"all_proxy": "http://127.0.0.1"}, {"all": "http://127.0.0.1"}),
         ({"TRAVIS_APT_PROXY": "http://127.0.0.1"}, {}),
     ],
 )
Repository: encode/httpx

Issue:
Instances of `tempfile.TemporaryFile` fail when used as an upload file.
When using `tempfile.TemporaryFile` the `file.name` attribute returns an integer, rather than the usual path string, which causes a breakage for us further down the line...

```shell
venv/lib/python3.7/site-packages/httpx/client.py:484: in post
    trust_env=trust_env,
venv/lib/python3.7/site-packages/httpx/client.py:616: in request
    cookies=cookies,
venv/lib/python3.7/site-packages/httpx/client.py:356: in build_request
    cookies=cookies,
venv/lib/python3.7/site-packages/httpx/models.py:696: in __init__
    content, content_type = self.encode_data(data, files, json)
venv/lib/python3.7/site-packages/httpx/models.py:619: in encode_data
    content, content_type = multipart_encode(data or {}, files)
venv/lib/python3.7/site-packages/httpx/multipart.py:100: in multipart_encode
    for field in iter_fields(data, files):
venv/lib/python3.7/site-packages/httpx/multipart.py:93: in iter_fields
    yield FileField(name=name, value=value)
venv/lib/python3.7/site-packages/httpx/multipart.py:51: in __init__
    self.filename = Path(getattr(value, "name", "upload")).name
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/pathlib.py:994: in __new__
    self = cls._from_parts(args, init=False)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/pathlib.py:649: in _from_parts
    drv, root, parts = self._parse_args(args)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cls = <class 'pathlib.PosixPath'>, args = (29,)

    @classmethod
    def _parse_args(cls, args):
        # This is useful when you don't want to create an instance, just
        # canonicalize some constructor arguments.
        parts = []
        for a in args:
            if isinstance(a, PurePath):
                parts += a._parts
            else:
>               a = os.fspath(a)
E               TypeError: expected str, bytes or os.PathLike object, not int
```

Have also confirmed that the issue *doesn't* occur with `tempfile.NamedTemporaryFile`.

I believe the resolution will be on this line...

https://github.com/encode/httpx/blob/1a32cf036a825f6eb35395af5388a3b23180a82e/httpx/multipart.py#L51

I assume that this would be sufficient...

```python
self.filename = Path(str(getattr(value, "name", "upload")).name 
```



Discussion:

Resolving patch:
diff --git a/httpx/multipart.py b/httpx/multipart.py
--- a/httpx/multipart.py
+++ b/httpx/multipart.py
@@ -48,7 +48,7 @@ def __init__(
     ) -> None:
         self.name = name
         if not isinstance(value, tuple):
-            self.filename = Path(getattr(value, "name", "upload")).name
+            self.filename = Path(str(getattr(value, "name", "upload"))).name
             self.file = value  # type: typing.Union[typing.IO[str], typing.IO[bytes]]
             self.content_type = self.guess_content_type()
         else:


Test patch:
Repository: encode/httpx

Issue:
Any explicit `Content-Type` header should take precedence over the encoding default.
https://github.com/encode/httpx/blob/eba29da632c3a0fce55a54fc18cd6b147db43726/httpx/models.py#L643

I send json argument and content type when i do request, but my content type is replacing to "application/json". I think it is bug


Discussion:
Hi @daxartio,

What exact use case are you facing that requires passing `json=...` while setting a `Content-Type` to something else than `application/json`?

As a workaround, using `data=...` should allow you to set the `Content-Type` yourself:

```python
data: bytes = serialize(some_dict)
r = await httpx.post(url, data=data, headers={"content-type": "application/x-myformat"})
```
Sure thing.

We shouold probably change this...

https://github.com/encode/httpx/blob/eba29da632c3a0fce55a54fc18cd6b147db43726/httpx/models.py#L627

To `self.headers.setdefault('Content-Type', content_type)`, so that if the user has *explicitly* set a Content-Type header that takes precedence.

(Or else leave that line as it is an use `if content_type and 'Content-Type' not in self.headers` in the previous line.)

Pull requests accepted.



Resolving patch:
diff --git a/httpx/models.py b/httpx/models.py
--- a/httpx/models.py
+++ b/httpx/models.py
@@ -624,7 +624,7 @@ def __init__(
             self.is_streaming = False
             self.content = content
             if content_type:
-                self.headers["Content-Type"] = content_type
+                self.headers.setdefault("Content-Type", content_type)
         elif isinstance(data, (str, bytes)):
             data = data.encode("utf-8") if isinstance(data, str) else data
             self.is_streaming = False


Test patch:
Repository: encode/httpx

Issue:
header value surrounded by whitespace
httpx [doesn't accept header value surrounded by whitespace](https://github.com/python-hyper/hyper-h2/blob/13005074d14c7d32f8eaf1683b854446a09d09d3/h2/utilities.py#L265), but other http2 implementations seems to accept them (nghttp2, browsers Firefox, Chrome).

Example:
```
async with httpx.Client(http2=True) as client:
	response = await client.get('https://a.searx.space/headervaluetest')
```

Result :
```
ProtocolError: Received header value surrounded by whitespace b'A value '
```

The nghttp2  implementation seems here:
* https://github.com/nghttp2/nghttp2/blob/bb519154fe62f7ff7e5eb7269e05043dec6d3682/lib/nghttp2_http.c#L332
* https://github.com/nghttp2/nghttp2/blob/bb519154fe62f7ff7e5eb7269e05043dec6d3682/lib/nghttp2_helper.c#L498

I'm not sure if it should be declared as a bug or not.


Discussion:
Gotcha. It *is* a protocol error, but it’d be better if we could prefer robust behaviour over errors. This’ll need filing against the hyper-h2 package.

Resolving patch:
diff --git a/httpx/dispatch/http2.py b/httpx/dispatch/http2.py
--- a/httpx/dispatch/http2.py
+++ b/httpx/dispatch/http2.py
@@ -2,6 +2,7 @@
 
 import h2.connection
 import h2.events
+from h2.config import H2Configuration
 from h2.settings import SettingCodes, Settings
 
 from ..concurrency.base import (
@@ -21,6 +22,7 @@
 
 class HTTP2Connection(OpenConnection):
     READ_NUM_BYTES = 4096
+    CONFIG = H2Configuration(validate_inbound_headers=False)
 
     def __init__(
         self,
@@ -31,7 +33,7 @@ def __init__(
         self.socket = socket
         self.backend = lookup_backend(backend)
         self.on_release = on_release
-        self.state = h2.connection.H2Connection()
+        self.state = h2.connection.H2Connection(config=self.CONFIG)
 
         self.streams = {}  # type: typing.Dict[int, HTTP2Stream]
         self.events = {}  # type: typing.Dict[int, typing.List[h2.events.Event]]


Test patch:
Repository: encode/httpx

Issue:
`params` overrides query string in url
Hi,

I noticed a difference between `httpx` and `requests` when processing query parameters. I provided the `params` for the url with query string, `requests` would **merge** `params` into query string, but `httpx` replaces the whole query string.

Here is a simple script:

```py
import requests
import httpx
import asyncio

URL = 'https://httpbin.org/get?a=1'

r = requests.get(URL, params={'b': 2})
print('requests', r.json()['args'])

async def f():
    r = await httpx.get(URL, params={'b': 2})
    print('httpx', r.json()['args'])
asyncio.run(f())
```

Results:
```
requests {'a': '1', 'b': '2'}
httpx {'b': '2'}
```

Is this an intended behavior?


Discussion:
Not intentional, nope.

It looks like we need to make sure the `merge_queryparams` also accepts the `url`, and includes any query params that are included there.

https://github.com/encode/httpx/blob/master/httpx/client.py#L370-L381
On second thoughts, we should actually be handling that bit on the `URL` class itself.

(So that manually building a request with `request = Request(url, params=params)` has the right merge behavior too.)

The signature there is `URL(url, params=None)`. When `params` are provided they *remove* any querystring on the `url`, and replace it with `params`. Instead we should merge any `params` argument with any query params that are present in the URL string.

Resolving patch:
diff --git a/httpx/models.py b/httpx/models.py
--- a/httpx/models.py
+++ b/httpx/models.py
@@ -90,9 +90,14 @@ def __init__(
         if self.is_absolute_url:
             self._uri_reference = self._uri_reference.normalize()
 
-        # Add any query parameters.
+        # Add any query parameters, merging with any in the URL if needed.
         if params:
-            query_string = str(QueryParams(params))
+            if self._uri_reference.query:
+                url_params = QueryParams(self._uri_reference.query)
+                url_params.update(params)
+                query_string = str(url_params)
+            else:
+                query_string = str(QueryParams(params))
             self._uri_reference = self._uri_reference.copy_with(query=query_string)
 
         # Enforce absolute URLs by default.


Test patch:
diff --git a/tests/models/test_url.py b/tests/models/test_url.py
--- a/tests/models/test_url.py
+++ b/tests/models/test_url.py
@@ -89,7 +89,7 @@ def test_url_params():
     assert str(url) == "https://example.org:123/path/to/somewhere?a=123"
 
     url = URL("https://example.org:123/path/to/somewhere?b=456", params={"a": "123"})
-    assert str(url) == "https://example.org:123/path/to/somewhere?a=123"
+    assert str(url) == "https://example.org:123/path/to/somewhere?b=456&a=123"
 
 
 def test_url_join():
Repository: encode/httpx

Issue:
DigestAuth should raise a clear error if the request cannot replay.
Our DigestAuth implementation cannot work with non-replayable requests.
We ought to raise a nice clear error if `request.stream.is_replayable` is not True.


Discussion:
See the client redirect code for an equivelent case with redirects.

Potentially a reasonable first issue.
@tomchristie hey can I work on this?
@kousikmitra Feel free to! We don’t tend to assign people to issues so feel free to go ahead and open a PR when you’d like to get feedback. :)

Resolving patch:
diff --git a/httpx/auth.py b/httpx/auth.py
--- a/httpx/auth.py
+++ b/httpx/auth.py
@@ -6,7 +6,7 @@
 from base64 import b64encode
 from urllib.request import parse_http_list
 
-from .exceptions import ProtocolError
+from .exceptions import ProtocolError, RequestBodyUnavailable
 from .models import Request, Response
 from .utils import to_bytes, to_str, unquote
 
@@ -104,6 +104,8 @@ def __init__(
         self.password = to_bytes(password)
 
     def __call__(self, request: Request) -> AuthFlow:
+        if not request.stream.can_replay():
+            raise RequestBodyUnavailable("Request body is no longer available.")
         response = yield request
 
         if response.status_code != 401 or "www-authenticate" not in response.headers:


Test patch:
diff --git a/tests/client/test_auth.py b/tests/client/test_auth.py
--- a/tests/client/test_auth.py
+++ b/tests/client/test_auth.py
@@ -5,7 +5,15 @@
 
 import pytest
 
-from httpx import URL, AsyncClient, DigestAuth, ProtocolError, Request, Response
+from httpx import (
+    URL,
+    AsyncClient,
+    DigestAuth,
+    ProtocolError,
+    Request,
+    RequestBodyUnavailable,
+    Response,
+)
 from httpx.auth import Auth, AuthFlow
 from httpx.config import CertTypes, TimeoutTypes, VerifyTypes
 from httpx.dispatch.base import Dispatcher
@@ -442,3 +450,16 @@ def __call__(self, request: Request) -> AuthFlow:
     assert resp2.history == [resp1]
 
     assert len(resp1.history) == 0
+
+
+@pytest.mark.asyncio
+async def test_digest_auth_unavailable_streaming_body():
+    url = "https://example.org/"
+    auth = DigestAuth(username="tomchristie", password="password123")
+    client = AsyncClient(dispatch=MockDispatch())
+
+    async def streaming_body():
+        yield b"Example request body"
+
+    with pytest.raises(RequestBodyUnavailable):
+        await client.post(url, data=streaming_body(), auth=auth)
Repository: encode/httpx

Issue:
AsnycIteratorStream should error if `__iter__` called twice.
For defensive programming, the `AsyncIteratorStream` implementation should track `.stream_consumed` in `__iter__`, and raise a `RuntimeError` if it is called twice.


Discussion:
Hello, I wanna work on this, but didn't understand the issue. As far as I explored the code, `is_stream_consumed flag` is on the `Response` class. And it seems everywhere this flag is being used, errors are raised appropriately. how would `AsyncIteratorStream` get access to this flag? The only connection I see between `Response` and `AsyncIteratorStream` is `HTTP2Connection` and `HTTP11Connection`
Heya. I didn't explain this fully - the class should have it's own `.is_stream_consumed` boolean.

Something along these lines...

```python
def __init__(self):
    ...
    self.is_stream_consumed = False

def __iter__(self):
    if self.is_stream_consumed:
        raise RuntimeError(...)
    self.is_stream_consumed = True
    ...  #OK
```

Plus a test in `test_content_streams.py` to ensure that it's working as expected.

Resolving patch:
diff --git a/httpx/content_streams.py b/httpx/content_streams.py
--- a/httpx/content_streams.py
+++ b/httpx/content_streams.py
@@ -7,6 +7,7 @@
 from pathlib import Path
 from urllib.parse import urlencode
 
+from .exceptions import StreamConsumed
 from .utils import format_form_param
 
 RequestData = typing.Union[dict, str, bytes, typing.AsyncIterator[bytes]]
@@ -81,6 +82,7 @@ def __init__(
     ) -> None:
         self.aiterator = aiterator
         self.close_func = close_func
+        self.is_stream_consumed = False
 
     def can_replay(self) -> bool:
         return False
@@ -89,6 +91,9 @@ def get_headers(self) -> typing.Dict[str, str]:
         return {"Transfer-Encoding": "chunked"}
 
     async def __aiter__(self) -> typing.AsyncIterator[bytes]:
+        if self.is_stream_consumed:
+            raise StreamConsumed()
+        self.is_stream_consumed = True
         async for part in self.aiterator:
             yield part
 


Test patch:
diff --git a/tests/test_content_streams.py b/tests/test_content_streams.py
--- a/tests/test_content_streams.py
+++ b/tests/test_content_streams.py
@@ -3,6 +3,7 @@
 import pytest
 
 from httpx.content_streams import encode
+from httpx.exceptions import StreamConsumed
 
 
 @pytest.mark.asyncio
@@ -39,6 +40,21 @@ async def hello_world():
     assert content == b"Hello, world!"
 
 
+@pytest.mark.asyncio
+async def test_aiterator_is_stream_consumed():
+    async def hello_world():
+        yield b"Hello, "
+        yield b"world!"
+
+    stream = encode(data=hello_world())
+    b"".join([part async for part in stream])
+
+    assert stream.is_stream_consumed
+
+    with pytest.raises(StreamConsumed) as _:
+        b"".join([part async for part in stream])
+
+
 @pytest.mark.asyncio
 async def test_json_content():
     stream = encode(json={"Hello": "world!"})
Repository: encode/httpx

Issue:
Redundant `boundary` if files/data is empty
If you try to send a post request with empty data and files — you will create a body with one `--boundary--` string, which can break the server.

It's because httpx create MultipartStream for `data={}, files={}`:
https://github.com/encode/httpx/blob/a82adcc933345c6b8cb1623b031eb85723e7665b/httpx/_content_streams.py#L332-L334

And this MultipartStream doesn't have special behavior for empty data/files:
https://github.com/encode/httpx/blob/a82adcc933345c6b8cb1623b031eb85723e7665b/httpx/_content_streams.py#L277-L283

Reproducible example:

```
from httpx.content_streams import encode

stream = encode(data={}, files={}, boundary=b"+++")
print(b"".join([part for part in stream]))
# b'--+++--\r\n'
```

I see two ways to solve it:
1. Check files not only for None but also for emptiness in `_content_streams.decode`
2. Check the count of fields in `MultipartStream.__init__` and doesn't write anything if fields are empty.

I already opened PR (#861) with a test example. If someone tells me which of these options is better, I will be happy to implement it  🙌


Discussion:

Resolving patch:
diff --git a/httpx/_content_streams.py b/httpx/_content_streams.py
--- a/httpx/_content_streams.py
+++ b/httpx/_content_streams.py
@@ -323,7 +323,7 @@ def encode(
     Handles encoding the given `data`, `files`, and `json`, returning
     a `ContentStream` implementation.
     """
-    if data is None:
+    if not data:
         if json is not None:
             return JSONStream(json=json)
         elif files:
@@ -331,7 +331,7 @@ def encode(
         else:
             return ByteStream(body=b"")
     elif isinstance(data, dict):
-        if files is not None:
+        if files:
             return MultipartStream(data=data, files=files, boundary=boundary)
         else:
             return URLEncodedStream(data=data)


Test patch:
diff --git a/tests/test_content_streams.py b/tests/test_content_streams.py
--- a/tests/test_content_streams.py
+++ b/tests/test_content_streams.py
@@ -189,6 +189,18 @@ async def test_multipart_data_and_files_content():
     )
 
 
+@pytest.mark.asyncio
+async def test_empty_request():
+    stream = encode(data={}, files={})
+    sync_content = b"".join([part for part in stream])
+    async_content = b"".join([part async for part in stream])
+
+    assert stream.can_replay()
+    assert stream.get_headers() == {}
+    assert sync_content == b""
+    assert async_content == b""
+
+
 def test_invalid_argument():
     with pytest.raises(TypeError):
         encode(123)
Repository: encode/httpx

Issue:
Document the exception hierarchy 
We've put a stack of great work into this for 0.14 - let's also show it off.


Discussion:
Refs #949 (for visibility there) :-)

Resolving patch:
diff --git a/httpx/_exceptions.py b/httpx/_exceptions.py
--- a/httpx/_exceptions.py
+++ b/httpx/_exceptions.py
@@ -45,15 +45,17 @@ class HTTPError(Exception):
     Base class for `RequestError` and `HTTPStatusError`.
 
     Useful for `try...except` blocks when issuing a request,
-    and then calling .raise_for_status().
+    and then calling `.raise_for_status()`.
 
     For example:
 
+    ```
     try:
         response = httpx.get("https://www.example.com")
         response.raise_for_status()
     except httpx.HTTPError as exc:
         print(f"HTTP Exception for {exc.request.url} - {exc.message}")
+    ```
     """
 
     def __init__(self, message: str, *, request: "Request") -> None:
@@ -72,7 +74,9 @@ def __init__(self, message: str, *, request: "Request") -> None:
 
 class TransportError(RequestError):
     """
-    Base class for all exceptions that are mapped from the httpcore API.
+    Base class for all exceptions that occur at the level of the Transport API.
+
+    All of these exceptions also have an equivelent mapping in `httpcore`.
     """
 
 
@@ -151,7 +155,7 @@ class CloseError(NetworkError):
 
 class ProxyError(TransportError):
     """
-    An error occurred while proxying a request.
+    An error occurred while establishing a proxy connection.
     """
 
 
@@ -192,7 +196,7 @@ class RemoteProtocolError(ProtocolError):
 
 class DecodingError(RequestError):
     """
-    Decoding of the response failed.
+    Decoding of the response failed, due to a malformed encoding.
     """
 
 
@@ -214,7 +218,7 @@ class RequestBodyUnavailable(RequestError):
 
 class HTTPStatusError(HTTPError):
     """
-    Response sent an error HTTP status.
+    The response had an error HTTP status of 4xx or 5xx.
 
     May be raised when calling `response.raise_for_status()`
     """


Test patch: