Repository: encode/httpx
Pull request #880 (2020-03-28 07:50:09)

Add issue templates
A second iteration over #741, with templates for questions, bug reports and feature requests.

I think with these in place we can hope to reduce maintainer burden, as contributors will *think* more thoroughly about alternatives to opening an issue, possible solutions, etc.

This should help improve the contributing experience for all. :-)

(I based these off of the default GitHub templates, as well as those we can find on the [FastAPI repo](https://github.com/tiangolo/fastapi/issues/new/choose).)


Patch:
diff --git a/.github/ISSUE_TEMPLATE/1-question.md b/.github/ISSUE_TEMPLATE/1-question.md
new file mode 100644
index 0000000..d93521a
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/1-question.md
@@ -0,0 +1,17 @@
+---
+name: Question
+about: Ask a question
+---
+
+### Checklist
+
+<!-- To help keep this issue tracker clean and focused, please make sure you tried *all* the following resources before submitting your question. -->
+
+- [ ] I searched the [HTTPX documentation](https://www.python-httpx.org) but couldn't find what I'm looking for.
+- [ ] I looked through similar issues on GitHub, but didn't find anything.
+- [ ] I looked up "How to do ... in HTTPX" on a search engine and didn't find any information.
+- [ ] I asked the [community chat](https://gitter.im/encode/community) for help but didn't get an answer.
+
+### Question
+
+<!-- Please ask your question here. -->
diff --git a/.github/ISSUE_TEMPLATE/2-bug-report.md b/.github/ISSUE_TEMPLATE/2-bug-report.md
new file mode 100644
index 0000000..a206030
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/2-bug-report.md
@@ -0,0 +1,61 @@
+---
+name: Bug report
+about: Report a bug to help improve this project
+---
+
+### Checklist
+
+<!-- Please make sure you check all these items before submitting your bug report. -->
+
+- [ ] The bug is reproducible against the latest release and/or `master`.
+- [ ] There are no similar issues or pull requests to fix it yet.
+
+### Describe the bug
+
+<!-- A clear and concise description of what the bug is. -->
+
+### To reproduce
+
+<!-- Provide a *minimal* example with steps to reproduce the bug locally.
+
+NOTE: try to keep any external dependencies *at an absolute minimum* (servers, proxies, certificates...).
+In other words, remove anything that doesn't make the bug go away.
+
+If you need a local server to replicate against, you can build one using eg. Starlette [0] and Uvicorn [1], or any tool you feel comfortable with. Check out other issues for examples and remember to share setup instructions here. :-)
+
+[0]: https://www.starlette.io
+[1]: https://www.uvicorn.org
+-->
+
+### Expected behavior
+
+<!-- A clear and concise description of what you expected to happen. -->
+
+### Actual behavior
+
+<!-- A clear and concise description of what actually happens. -->
+
+### Debugging material
+
+<!-- Any tracebacks, screenshots, etc. that can help understanding the problem.
+
+NOTE:
+- Please list tracebacks in full (don't truncate them).
+- If relevant, consider turning on DEBUG or TRACE logs for additional details (see https://www.python-httpx.org/environment_variables/#httpx_log_level).
+- Consider using `<details>` to make tracebacks/logs collapsible if they're very large (see https://gist.github.com/ericclemmons/b146fe5da72ca1f706b2ef72a20ac39d).
+-->
+
+### Environment
+
+- OS: <!-- eg Linux/Windows/macOS. -->
+- Python version: <!-- eg 3.8.2 (get it with `$ python -V`). -->
+- HTTPX version: <!-- eg 0.12.0 (get it with `$ pip show httpx`). -->
+- Async environment: <!-- eg asyncio/trio. If using asyncio, include whether the bug reproduces on trio (and vice versa). -->
+- HTTP proxy: <!-- yes/no, if yes please try reproducing without it. -->
+- Custom certificates: <!-- yes/no, if yes please try reproducing without them. If the bug is related to SSL/TLS, you can setup HTTPS on a local server using these instructions: https://www.python-httpx.org/advanced/#making-https-requests-to-a-local-server. -->
+
+### Additional context
+
+<!-- Any additional information that can help understanding the problem.
+
+Eg. linked issues, or a description of what you were trying to achieve. -->
diff --git a/.github/ISSUE_TEMPLATE/3-feature-request.md b/.github/ISSUE_TEMPLATE/3-feature-request.md
new file mode 100644
index 0000000..a4237e2
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/3-feature-request.md
@@ -0,0 +1,34 @@
+---
+name: Feature request
+about: Suggest an idea for this project.
+---
+
+### Checklist
+
+<!-- Please make sure you check all these items before submitting your feature request. -->
+
+- [ ] There are no similar issues or pull requests for this yet.
+- [ ] I discussed this idea on the [community chat](https://gitter.im/encode/community) and feedback is positive.
+
+### Is your feature related to a problem? Please describe.
+
+<!-- A clear and concise description of what you are trying to achieve.
+
+Eg "I want to be able to [...] but I can't because [...]". -->
+
+## Describe the solution you would like.
+
+<!-- A clear and concise description of what you would want to happen.
+
+For API changes, try to provide a code snippet of what you would like the API to look like.
+-->
+
+## Describe alternatives you considered
+
+<!-- Please describe any alternative solutions or features you've considered to solve
+your problem and why they wouldn't solve it. -->
+
+## Additional context
+
+<!-- Provide any additional context, screenshots, tracebacks, etc. about the feature here. -->
+
diff --git a/README.md b/README.md
index 574c7c3..0693a5a 100644
--- a/README.md
+++ b/README.md
@@ -89,7 +89,7 @@ Install with pip:
 $ pip install httpx
 ```
 
-httpx requires Python 3.6+
+HTTPX requires Python 3.6+.
 
 ## Documentation
 
@@ -109,7 +109,7 @@ If you want to contribute with HTTPX check out the [Contributing Guide](https://
 
 ## Dependencies
 
-The httpx project relies on these excellent libraries:
+The HTTPX project relies on these excellent libraries:
 
 * `urllib3` - Sync client support.
 * `h11` - HTTP/1.1 support.
diff --git a/docs/api.md b/docs/api.md
index 3889d8c..c63d4ec 100644
--- a/docs/api.md
+++ b/docs/api.md
@@ -38,6 +38,13 @@
     :docstring:
     :members: headers cookies params request get head options post put patch delete build_request send close
 
+## `AsyncClient`
+
+::: httpx.AsyncClient
+    :docstring:
+    :members: headers cookies params request get head options post put patch delete build_request send
+
+
 ## `Response`
 
 *An HTTP response.*
diff --git a/httpx/_client.py b/httpx/_client.py
index 2a310e2..50279b0 100644
--- a/httpx/_client.py
+++ b/httpx/_client.py
@@ -426,7 +426,7 @@ class Client(BaseClient):
     request URLs.
     * **dispatch** - *(optional)* A dispatch class to use for sending requests
     over the network.
-    * **app** - *(optional)* An ASGI application to send requests to,
+    * **app** - *(optional)* An WSGI application to send requests to,
     rather than sending actual network requests.
     * **trust_env** - *(optional)* Enables or disables usage of environment
     variables for configuration.
@@ -680,8 +680,8 @@ class Client(BaseClient):
             # Add the original request to any HTTPError unless
             # there'a already a request attached in the case of
             # a ProxyError.
-            if exc.request is None:
-                exc.request = request
+            if exc._request is None:
+                exc._request = request
             raise
 
         self.cookies.extract_cookies(response)
@@ -1198,8 +1198,8 @@ class AsyncClient(BaseClient):
             # Add the original request to any HTTPError unless
             # there'a already a request attached in the case of
             # a ProxyError.
-            if exc.request is None:
-                exc.request = request
+            if exc._request is None:
+                exc._request = request
             raise
 
         self.cookies.extract_cookies(response)
diff --git a/httpx/_content_streams.py b/httpx/_content_streams.py
index afffed3..e6a3881 100644
--- a/httpx/_content_streams.py
+++ b/httpx/_content_streams.py
@@ -8,6 +8,7 @@ from pathlib import Path
 from urllib.parse import urlencode
 
 from ._exceptions import StreamConsumed
+from ._types import StrOrBytes
 from ._utils import format_form_param
 
 RequestData = typing.Union[
@@ -18,15 +19,16 @@ RequestFiles = typing.Dict[
     str,
     typing.Union[
         # file (or str)
-        typing.Union[typing.IO[typing.AnyStr], typing.AnyStr],
+        typing.Union[typing.IO[str], typing.IO[bytes], StrOrBytes],
         # (filename, file (or str))
         typing.Tuple[
-            typing.Optional[str], typing.Union[typing.IO[typing.AnyStr], typing.AnyStr],
+            typing.Optional[str],
+            typing.Union[typing.IO[str], typing.IO[bytes], StrOrBytes],
         ],
         # (filename, file (or str), content_type)
         typing.Tuple[
             typing.Optional[str],
-            typing.Union[typing.IO[typing.AnyStr], typing.AnyStr],
+            typing.Union[typing.IO[str], typing.IO[bytes], StrOrBytes],
             typing.Optional[str],
         ],
     ],
@@ -71,8 +73,6 @@ class ByteStream(ContentStream):
         self.body = body.encode("utf-8") if isinstance(body, str) else body
 
     def get_headers(self) -> typing.Dict[str, str]:
-        if not self.body:
-            return {}
         content_length = str(len(self.body))
         return {"Content-Length": content_length}
 
@@ -224,7 +224,9 @@ class MultipartStream(ContentStream):
         """
 
         def __init__(
-            self, name: str, value: typing.Union[typing.IO[typing.AnyStr], tuple]
+            self,
+            name: str,
+            value: typing.Union[typing.IO[str], typing.IO[bytes], tuple],
         ) -> None:
             self.name = name
             if not isinstance(value, tuple):
diff --git a/httpx/_dispatch/wsgi.py b/httpx/_dispatch/wsgi.py
index 60540a8..8622773 100644
--- a/httpx/_dispatch/wsgi.py
+++ b/httpx/_dispatch/wsgi.py
@@ -1,4 +1,5 @@
 import io
+import itertools
 import typing
 
 from .._config import TimeoutTypes
@@ -7,6 +8,14 @@ from .._models import Request, Response
 from .base import SyncDispatcher
 
 
+def _skip_leading_empty_chunks(body: typing.Iterable) -> typing.Iterable:
+    body = iter(body)
+    for chunk in body:
+        if chunk:
+            return itertools.chain([chunk], body)
+    return []
+
+
 class WSGIDispatch(SyncDispatcher):
     """
     A custom SyncDispatcher that handles sending requests directly to an WSGI app.
@@ -88,6 +97,9 @@ class WSGIDispatch(SyncDispatcher):
             seen_exc_info = exc_info
 
         result = self.app(environ, start_response)
+        # This is needed because the status returned by start_response
+        # shouldn't be used until the first non-empty chunk has been served.
+        result = _skip_leading_empty_chunks(result)
 
         assert seen_status is not None
         assert seen_response_headers is not None
diff --git a/httpx/_exceptions.py b/httpx/_exceptions.py
index 0f0ea06..9b6bd22 100644
--- a/httpx/_exceptions.py
+++ b/httpx/_exceptions.py
@@ -6,15 +6,22 @@ if typing.TYPE_CHECKING:
 
 class HTTPError(Exception):
     """
-    Base class for all httpx exceptions.
+    Base class for all HTTPX exceptions.
     """
 
     def __init__(
         self, *args: typing.Any, request: "Request" = None, response: "Response" = None
     ) -> None:
-        self.response = response
-        self.request = request or getattr(self.response, "request", None)
         super().__init__(*args)
+        self._request = request or (response.request if response is not None else None)
+        self.response = response
+
+    @property
+    def request(self) -> "Request":
+        # NOTE: this property exists so that a `Request` is exposed to type
+        # checkers, instead of `Optional[Request]`.
+        assert self._request is not None  # Populated by the client.
+        return self._request
 
 
 # Timeout exceptions...
diff --git a/httpx/_models.py b/httpx/_models.py
index 2bfa367..9465e45 100644
--- a/httpx/_models.py
+++ b/httpx/_models.py
@@ -39,6 +39,7 @@ from ._exceptions import (
     StreamConsumed,
 )
 from ._status_codes import StatusCode
+from ._types import StrOrBytes
 from ._utils import (
     ElapsedTimer,
     flatten_queryparams,
@@ -67,8 +68,8 @@ QueryParamTypes = typing.Union[
 
 HeaderTypes = typing.Union[
     "Headers",
-    typing.Dict[typing.AnyStr, typing.AnyStr],
-    typing.List[typing.Tuple[typing.AnyStr, typing.AnyStr]],
+    typing.Dict[StrOrBytes, StrOrBytes],
+    typing.Sequence[typing.Tuple[StrOrBytes, StrOrBytes]],
 ]
 
 CookieTypes = typing.Union["Cookies", CookieJar, typing.Dict[str, str]]
diff --git a/httpx/_types.py b/httpx/_types.py
new file mode 100644
index 0000000..cb13a60
--- /dev/null
+++ b/httpx/_types.py
@@ -0,0 +1,7 @@
+"""
+Type definitions for type checking purposes.
+"""
+
+from typing import Union
+
+StrOrBytes = Union[str, bytes]
diff --git a/httpx/_utils.py b/httpx/_utils.py
index b884f81..83312fd 100644
--- a/httpx/_utils.py
+++ b/httpx/_utils.py
@@ -14,6 +14,7 @@ from types import TracebackType
 from urllib.request import getproxies
 
 from ._exceptions import NetworkError
+from ._types import StrOrBytes
 
 if typing.TYPE_CHECKING:  # pragma: no cover
     from ._models import PrimitiveData
@@ -29,7 +30,7 @@ _HTML5_FORM_ENCODING_RE = re.compile(
 )
 
 
-def normalize_header_key(value: typing.AnyStr, encoding: str = None) -> bytes:
+def normalize_header_key(value: StrOrBytes, encoding: str = None) -> bytes:
     """
     Coerce str/bytes into a strictly byte-wise HTTP header key.
     """
@@ -38,7 +39,7 @@ def normalize_header_key(value: typing.AnyStr, encoding: str = None) -> bytes:
     return value.encode(encoding or "ascii").lower()
 
 
-def normalize_header_value(value: typing.AnyStr, encoding: str = None) -> bytes:
+def normalize_header_value(value: StrOrBytes, encoding: str = None) -> bytes:
     """
     Coerce str/bytes into a strictly byte-wise HTTP header value.
     """
@@ -204,8 +205,8 @@ SENSITIVE_HEADERS = {"authorization", "proxy-authorization"}
 
 
 def obfuscate_sensitive_headers(
-    items: typing.Iterable[typing.Tuple[typing.AnyStr, typing.AnyStr]]
-) -> typing.Iterator[typing.Tuple[typing.AnyStr, typing.AnyStr]]:
+    items: typing.Iterable[typing.Tuple[StrOrBytes, StrOrBytes]]
+) -> typing.Iterator[typing.Tuple[StrOrBytes, StrOrBytes]]:
     for k, v in items:
         if to_str(k.lower()) in SENSITIVE_HEADERS:
             v = to_bytes_or_str("[secure]", match_type_of=v)
@@ -301,7 +302,7 @@ def to_str(value: typing.Union[str, bytes], encoding: str = "utf-8") -> str:
     return value if isinstance(value, str) else value.decode(encoding)
 
 
-def to_bytes_or_str(value: str, match_type_of: typing.AnyStr) -> typing.AnyStr:
+def to_bytes_or_str(value: str, match_type_of: StrOrBytes) -> StrOrBytes:
     return value if isinstance(match_type_of, str) else value.encode()
 
 
diff --git a/requirements.txt b/requirements.txt
index ec60773..056a393 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -4,8 +4,7 @@
 brotlipy==0.7.*
 
 autoflake
-# See: https://github.com/encode/httpx/pull/639#issuecomment-566235998
-black --no-binary=regex
+black
 cryptography
 flake8
 flake8-bugbear


Test patch:
diff --git a/tests/client/test_headers.py b/tests/client/test_headers.py
index 22b0ba1..5c4dbb4 100755
--- a/tests/client/test_headers.py
+++ b/tests/client/test_headers.py
@@ -40,6 +40,7 @@ async def test_client_header():
             "accept": "*/*",
             "accept-encoding": "gzip, deflate, br",
             "connection": "keep-alive",
+            "content-length": "0",
             "example-header": "example-value",
             "host": "example.org",
             "user-agent": f"python-httpx/{__version__}",
@@ -61,6 +62,7 @@ async def test_header_merge():
             "accept": "*/*",
             "accept-encoding": "gzip, deflate, br",
             "connection": "keep-alive",
+            "content-length": "0",
             "host": "example.org",
             "user-agent": "python-myclient/0.2.1",
             "x-auth-token": "FooBarBazToken",
@@ -82,6 +84,7 @@ async def test_header_merge_conflicting_headers():
             "accept": "*/*",
             "accept-encoding": "gzip, deflate, br",
             "connection": "keep-alive",
+            "content-length": "0",
             "host": "example.org",
             "user-agent": f"python-httpx/{__version__}",
             "x-auth-token": "BazToken",
@@ -105,6 +108,7 @@ async def test_header_update():
             "accept": "*/*",
             "accept-encoding": "gzip, deflate, br",
             "connection": "keep-alive",
+            "content-length": "0",
             "host": "example.org",
             "user-agent": f"python-httpx/{__version__}",
         }
@@ -117,6 +121,7 @@ async def test_header_update():
             "accept-encoding": "gzip, deflate, br",
             "another-header": "AThing",
             "connection": "keep-alive",
+            "content-length": "0",
             "host": "example.org",
             "user-agent": "python-myclient/0.2.1",
         }
@@ -147,6 +152,7 @@ async def test_host_with_auth_and_port_in_url():
             "accept": "*/*",
             "accept-encoding": "gzip, deflate, br",
             "connection": "keep-alive",
+            "content-length": "0",
             "host": "example.org",
             "user-agent": f"python-httpx/{__version__}",
             "authorization": "Basic dXNlcm5hbWU6cGFzc3dvcmQ=",
@@ -171,6 +177,7 @@ async def test_host_with_non_default_port_in_url():
             "accept": "*/*",
             "accept-encoding": "gzip, deflate, br",
             "connection": "keep-alive",
+            "content-length": "0",
             "host": "example.org:123",
             "user-agent": f"python-httpx/{__version__}",
             "authorization": "Basic dXNlcm5hbWU6cGFzc3dvcmQ=",
diff --git a/tests/client/test_redirects.py b/tests/client/test_redirects.py
index 190ff09..63403a6 100644
--- a/tests/client/test_redirects.py
+++ b/tests/client/test_redirects.py
@@ -279,7 +279,7 @@ async def test_body_redirect():
     response = await client.post(url, data=data)
     assert response.url == URL("https://example.org/redirect_body_target")
     assert response.json()["body"] == "Example request body"
-    assert "content-length" in response.json()["headers"]
+    assert response.json()["headers"]["content-length"] == "20"
 
 
 @pytest.mark.usefixtures("async_environment")
@@ -293,7 +293,7 @@ async def test_no_body_redirect():
     response = await client.post(url, data=data)
     assert response.url == URL("https://example.org/redirect_body_target")
     assert response.json()["body"] == ""
-    assert "content-length" not in response.json()["headers"]
+    assert response.json()["headers"]["content-length"] == "0"
 
 
 @pytest.mark.usefixtures("async_environment")
diff --git a/tests/models/test_requests.py b/tests/models/test_requests.py
index c72e0af..0461394 100644
--- a/tests/models/test_requests.py
+++ b/tests/models/test_requests.py
@@ -10,7 +10,7 @@ def test_request_repr():
 
 def test_no_content():
     request = httpx.Request("GET", "http://example.org")
-    assert "Content-Length" not in request.headers
+    assert request.headers["Content-Length"] == "0"
 
 
 def test_content_length_header():
diff --git a/tests/test_content_streams.py b/tests/test_content_streams.py
index c5eb961..84999ef 100644
--- a/tests/test_content_streams.py
+++ b/tests/test_content_streams.py
@@ -25,7 +25,9 @@ async def test_empty_content():
     async_content = b"".join([part async for part in stream])
 
     assert stream.can_replay()
-    assert stream.get_headers() == {}
+    assert stream.get_headers() == {
+        "Content-Length": "0",
+    }
     assert sync_content == b""
     assert async_content == b""
 
@@ -196,7 +198,9 @@ async def test_empty_request():
     async_content = b"".join([part async for part in stream])
 
     assert stream.can_replay()
-    assert stream.get_headers() == {}
+    assert stream.get_headers() == {
+        "Content-Length": "0",
+    }
     assert sync_content == b""
     assert async_content == b""
 
diff --git a/tests/test_wsgi.py b/tests/test_wsgi.py
index 0dcf995..1786d9e 100644
--- a/tests/test_wsgi.py
+++ b/tests/test_wsgi.py
@@ -5,18 +5,20 @@ import pytest
 import httpx
 
 
-def hello_world(environ, start_response):
-    status = "200 OK"
-    output = b"Hello, World!"
+def application_factory(output):
+    def application(environ, start_response):
+        status = "200 OK"
 
-    response_headers = [
-        ("Content-type", "text/plain"),
-        ("Content-Length", str(len(output))),
-    ]
+        response_headers = [
+            ("Content-type", "text/plain"),
+        ]
 
-    start_response(status, response_headers)
+        start_response(status, response_headers)
 
-    return [output]
+        for item in output:
+            yield item
+
+    return application
 
 
 def echo_body(environ, start_response):
@@ -25,7 +27,6 @@ def echo_body(environ, start_response):
 
     response_headers = [
         ("Content-type", "text/plain"),
-        ("Content-Length", str(len(output))),
     ]
 
     start_response(status, response_headers)
@@ -56,7 +57,6 @@ def raise_exc(environ, start_response):
 
     response_headers = [
         ("Content-type", "text/plain"),
-        ("Content-Length", str(len(output))),
     ]
 
     try:
@@ -69,7 +69,7 @@ def raise_exc(environ, start_response):
 
 
 def test_wsgi():
-    client = httpx.Client(app=hello_world)
+    client = httpx.Client(app=application_factory([b"Hello, World!"]))
     response = client.get("http://www.example.org/")
     assert response.status_code == 200
     assert response.text == "Hello, World!"
@@ -93,3 +93,19 @@ def test_wsgi_exc():
     client = httpx.Client(app=raise_exc)
     with pytest.raises(ValueError):
         client.get("http://www.example.org/")
+
+
+def test_wsgi_generator():
+    output = [b"", b"", b"Some content", b" and more content"]
+    client = httpx.Client(app=application_factory(output))
+    response = client.get("http://www.example.org/")
+    assert response.status_code == 200
+    assert response.text == "Some content and more content"
+
+
+def test_wsgi_generator_empty():
+    output = [b"", b"", b"", b""]
+    client = httpx.Client(app=application_factory(output))
+    response = client.get("http://www.example.org/")
+    assert response.status_code == 200
+    assert response.text == ""