Repository: encode/httpx

Issue:
Python 3.6 RuntimeError: read() called while another coroutine is already waiting for incoming data
Running in python 3.7 and 3.8 have been going beautifully, but now I've decided I wanted to add my API client using httpx to a Python 3.6 project, and I'm getting the above error whenever I try to call httpx. Below is the stack trace of one of the calls:

```py
src/emcee/api.py:119: in get_with_retry
    return self.session.get(path, params=params)
.nox/test-3-6/lib/python3.6/site-packages/httpx/client.py:818: in get
    proxies=proxies,
.nox/test-3-6/lib/python3.6/site-packages/httpx/client.py:729: in request
    proxies=proxies,
.nox/test-3-6/lib/python3.6/site-packages/httpx/client.py:760: in send
    async_response = concurrency_backend.run(coroutine, *args, **kwargs)
.nox/test-3-6/lib/python3.6/site-packages/httpx/concurrency/asyncio.py:241: in run
    return self.loop.run_until_complete(coroutine(*args, **kwargs))
../../../.pyenv/versions/3.6.9/lib/python3.6/asyncio/base_events.py:484: in run_until_complete
    return future.result()
.nox/test-3-6/lib/python3.6/site-packages/httpx/client.py:230: in _get_response
    return await get_response(request)
.nox/test-3-6/lib/python3.6/site-packages/httpx/middleware/redirect.py:31: in __call__
    response = await get_response(request)
.nox/test-3-6/lib/python3.6/site-packages/httpx/client.py:191: in get_response
    request, verify=verify, cert=cert, timeout=timeout
.nox/test-3-6/lib/python3.6/site-packages/httpx/dispatch/connection_pool.py:126: in send
    raise exc
.nox/test-3-6/lib/python3.6/site-packages/httpx/dispatch/connection_pool.py:121: in send
    request, verify=verify, cert=cert, timeout=timeout
.nox/test-3-6/lib/python3.6/site-packages/httpx/dispatch/connection.py:65: in send
    response = await self.h11_connection.send(request, timeout=timeout)
.nox/test-3-6/lib/python3.6/site-packages/httpx/dispatch/http11.py:53: in send
    http_version, status_code, headers = await self._receive_response(timeout)
.nox/test-3-6/lib/python3.6/site-packages/httpx/dispatch/http11.py:133: in _receive_response
    event = await self._receive_event(timeout)
.nox/test-3-6/lib/python3.6/site-packages/httpx/dispatch/http11.py:174: in _receive_event
    self.READ_NUM_BYTES, timeout, flag=self.timeout_flag
.nox/test-3-6/lib/python3.6/site-packages/httpx/concurrency/asyncio.py:79: in read
    data = await asyncio.wait_for(self.stream_reader.read(n), read_timeout)
../../../.pyenv/versions/3.6.9/lib/python3.6/asyncio/tasks.py:339: in wait_for
    return (yield from fut)
../../../.pyenv/versions/3.6.9/lib/python3.6/asyncio/streams.py:634: in read
    yield from self._wait_for_data('read')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <StreamReader t=<asyncio.sslproto._SSLProtocolTransport object at 0x10b508470>>, func_name = 'read'

    @coroutine
    def _wait_for_data(self, func_name):
        """Wait until feed_data() or feed_eof() is called.

        If stream was paused, automatically resume it.
        """
        # StreamReader uses a future to link the protocol feed_data() method
        # to a read coroutine. Running two read coroutines at the same time
        # would have an unexpected behaviour. It would not possible to know
        # which coroutine would get the next data.
        if self._waiter is not None:
            raise RuntimeError('%s() called while another coroutine is '
>                              'already waiting for incoming data' % func_name)
E           RuntimeError: read() called while another coroutine is already waiting for incoming data

../../../.pyenv/versions/3.6.9/lib/python3.6/asyncio/streams.py:452: RuntimeError
```

Python: 3.6.9
httpx: 0.7.3

It appears to be with the way I've instantiated TimeoutConfig (with no values, to allow infinite timeout). Here's a minimally reproducible example:

```py
>>> import httpx

>>> no_timeout = httpx.config.TimeoutConfig()
>>> session = httpx.Client(timeout=no_timeout)
>>> session.get("https://httpbin.org")
<Response [200 OK]>
```

Works fine in 3.7+, fails in 3.6


Discussion:
Thanks for reporting this :)

Could be that in 3.6 asyncio’s read() only allowed one coroutine to call it at a time, while that constraint was relieved in 3.7?

Trio has a similar constraint on the write operation, and I remember we solved it using a simple lock.

Do you get this behavior with the usual timeout config?
I do not. Further testing giving no timeout value to httpx.Client i.e.
```
session = httpx.Client()
```
or giving a concrete timeout value i.e.
```
timeout = httpx.config.TimeoutConfig(timeout=300)
session = httpx.Client(timeout=timeout)
```
both proceed as expected, with no RuntimeError in py3.6 or others.
I did see something about using an asyncio lock in my searches... https://stackoverflow.com/a/25799871
My question is why would more than one .read() coroutine call be active at any one time?

Resolving patch:
diff --git a/httpx/concurrency/asyncio.py b/httpx/concurrency/asyncio.py
--- a/httpx/concurrency/asyncio.py
+++ b/httpx/concurrency/asyncio.py
@@ -81,6 +81,12 @@ async def read(
             except asyncio.TimeoutError:
                 if should_raise:
                     raise ReadTimeout() from None
+                # FIX(py3.6): yield control back to the event loop to give it a chance
+                # to cancel `.read(n)` before we retry.
+                # This prevents concurrent `.read()` calls, which asyncio
+                # doesn't seem to allow on 3.6.
+                # See: https://github.com/encode/httpx/issues/382
+                await asyncio.sleep(0)
 
         return data
 


Test patch:
diff --git a/tests/test_timeouts.py b/tests/test_timeouts.py
--- a/tests/test_timeouts.py
+++ b/tests/test_timeouts.py
@@ -2,6 +2,7 @@
 
 from httpx import (
     AsyncClient,
+    Client,
     ConnectTimeout,
     PoolLimits,
     PoolTimeout,
@@ -47,3 +48,13 @@ async def test_pool_timeout(server, backend):
             await client.get("http://localhost:8000/")
 
         await response.read()
+
+
+def test_sync_infinite_timeout(server):
+    """Regression test for a bug that occurred under Python 3.6.
+
+    See: https://github.com/encode/httpx/issues/382
+    """
+    no_timeout = TimeoutConfig()
+    with Client(timeout=no_timeout) as client:
+        client.get(server.url.copy_with(path="/slow_response/50"))