iter_content() not actually reading/raising exception

I've the following code ('stream' set to True):

``` python
try:
  for chunk in r.iter_content(chunk_size=512 * 1024): 
    if chunk: # filter out keep-alive new chunks
      f_temp.write(chunk)
```

In case of network issue, I was expecting an exception at the 'iter_content' line, but got the following stack trace:

``` python
Traceback (most recent call last):
  File "/usr/lib/python3.5/site-packages/urllib3/response.py", line 232, in _error_catcher
    yield
  File "/usr/lib/python3.5/site-packages/urllib3/response.py", line 314, in read
    data = self._fp.read(amt)
  File "/usr/lib/python3.5/http/client.py", line 448, in read
    n = self.readinto(b)
  File "/usr/lib/python3.5/http/client.py", line 488, in readinto
    n = self.fp.readinto(b)
  File "/usr/lib/python3.5/socket.py", line 575, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

Traceback (most recent call last):
  File "/usr/lib/python3.5/site-packages/requests/models.py", line 664, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "/usr/lib/python3.5/site-packages/urllib3/response.py", line 357, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/lib/python3.5/site-packages/urllib3/response.py", line 324, in read
    flush_decoder = True
  File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python3.5/site-packages/urllib3/response.py", line 237, in _error_catcher
    raise ReadTimeoutError(self._pool, None, 'Read timed out.')
requests.packages.urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='vpn.media.shagma.net', port=8888): Read timed out.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<my file>", line 434, in <my function>
    f_temp.write(chunk)
  File "/usr/lib/python3.5/site-packages/requests/models.py", line 671, in generate
    raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='<my proxy address>', port=8888): Read timed out.
```

=> exception is raised during .write() command, not iter_content() ? Took me a while to figure out... Is it a bug ?

(python 3.5.2, requests 2.10, archlinux)

side note: I just read the doc and chunk_size=None may be a better option, but should not be related to my issue, I think... :)


Without knowing what `f_temp` is or being able to see how you're using requests, I don't think we can definitely say this is a bug. That said, it is very strange that you're seeing that in your call to `write`. The chunk should just be a bytes object, nothing fancy.


The exception is not coming in the call to `write`. The original exception is coming in the call to `stream`. The problem is that exception tracebacks can be somewhat misleading when using generators, which you are here.

The `iter_content` method _returns_ a generator, which is where the exception is actually thrown from. Because of the way `iter_content` is structured, it actually has an inner function (called `generate`) that is called to get the returned generator. That means that, when Python attempts to reconstruct the traceback, it only gets to `generate`: nothing is holding on to the original `iter_content` call anymore.

Everything is working as expected. =)


@Lukasa Not sure to understand but if you say so... :)
That means that the traceback is wrong and the following should work (it quite hard to test here) ?

``` python
try:
  for chunk in r.iter_content(chunk_size=512 * 1024): 
    if chunk: # filter out keep-alive new chunks
      try:
        f_temp.write(chunk)
      except:
        # error during write (disk full, whatever...)
except requests.exceptions.ConnectionError:
  # error during read
```


@dvasseur Yes. =)


@Lukasa ok, thanks!