iter_lines method will always hold the last response in the server in a buffer

The implementation of the `iter_lines` and `iter_content` methods in `requests` means that when receiving line-by-line data from a server in "push" mode, the latest line received from the server will almost invariably be smaller than the `chunk_size` parameter, causing the final read operation to block.

 A good example of this is the Kubernetes `watch` api, which produces one line of JSON output per event, like this:

```
{"type":"ADDED","object":{"kind":"Service","id":"kubernetes-ro","uid":"5718d954-a31a-11e4-8c74-20cf30467e62","creationTimestamp":"2015-01-23T11:10:25-05:00","selfLink":"/api/v1beta1/services/kubernetes-ro","resourceVersion":4,"apiVersion":"v1beta1","namespace":"default","port":80,"protocol":"TCP","labels":{"component":"apiserver","provider":"kubernetes"},"selector":null,"containerPort":0,"portalIP":"10.254.6.100"}}
```

If you compare the output of:

```
import requests
r = requests.get('http://localhost:8080/api/v1beta1/watch/services',
                 stream=True)

for line in r.iter_lines():
    print line
```

With the output of `curl` running against the same URL, you will see
that the output from the Python code lags behind the output seen by
`curl` by one line.

I was able to work around this behavior by writing my own `iter_lines`
method, which looks like this:

```
def iter_lines(fd, chunk_size=1024):
    '''Iterates over the content of a file-like object line-by-line.'''

    pending = None

    while True:
        chunk = os.read(fd.fileno(), chunk_size)
        if not chunk:
            break

        if pending is not None:
            chunk = pending + chunk
            pending = None

        lines = chunk.splitlines()

        if lines and lines[-1]:
            pending = lines.pop()

        for line in lines:
            yield line

    if pending:
        yield(pending)
```

This works around the problem partly by calling `os.read`, which will
happily return fewer bytes than requested in `chunk_size`.  With the
above routing available, the following code behaves correctly:

```
import requests
r = requests.get('http://localhost:8080/api/v1beta1/watch/services',
                 stream=True)

for line in iter_lines(r.raw):
    print line
```

This code will always print out the most recent reply from the server
when it is received.


Generally speaking I'd be in favour of changing this behaviour. It's been stupid for a long time now.


You also have my support. The trick is doing this in a way that's backwards compatible so we can help you out before 3.0


The above change works for me with python 2.7.8 and 3.4.1 (both with `urllib3` available).


I've just encountered this unfortunate behavior trying to consume a feed=continuous changes feed from couchdb which has much the same semantics. Any chance of this going in?


note that this doesn't seem to work if you don't have urllib3 installed and using r.raw means requests emits the raw chunks of the chunked transfer mode.


> note that this doesn't seem to work if you don't have urllib3 installed and using r.raw means requests emits the raw chunks of the chunked transfer mode.

I'm sorry. I don't understand that at all. Could you help me understand? If you're using requests from PyPI, you always have urllib3 installed as `requests.packages.urllib3`. Are you using requests from one of the distribution packages without urllib3 installed? If so, how is requests even working?


an excellent question but likely off-topic (I only noticed that 'pip install urllib3' installed the library, and then I uninstalled it, but of course I probably have another copy somewhere else).

The bug in iter_lines is real and affects at least two use cases, so great to see it destined for 3.0, thanks :)


I am pretty sure we've seen another instance of this bug [in the wild](http://stackoverflow.com/questions/39662596/python-requests-get-ignores-the-last-record).


Is this really still a bug? At the very least this should be well documented -- I would imagine most people would just not use `iter_lines` if they knew about this

> in a way that's backwards compatible

@sigmavirus24 I'm having trouble understanding that. It's a bug, right? It's not intended behavior that's being broken, it's fixing it to work as intended.

@eschwartz I'm no longer involved in this project. Please don't mention me on this or other issues.