iter_lines() sometimes yields broken results when chunk_size is small

Here is a small test program to demonstrate the bug:
```
import requests

url = "http://lohas.pixnet.net/blog"
r = requests.get(url)
iter_lines = [line for line in r.iter_lines(chunk_size=7, decode_unicode=False)]
split_lines = r.content.splitlines()
for index, (iline, sline) in enumerate(zip(iter_lines, split_lines)):
	if iline != sline:
		print("line {} is broken".format(index))
		print(iline, len(iline))
		print(sline, len(sline))
		break
```

Expected behavior:
r.iter_lines() should give the same result as r.content.splitlines()

Actual behavior:
The test program generates the following output:
```
line 275 is broken
b'' 0
b'\t\t\t<tbody>' 10
```

Changing chunk_size can break different lines.


It's not caused by incorrect unicode decoding.
I tested this with both `decode_unicode=True` and `decode_unicode=False`. In both cases the bug can be reproduced if chunk_size is small enough.

I currently use the following code snip as a drop-in replacement.
```
def my_iter_lines(result):
            buf = b''
            for chunk in result.iter_content(chunk_size=64 * 1024, decode_unicode=False):
                buf += chunk
                pos = 0
                while True:
                    eol = buf.find(b'\n', pos)
                    if eol != -1:
                        yield buf[pos:eol]
                        pos = eol + 1
                    else:
                        buf = buf[pos:]
                        break
            if buf:
                yield buf
```


This is not a breakage, it's entirely expected behaviour. `iter_lines` takes a `chunk_size` argument that limits the size of the chunk it will return, which means it will occasionally yield before a line delimiter is reached. This is the behaviour `iter_lines` has always had and is expected to have by the vast majority of requests users.

To avoid this issue, you can set the chunk_size to be very large indeed. Alternatively, you'll need to process it by wrapping the `read` method on `response.raw` yourself and handling the line splitting using the IO module.

@Lukasa no this is not. See the API doc.
```
iter_lines(chunk_size=512, decode_unicode=None, delimiter=None)[source]
Iterates over the response data, one line at a time. When stream=True is set on the request, this avoids reading the content at once into memory for large responses.
```
chunk_size is not used to limit the max line size. It's pass into iter_content to limit the buffer size used to read the stream. In the source code of iter_lines(), you can see it actually tries to concatenate the data from different chunks rather than limiting max line size.
FYI: http://docs.python-requests.org/en/master/_modules/requests/models/#Response.iter_lines
See the `pending` stuff.

Sorry, you're quite right. That's what I get for working while jetlagged.

My guess is that this is likely a duplicate of #2431, and so would have been fixed by #3923. That fix was landed into the 3.0 branch because it unfortunately does break compatibility. However, it should be fixed in the future 3.0 release. Would you like to investigate by trying the 3.0 branch with your example?

@Lukasa This looks like a duplicate of #2431 indeed. However, I just tested the 3.0 branch and the above problem still exists. I'll look into the 3.0 branch so see what I can do here.

Provide another simpler test:
```
import requests

url = "http://lohas.pixnet.net/blog"
r = requests.get(url)
iter_lines = r.iter_lines(chunk_size=7, decode_unicode=False, delimiter=b'\n')
split_lines = r.content.split(b'\n')
for index, (iline, sline) in enumerate(zip(iter_lines, split_lines)):
	if iline != sline:
		print("line {} is broken".format(index))
		print(iline, len(iline))
		print(sline, len(sline))
		break
```
Even if I call split() and specify delimiter b'\n' explicitly, their results remain different.
```
[pcman@pcman-arch-laptop requests]$ python test_requests.py 
line 281 is broken
b'' 0
b'     style="display:inline-block;width:300px;height:250px"' 58
```

My bad. Tested the wrong code. Specifying b'\n' as delimiter for iter_lines() and split() works.
However, the test case with delimiter=None still yield different result from that of content.splitlines().


Yup, that inconsistency seems to be present still. Let me dig into what exactly is going on here.

Ah, yes, I can see the problem. It boils down to this:

```python
>>> 'test\r\n'.splitlines()
['test']
>>> 'test\r'.splitlines()
['test']
```

Basically, the code as written has an implicit assumption that there will only be UNIX-style newline characters in the document, but that's not guaranteed to be true. `splitlines` handles `\r\n` equivalently to `\n`. That means that if a chunk ends up looking like `this\r`, our code will assume that this was a complete line and eat the carriage return. If the first character of the next chunk is a `\n` then the code will have *forgotten* that the last character was a `\r` and so will treat it as a *new* line terminating character, and so will emit a bonus blank line.

This is a moderately tricky thing to handle. Most of the simplistic solutions have nasty side effects: for example, we cannot attempt to over-read because that will in some cases cause us to block and prevent us emitting a line we do actually have. We cannot simply try to memorize what kind of line terminator we have, partly because it will expose us to this failure mode on the *first* line and partly because line termination characters can be mixed throughout a document.

I think there are two approaches, in the end. The first is to fall back to using `split` on `\r` and `\n`, and then to manually transform to universal newlines (that is, to spot empty lines and check what's going on). This is strictly more complex than the solution I'd actually propose, which is to just remember when we have treated `\r` as a final line terminator.

Basically, we'd update the code to add a check when using splitlines:

```python
lines = chunk.splitlines()
last_split_on_cr = lines[-1] and lines[-1] == '\r'
```

Then, when we come through the loop again, if `last_split_on_cr` is `True` we'd check whether the *first* character of the next line is a line feed and, if it is, we'd strip it off.

```python
if lines[0] and lines[0][0] == '\n':
    lines[0] = lines[0][1:]
```

I believe this would resolve the issue.

Are you interested in supplying the necessary PR @PCMan, along with an appropriate test?

@Lukasa Yes, I'm interested in digging into the issue. I'll see if I can provide a patch. :)

@Lukasa Here is the pull request fixing the issue.
https://github.com/kennethreitz/requests/pull/3984
Thanks for your review :-)

This should be resolved in 3.0.0 with #3984. Thanks again @PCMan!

This bug is still present in the latest release. Are there plans to release the fix soon?