toastdriven/pysolr — pull request #100 by megsa (2013-11-04 23:46:58+00:00)
Title: updating required requests version in README

Older versions of requests convert header values, but not keys, to strings. Compare the old prepare_headers to the newer version of prepare_headers:

Requests: Pre current version:
```
def prepare_headers(self, headers):
"""change to list"""

if headers:
self.headers = CaseInsensitiveDict(self.headers)
else:
self.headers = CaseInsensitiveDict()
```

Requests: Current version:
```
def prepare_headers(self, headers):
"""Prepares the given HTTP headers."""

if headers:
self.headers = CaseInsensitiveDict((to_native_string(name), value) for name, value in headers.items())
else:
self.headers = CaseInsensitiveDict()
```

Header values are later converted to strings in httplib/http.client as follows.
In ```_send_request()``` the headers are iterated over:
```
for hdr, value in headers.iteritems():
self.putheader(hdr, value)
```
and in ```putheader()``` the values are converted to strings:
```
def putheader(self, header, *values):
.
.
.
hdr = '%s: %s' % (header, '\r\n\t'.join([str(v) for v in values]))
self._output(hdr)
```
where ```self._output()``` adds the headers to ```self._buffer```

The problem arises when this gets to httplib (http.client in 3.x).
The following is called in ```_send_output()```:
```
self._buffer.extend(("", ""))
msg = "\r\n".join(self._buffer)
.
.
.
if isinstance(message_body, str):
msg += message_body
```
Since pysolr is no longer forcing headers to bytes, unicode header keys and values are being passed over. With pysolr and the current version of requests, msg will be a bytestring of headers, whereas with pysolr and the older versions it will be unicode. When this tries to concatenate with the bytestring message_body there are sometimes decoding issues.

Updating the current version of requests ensures that headers will always be bytestrings and that httplib/http.client will be able to properly handle the request, even though pysolr is now passing over unicode.