kennethreitz/requests — pull request #1286 by Lukasa (2013-04-03 19:33:14+00:00)
Title: Consistent String Encoding

Over the last few months there have been multiple issues raised about problems with string encodings. At best Requests is being inconsistent (#1250), and at worst perfectly valid combinations of input are failing (#1279).

I think it's time for Requests to enforce a very consistent behaviour when it comes to string encodings. We need to establish what format the data will be in when we pass it to urllib3, and where we're going to transform it. We need to be consistent so that when users inevitably encounter errors, we can make it very clear what they should have passed in.

@shazow has said that urllib3 wants the following input: bytes in the body, and native strings everywhere else. This defines our interface to urllib3.

To keep in sync with the above, we need to do the following things.

- On Python 2, any unicode strings anywhere on a `PreparedRequest` other than the body should be encoded as UTF-8. The body should also be encoded as UTF-8 if it is a unicode string (unlikely, but unless we can guarantee it won't be we should do the right thing).
- On Python 3, any bytestrings in a PreparedRequest should be decoded to unicode strings. Because we cannot make any intelligent guess, we will use Python's (stupid, stupid, _stupid_) locale-based decoding. This will almost-certainly throw exceptions if an incorrect encoding has been used, alerting the user.

Thoughts?