[WIP] [2.0] Header keys should be native strings.

_Note_: This PR follows a discussion on IRC. The logs can be found [here](https://botbot.me/freenode/python-requests/msg/2942059/).
### Summary

This fixes a problem that @https://github.com/va1en0k _would_ have had in #1335 if the current Case-Insensitive Dict didn't behave so weirdly (see #1333 for progress on that).

Right now, Requests unconditionally encodes all header values as bytes on all Python versions. Kenneth and I are (tentatively) in agreement that header values should actually be native strings on all platforms. This pull request introduces code that explicitly encodes or decodes string objects to satisfy that requirement.
### Notes

Firstly, **this is a breaking API change on Python 3**. On Python 2 the behaviour will be the same unless the user has changed `sys.defaultencoding`, and if they have they deserve what's coming to them. On Python 3, all header keys on `PreparedRequest`s will go from being bytes to unicode objects, and so anyone who has correctly programmed to this interface will have to change their code.

Secondly, this code currently uses ASCII as the codec in both directions. I think this is OK for now. I'm pretty sure that header values should actually be Latin-1, but I can understand if we want to keep the current behaviour in place. I'm open to feedback here.


Ref #1339, completed CaseInsensitiveDict rewrite.


CID merged in e6e9b55.


@Lukasa rebase ;)


Done. =)


@Lukasa the tests fail on python3 where you have `'accept'.encode('ascii')` because of the new CaseInsensitiveDict (#1339)


That should resolve it. =D I wish Travis still emailed me when I'd broken the build.


Yeah that fixes it. Travis just has your 2.6 build queued so @kennethreitz might be wary until it completes (not that he need be).


@cdunklau That was a great idea, and is now done.


What's the status here?


I believe the status is :cake: but I'm not 100% sure.


The status is :cake:, unless there's something more you want here. In which case, I am your humble servant. =)


Merge please.


@oliverjanik: The reason this wasn't merged straight away is that it is a significant, backward-incompatible change. It is generally unwise to merge such a change immediately. It's important to add it in a release which gets a minor version bump (e.g. 1.2.x to 1.3.x), and with good documentation (which reminds me, I should document this change somewhere).

This, or something similar, will get merged in good time. =)


Rebased again, I'll close and reopen to get GitHub to realise it.


Let's pretend I didn't push up a merge conflict. =P


What's the status here?


The answer is that this _should_ be fine. It's a breaking API change, so if you want to sit on it for a while that would be totally reasonable.


I vote for merging, because this bug makes requests behave differently on Python2 and Python3 and there is no reasonable work-around. 

While it's true that it breaks public API it breaks undocumented behaviour, which nobody should rely on.


@oliverjanik If only it were that simple. =) This will affect the behaviour of the headers dictionary. People will have spotted that Requests sets its headers to bytes on Python3 and have written code that takes that into account. That code will break.

We simply can't rush this into a Requests release. With that said, this (or something like it) will definitely end up in Requests eventually. =)


Under Python3 with requests 1.2.3, header overrides are currently broken.  That is, if I specify a header that requests would normally specify itself (such as Accept-Encoding or Content-Type), then it simply adds my header instead of replacing it.  

Approximate script I used to test:

approximate script:

```
host = 'localhost'
port = 443
protocol = 'https'
data = 'mystring'

session = requests.Session()

method = 'POST'
path = '/my/web/service'
headers = {'x-custom-header1': 'x',
           'x-custom-header2': data,
           'x-custom-header3': 'x',
           'User-Agent': 'myagent',
           'Content-Type': 'text/xml',
           'x-custom-header4': '12345'}
url = "%s://%s:%d%s" % (protocol,host,port,path)
body = (b'<myxml></myxml>\n')

return session.request(method, url, headers=headers, data=body,
                       cert=('cert.pem', 'key.pem'),
                       verify=False)
```

HTTP request generated:

  POST /my/web/service HTTP/1.1
  Host: localhost
  Accept-Encoding: identity
  Content-Type: application/x-www-form-urlencoded
  Content-Length: 64
  Accept-Encoding: gzip, deflate, compress
  x-custom-header1: x
  x-custom-header2: mystring
  x-custom-header3: x
  Accept: _/_
  User-Agent: myagent
  Content-Type: text/xml
  x-custom-header4: 12345

  <myxml></myxml>

Note the two content types.  This same script in python 2 behaved correctly.


@adian0: Indeed. Try replacing the `'User-Agent'` and `'Content-Type'` header keys with `b'User-Agent'` and `b'Content-Type'`. If that works, then the problem is one that will be fixed by this pull request.


No, already I tried using bytes for those.  It threw this exception:
...
 File "/usr/local/lib/python3.2/dist-packages/requests/models.py", line 340, in <genexpr>
    headers = dict((name.encode('ascii'), value) for name, value in headers.items())
AttributeError: 'bytes' object has no attribute 'encode'


The cause of the behavior reported by @adian0 seems to be the same I described in #1250.


@pmcnr Agreed, and this PR should resolve it. =)


@Lukasa It's a shame this PR is a breaking API change in Python 3. I've really been looking forward to the next major release of requests when you can finally include it. A big thank you for the excellent work!


#1459 


Closing since this is merged into #1509