Header capitalization causes problems with some web servers

As raised here:
http://bugs.python.org/issue12455

Although your documentation states:
"The dictionary is special, though: it’s made just for HTTP headers. According to RFC 2616 , HTTP Headers are case-insensitive"

It should be noted that many web servers do not conform to this standard, and enforcing it makes some requests break.

Therefore, I'd like to request a patch on this library which allows the user to disable the automatic header capitalization (i.e. just keep the headers as it received them).

Cal


Also note, this is affecting cookies (name/value) as well.


Headers should be sent upstream untouched. If they aren't currently, there's a bug.


I'm making a patch for this now, along with full debug, will send shortly.


Thanks a ton!


Only queries should be case-insensitive. 


Okay, seems to be unrelated to the original python bug I pasted, but raises a totally new bug:

BROKEN (using oreos.monkeys.SimpleCookie) - line 471.

```

{'Accept-Encoding': 'identity, deflate, compress, gzip', 'Accept': '*/*', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1', 'Cookie': 'LOB=Auto;  NIAX_JSESSIONID="0001ZjlThXjY9q7bIws1je8WMFh:14a8mcptb";  TLTSID=B631D51A4AC3104AC60FC1A310F28ADB;  siteserver=b632b571-4ac3-11e1-8776-7b5f576f558c;  state=PA', 'Referer': 'https://buyonline.nationwide.com/autoquote/initiateNewQuote.do', 'Content-Type': 'application/x-www-form-urlencoded'}

```

WORKING:

```

{'Accept-Encoding': 'identity, deflate, compress, gzip', 'Accept': '*/*', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1', 'Cookie': 'LOB=Auto;  NIAX_JSESSIONID=0001ZjlThXjY9q7bIws1je8WMFh:14a8mcptb;  TLTSID=B631D51A4AC3104AC60FC1A310F28ADB;  siteserver=b632b571-4ac3-11e1-8776-7b5f576f558c;  state=PA;', 'Referer': 'https://buyonline.nationwide.com/autoquote/initiateNewQuote.do', 'Content-Type': 'application/x-www-form-urlencoded'}

```

Notice how the cookie 'NIAX_JSESSIONID' has additional quoting around its value, where as browsers would not do this (and as such, it is causing the web server to throw a 5XX)

It seems to stem from this method:

```

def _quote(str, LegalChars=_LegalChars,
           idmap=_idmap, translate=string.translate):
    #
    # If the string does not need to be double-quoted,
    # then just return the string.  Otherwise, surround
    # the string in doublequotes and precede quote (with a \)
    # special characters.
    #
    if "" == translate(str, idmap, LegalChars):
        return str
    else:
        return '"' + _nulljoin( map(_Translator.get, str, str) ) + '"'
# end _quote

```

There is a whole set of rules in the comments above there, but suffice to say, the browsers I have tested this in don't act in the same manner.

Any thoughts??

Cal


Also, I just tried this with Python built in 'Cookie' lib, and the same problem happens there also:

```

            cookies = {
                'siteserver' : 'b632b571-4ac3-11e1-8776-7b5f576f558c',
                'TLTSID' : 'B631D51A4AC3104AC60FC1A310F28ADB',
                'NIAX_JSESSIONID' : '0001ZjlThXjY9q7bIws1je8WMFh:14a8mcptb',
                'LOB' : 'Auto',
                'state' : 'PA'
            }

            from Cookie import SimpleCookie
            _c = SimpleCookie(cookies)
            print _c.output()

```

```

Set-Cookie: LOB=Auto
Set-Cookie: NIAX_JSESSIONID="0001ZjlThXjY9q7bIws1je8WMFh:14a8mcptb"
Set-Cookie: TLTSID=B631D51A4AC3104AC60FC1A310F28ADB
Set-Cookie: siteserver=b632b571-4ac3-11e1-8776-7b5f576f558c
Set-Cookie: state=PA

```

But given that oreo SimpleCookie is forked from 'Cookie', even if it was patched in python core (which would take months), it would then need to filter down the chain, so we're looking at maybe 6-12 months until the bug was fixed.

Given this situation, would you be willing to patch in requests lib, or would you hold back until it was patched in the core (if).

Cal


Oh, and the specific reason why it's failing in my case is:

```

LegalChars = string.ascii_letters + string.digits + "!#$%&'*+-.^_`|~[]_"

```

My cookie value contains ':' which isn't in the LegalChars - however adding quotes around the value seems to break the app, so I'm wondering if the most appropriate monkey patch would be to just remove the LegalChars.. (but then what happens if someone includes ';' in their cookies. argh).


Well, I've put in a fix which (for the time being) fixed our bug, but it is alpha as fuck and may cause a whole bunch of other problems. Hopefully this will be useful to anyone else that stumbles across this problem.

```

#!/usr/bin/env python
import requests
def _quote(val, *args, **kwargs):
    print "WARNING: cookie value has been monkey patched"
    return val

requests.packages.oreos.monkeys._quote= _quote

```

Cal Leeming
Simplicity Media Ltd
cal.leeming@simplicitymedialtd.co.uk


Should be fixed now.