Requests (via oreos.monkeys) quotes cookie *values* -- browsers do not

I'm having an issue that has me questioning Requests's (Oreo's?) cookie handling.  Using Requests, I submit a request that successfully authenticates me to a particular server. The response includes a cookie whose _value_ contains an equals ("=") character.  Because the cookie's value contains "=", which is not a "legal" char (as per _LegalChars in monkeys.py), Requests quotes the entire cookie value when it submits it in subsequent requests.  The server, though, considers the quoted cookie to be different than the cookie it sent to the client -- so the client is no longer authenticated.

See this flow:

```
In [6]: session = requests.session(headers=_default_headers)

In [7]: data = {
   ...:     'userID': 'myuser',
   ...:     'password': 'mypassword',
   ...: }

In [8]: resp = session.post('https://www.discovercard.com/cardmembersvcs/loginlogout/app/signin', data=data)

In [9]: resp.headers
Out[9]: 
{'cache-control': 'no-cache="set-cookie, set-cookie2"',
 'connection': 'Keep-Alive',
<snip>
 'set-cookie': 'v1st=6B2AF394DCD43A57; path=/; expires=Wed, 19 Feb 2020 14:28:00 GMT; domain=.discovercard.com,
 dfsedskey=62689366; Expires=Sun, 17 Feb 2013 17:25:06 GMT; Path=/; Domain=.discovercard.com,
 PMData=PMV3EKKwYWpRB0ZAh7NnXgngf+fGUH7/c6RfKfa7Hiv5v2h1S1iS4MjexZtrnrPMErcNR0eyCzoWUfNFrj6bUkqxR0aQ==; Expires=Sun, 17 Feb 2013 17:25:06 GMT; Path=/; Domain=.discovercard.com; Secure,
 dcsession=E7dc4gmtmOk9LkTB4xeTjRs82g8=; Path=/; Domain=.discovercard.com; Secure,
 ACLOGIN=incntvtyp1=CBB; Expires=Sun, 17 Feb 2013 17:25:06 GMT; Path=/; Domain=.discovercard.com,
 ACHASH=1338528939; Expires=Sun, 17 Feb 2013 17:25:06 GMT; Path=/; Domain=.discovercard.com, dcuserid=***CLEARED***; Path=/; Domain=.discovercard.com,
 JSESSIONID=0001cT7y8RWiiQDuizJpy2gjQWw:16cfscb3m; Path=/',
<snip>
}
```

So the _value_ of the cookie with key "dcsession" is "E7dc4gmtmOk9LkTB4xeTjRs82g8=".  The cookies dict is as expected:

```
In [10]: session.cookies['dcsession']
Out[10]: 'E7dc4gmtmOk9LkTB4xeTjRs82g8='

In [11]: resp.cookies['dcsession']
Out[11]: 'E7dc4gmtmOk9LkTB4xeTjRs82g8='
```

But when I make my next request to the server:

```
In [12]: resp = session.get('https://www.discovercard.com/cardmembersvcs/achome/homepage')
```

The response to that request indicates "session timed out or not established".  This is because of how the cookie was sent in this request:

```
In [14]: resp.request.headers['Cookie']
Out[14]: 'ACHASH=1338528939;  ACLOGIN="incntvtyp1=CBB";  JSESSIONID="0001cT7y8RWiiQDuizJpy2gjQWw:16cfscb3m";
PMData="PMV3EKKwYWpRB0ZAh7NnXgngf+fGUH7/c6RfKfa7Hiv5v2h1S1iS4MjexZtrnrPMErcNR0eyCzoWUfNFrj6bUkqxR0aQ==";
dcsession="E7dc4gmtmOk9LkTB4xeTjRs82g8=";  dcuserid=***CLEARED***;  dfsedskey=62689366;  v1st=6B2AF394DCD43A57'
```

Because the server considers the quoted cookie being returned as being different from the non-quoted cookie it sent, the session is no longer authenticated.

The condition where the cookie comes back from the server containing the equals character is not rare or intermittent -- it always comes back like this, and so the authentication always fails.  This is not broken for people using browsers!  No browser I've tried (Chrome, Safari) quotes the cookie values, so there's no problem authenticating and carrying on a session.  Obviously the server in this case is an established financial institution, presumably with many millions of users, so I think that the way Requests/Oreos is handling cookies is a problem.

From what I can tell from the code, Requests instantiates a monkeys.SimpleCookie.  SimpleCookie decides that, since the value contains something other than what's in _LegalChars, it needs to quote the value.  I can patch this so that it works for me any number of ways: adding "=" into _LegalChars, or preventing SimpleCookie from ever quoting a value, or having Requests use BaseCookie rather than SimpleCookie, or having my application re-construct the cookie header from the cookies before my next request, etc.  But this does feel like a bug that should be fixed in the main branch for everybody.  I'm happy to contribute a fix but am interested to know what Kenneth and/or anybody else thinks is the right way to do it.


we can add `=` to the list of safe chars, as long as it doesn't have any negative implications. That's the whole reason oreos exists right now.


Yeah, that would definitely fix it for me. But I guess I'm confused as to why it's ever necessary/desired to quote a cookie's value. Why would we ever want a client to submit a cookie to the server that's not exactly what the server sent to the client?


The reason it's necessary/desired is that [RFC 2965](http://www.ietf.org/rfc/rfc2965.txt) requires that a cookie value either be a token, [which may not contain an '=' character](http://www.ietf.org/rfc/rfc2068.txt), or a quoted string. If the server isn't quoting the string, the server is non-compliant. As @kennethreitz says, Oreos already allows non-RFC-compliant characters in cookie keys because they're commonly sent out.


Well it doesn't seem to break any of the test suite if I add = to the _RFC2965Forbidden list. Oreos accepts them in the cookie key in _CookiePattern, so everything should be fine.


Let's do it.


This issue is a duplicate of #286


Thanks for the explanation and the fix!  Adding "=" into the LegalChars solves my problem just fine.

Upon further reading through the specs, though, I still think that request's handling/expectations of cookies isn't quite right.  [RFC2965](http://www.ietf.org/rfc/rfc2965.txt) is the spec for "new-style" cookies, but it acknowledges (in section 9) that "old-style" cookies may still be in use by some servers.  The spec for old-style cookies is the ["Netscape spec"](http://curl.haxx.se/rfc/cookie_spec.html).  New-style cookies are sent from a server via the **Set-Cookie2** header; RFC2965 specifies that their values can contain only the characters you specify (e.g., not "="), or else they need to be quoted.  Old-style cookies are sent from a server via the **Set-Cookie** header; the Netscape spec specifies that these cookies are _not_ quoted, and their value is "a sequence of characters excluding semi-colon, comma and white space".

Apparently RFC2965 and new-style cookies haven't been widely adopted.  And it appears that requests doesn't even check for the Set-Cookie2 header; it just takes the cookie from the Set-Cookie header.  So requests shouldn't expect that what it gets in the Set-Cookie header should be an RFC2965-compliant cookie, just a Netscape spec-compliant cookie.  And it should return this cookie back to the server unchanged (i.e., unquoted).  (And this would match with the way browsers seem to be handling these "=" containing cookies.)

Note further that [RFC6265](http://tools.ietf.org/html/rfc6265) obsoletes RFC2965 (eesh).  RFC6265 acknowledges that Set-Cookie2 cookies never caught on in practice.

[This StackOverflow thread](http://stackoverflow.com/questions/577191/what-is-the-current-state-of-the-cookie2-specification) is also helpful.  It states that most browsers still only support the initial Netscape spec.

So the cookie specs over time have been Netscape spec -> RFC2109 -> RFC2965 -> RFC6265, but it's really only the Netscape spec that's been widely adopted.


First, props for the time you clearly had to spend delving through IETF specs. Reading them usually makes me want to gouge my eyes out with a spoon, so thanks for doing the research for me. =D

So, my reading of RFC6265 is that valid cookie chars are any ASCII char excepting comma, colon, backslash, double-quote, control characters and whitespace, and any ASCII percent-encoding that does not give a comma, colon, backslash, double-quote or control characters. A valid cookie key is made up of these chars or a quoted version of them.

Does that agree with other people's reading of this spec?


I just looked at 6265 again, remembering [not to take the cork off the fork](http://www.youtube.com/watch?v=nqMc9B7uDV8), and yes, that's the way I read it too. :)


Hmm, is this resolved?

```
>>> import requests
>>> print requests.get('http://httpbin.org/headers', cookies={"this_is_a": "slash/"}).text
{
  "headers": {
    "Content-Length": "", 
    "Accept-Encoding": "identity, deflate, compress, gzip", 
    "Connection": "keep-alive", 
    "Accept": "*/*", 
    "User-Agent": "python-requests/0.12.0", 
    "Host": "httpbin.org", 
    "Cookie": "this_is_a=slash/", 
    "Content-Type": ""
  }
}
>>> print requests.get('http://httpbin.org/headers', cookies={"this_is_a": "equals=sign"}).text
{
  "headers": {
    "Content-Length": "", 
    "Accept-Encoding": "identity, deflate, compress, gzip", 
    "Connection": "keep-alive", 
    "Accept": "*/*", 
    "User-Agent": "python-requests/0.12.0", 
    "Host": "httpbin.org", 
    "Cookie": "this_is_a=equals=sign", 
    "Content-Type": ""
  }
}
```

```
>>> import requests
>>> c = requests.get('http://httpbin.org/headers', cookies={"this_is_a": "equals=sign"}).cookies
>>> c = requests.get('http://httpbin.org/cookies', cookies=c).cookies
>>> print c
<<class 'requests.cookies.RequestsCookieJar'>[<Cookie this_is_a=equals=sign for />]>
>>> print c['this_is_a']
equals=sign
```

I'm not sure if these test cases are relevant and right, though.


looks good to me ;)