Can't modify cookies from auth plugin

HI,

I'm writing an auth plugin in which I want to manipulate the cookies set by a response. Specifically in this case I need to use cookies for the auth exchange but i don't want them added to the session. When the response hooks and similar are finished the response is returned, the history examined and cookies merged into the cookie jar using the _original_response.msg httpmessage that urllib3 sets. This means there is no public way for me to manipulate the headers to prevent these cookies being added to the session. 

From a quick examination my guess to address this would be to regenerate the httpmessage header list from the response.headers. 

My solution for now is going to be not include the intermediate requests in the history so that the cookies don't get picked up. I'm also going to look into a cookie policy that simply denies everything.


I'm sorry, it's not clear what exactly the bug is here. What function would you like that you don't have?


If I know @jamielennox, they're working on SAML auth plugin. Part of (some of) the SAML authentication flow(s) includes cookies being sent that aren't meant to be persisted on the session.

I think what @jamielennox wants is for those cookies that are meant to be part of an intermediate step in authentication not to be persisted. After a series of redirects, we process everything and update the Session cookie jar with the cookies from individual responses in the history. I suspect @jamielennox was removing the cookies individually from the intermediate responses hoping that would prevent them from being persisted on the Session but that isn't working because we don't use those attributes when extracting cookies to a session cookie jar. (Further, this is likely a plugin for OpenStack.)

I don't think there is necessarily a bug here. We're doing the right thing for the 99% case.

@jamielennox I suspect the cookie names are predictable and you could have a CookieJar that ignores those cookie values (along the lines of a far more selective ForgetfulCookieJar which @ceaess is adding to the requests-toolbelt).


Right, sorry i was trying to make the issue a bit more generic than my case. Requests is absolutely doing the right thing in 99% of cases. We have the issue that we are in some cases reusing sessions between different authenticated flows. This hasn't been an issue in the past because we are doing our own token based authentication against REST apis so we just haven't had to contend with cookies. 

Essentially my case is that i don't want these cookies that are part of the auth flow to end up in the general session as then they are available across user flows.

My hope here was that i could develop a generic SAML plugin that we could contribute back to requests but then subclass it from an openstack perspective to remove those cookies. However there was no was no way that i could strip these cookies. If i cleared the cookie jar or removed all the Set-Cookie headers from response.raw.getheaders or response.headers the cookies were still persisted because the CookieJar is looking directly at the _original_message  (https://github.com/kennethreitz/requests/blob/dd0f164f8ed8e6f936c8af4442f6ddfa4d302fd5/requests/cookies.py#L122) which is a private variable that i cannot (should not) modify from the plugin. 

I was intending to say on that initial report that i don't know if my case is worth supporting. And i'm going to look into either a disable all cookie jar policy or just doing my own no-op cookie jar for our case. I just thought i would let you know it was a problem i hit because i doubt anyone has seen it before.

I will have a look into ForgetfulCookieJar as well. 

Thanks


Hmm. Just to clarify my own understanding, the core problem here is that you are acting as a middleman between some set of users on one side and a set of REST-accessible resources on the other. You are handling requests on behalf of some set of users _n_ with some set of sessions _m_ where _m_<_n_, which means that each session may have _n_/_m_ authenticated users running through it. The problem you have there is that you don't want your auth cookies to end up in the Session, lest they accidentally extend their auth scope to one of the other users. Is that about right?

Assuming it is (I'm aware our timezone overlap is bad, so I'm going to make that assumption about your answer to give more information: shout if my assumption is wrong), the actual problem you have is cookies in general. Sessions keep all cookies in one place, which means there's no way of keeping cookies that semantically "belong" to different users apart. This is clearly a problem for the SAML auth flow cookies, but is also a wider problem: if you were using a REST API that happened to set cookies, you'd be unable to keep those isolated either.

For that reason, you either want one Session per user, or you want to disallow cookies altogether. The first is more generally correct at the cost of being _substantially_ more resource intensive, the other potentially has a weird failure mode (refusing to save a cookie that a service needs) but avoids the resource intensity.


@jamielennox I'm currently writing tests for the ForgetfulCookieJar and I'm confident it should be available in requests-toolbelt in the next week or so.


@Lukasa Yes that's all correct, it's just that the SAML case is the first time i've run across cookies being used at all in OpenStack. I need to look into a way of managing cookies seperate to what the requests.Session is doing. For almost all cases what i need now is just drop all cookies.


@jamielennox so you can lift the couple lines of code that constitute the [ForgetfulCookieJar](https://github.com/sigmavirus24/requests-toolbelt/blob/master/requests_toolbelt/cookies/forgetful.py#L5) or you can wait for/help with requests-toolbelt 0.5 so you can add it to OpenStack.