Separate password management from authentication managers

In urllib2, the semantic concept of password management is separate from the authentication mechanisms. This means that one can implement password management (resolution of a username and password in a realm) independent of authentication mechanisms (basic, digest, etc).

With requests, it's not clear how one could do this if it's even possible. In particular, I'm working on a library that allows resolving a password late and only after the server has prompted for it. From the documentation and reading the code, it appears that requests requires that the authentication be known before the initial request is made, or for digest auth, it can be resolved later.

For example, see [this code](https://bitbucket.org/jaraco/lpaste/src/88a8a13b7ba79651102fb3937220a8eddbd9b6c4/lpaste/lpaste.py?at=default#cl-129) where the password manager is set up independently of the auth mechanism, and where the keyring-based implementation is only invoked if authentication is required.

I'd like to port that code to requests, but it seems to do that, I'd have to write a custom AuthBase, borrow the handle_401 from DigestAuth, borrow the password encoding from the basic auth class as well, plus customize it to the library's specific needs.

What hooks/options does requests provide that might help facilitate this need? Should requests be extended here to support these sorts of use cases? Should requests follow the same separation of responsibilities as urllib2 does, or is there a better approach in the spirit of requests?


Hi @jaraco, thanks for opening this issue! =)

I'll begin by addressing your actual problem. Simply put, the easiest way to resolve the password after the server has prompted for it is to not attach an auth handler to the original request, then check the response code:

``` python
r = requests.get('http://authpage.com/')

if r.status_code == 401:
    user, pass = prompt_for_auth()
    r = requests.get('http://authpage.com/', auth=(user, pass))
```

If you feel as though this has too much boilerplate for you (which it would do if you do this a lot), take a look at what the Digest Auth handler does to handle 401 responses. You can easily write a Basic Auth handler that exhibits the same behaviour:

``` python
from requests.auth import AuthBase, _basic_auth_str

class DelayedBasicAuth(AuthBase):
    """
    A class that implements HTTP Basic Authentication by waiting for a 401
    response and then attaching the auth data.
    """
    def __init__(self):
        pass

    def __call__(self, r):
        r.register_hook('response', self.handle_401)
        return r

    def credentials(self):
        if not (self.username and self.password):
            self.username, self.password = prompt_for_auth()

        return self.username, self.password

    def handle_401(self, r, **kwargs):
        num_401_calls = getattr(self, 'num_401_calls', 1)
        auth_hdr = r.headers.get('www-authenticate', '')

        if 'basic' in auth_hdr.lower() and num_401_calls < 2:
            setattr(self, 'num_401_calls', num_401_calls + 1)

            r.content
            r.raw.release_conn()

            user, pword = self.credentials()

            r.request.headers['Authorization'] = _basic_auth_str(user, pword)
            _r = r.connection.send(r.request, **kwargs)
            _r.history.append(r)

            return _r

        setattr(self, 'num_401_calls', 1)
        return r
```

These should both allow the kind of delayed activation you want, and indeed the `handle_401()` hook should be able to block as long as necessary (e.g. to accept user input).

Hopefully these address your specific problem. Now I'll address your more philosophical questions.

One of the things that Requests desperately tries to avoid is the complexity of urllib2's password API. It's not that we don't think separating credential management and authentication is _ipso facto_ a bad idea, but the API is dire.

Requests is pragmatic. Design decisions are made with simplicity and pragmatism as the primary two concerns. This means we have to answer two questions:
1. Can this separation of concerns be implemented in a way that doesn't inconvenience the vast majority of users, who honestly couldn't care less?
2. Is this separation of concerns actually worth having in the library proper?

I'm not sure about (1), though my gut feeling is 'no'. I'm certainly open to someone proving me wrong. More importantly, I don't know that I think the answer to (2) is 'yes'. I'm prepared to be convinced, but at the moment I'm pretty comfortably leaning towards this being something that people who want it have to implement themselves.


@Lukasa 
Thanks so much for the detailed and informative response.

I had gone down the 'handle_401' route, but didn't like it because it didn't seem reusable and seemed to combine too many factors (authentication, request/response workflow, password management), so that's why I abandoned it and ended up here.

I thought maybe I was just framing the problem wrong and sure enough, your simple example works very nicely for my needs.

Modeled after your example, I [implemented this](https://bitbucket.org/jaraco/lpaste/src/4e127b565867eb149b62d753d9c467cbfd196453/lpaste/lpaste.py?at=default#cl-139), which greatly simplified the implementation.

Regarding the separation of concerns, I believe it should be possible to provide an extensible mechanism for auth that separate concerns but degrade nicely to the simple, default pattern. I'll have to defer that work for later, though, and leave this issue closed in the meantime.