Exception messages

As a user I would like it to be easy to generate simple helpful messages upon an exception. A common way this is done in is to simply cast the exception to a string. However, with requests, the result is often something you don't want to show an end user. For example:

``` python
    try:
        downloaded = requests.get(url)
    except (requests.Timeout) as err:
        print(str(err))
```

Results in the following message to the user:

```
    HTTPSConnectionPool(host='cal.example.com', port=443): Max retries exceeded with url: /ken/ken.ics/00832974-ffb3-42ea-ba3e-84ba3c0a30f6.ics (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7fd4644ef400>, 'Connection to cal.example.com timed out. (connect timeout=0.1)'))
```

There is useful information in this message, but it is not easily user accessible and is rather intimidating for end users. The information is probably available in the exception itself, but it is not clear how to get it. Also, it seems like accessing it would likely be different for each type of exception, which greatly increases the complexity of catching and reporting exceptions.

What I would expect is something like::

```
    Connection to cal.example.com timed out.
```

It would be very helpful if there were an easy way to generate user friendly error messages from requests exceptions. If there is such a way, I have not been able to find it. Thus, I suggest it be added to the otherwise excellent introduction to requests. If there is not such a way, I would like to to suggest that it be added.


So a little introspection shows this:

``` py
>>> import requests
>>> try:
...   r = requests.get('https://httpbin.org/delay/5', timeout=2)
... except requests.Timeout as terr:
...   pass
...
>>> terr
ReadTimeout(ReadTimeoutError("HTTPSConnectionPool(host='httpbin.org', port=443): Read timed out. (read timeout=2)",),)
>>> terr.message
ReadTimeoutError("HTTPSConnectionPool(host='httpbin.org', port=443): Read timed out. (read timeout=2)",)
>>> terr.message.message
"HTTPSConnectionPool(host='httpbin.org', port=443): Read timed out. (read timeout=2)"
>>> str(terr)
"HTTPSConnectionPool(host='httpbin.org', port=443): Read timed out. (read timeout=2)"
```

The best that can be done is to call `str(terr)` to get the lowest level error message from urllib3. Alternatively, if you always know it's a Timeout in that case, you can do this:

``` py
try:
    r = requests.get(url)
except requests.Timeout:
    print('Request to {} timed out.'.format(url))
```


The appeal of requests is its simplicity and ease of use. However it seems that exceptions appear to be, well, the exception. It would be nice if there were some easy way of getting an error message suitable for the end user that would work for all of the various exceptions generated by requests.


@KenKundert That's sadly a bit of a tricky business. Because of the nature of Python code the total range of exceptions that may be thrown out of requests code is actually very nearly unbounded: our dependencies may throw a really quite dramatic range.

While Timeout errors potentially have nice user-friendly messages, not all the others do, or they represent a painfully large range of problems (SSLError is a particular culprit here, due to the fact that there's only one exception used and it uses OpenSSL's extremely inscrutable error messages to generate its text.

I think the best we could do is enhance the API documentation to indicate how to get good error messages, or to indicate what the error messages could/should be for each type of exception.


@KenKundert I think there's a disconnect here. requests is meant to be powerful and simple for _developers_ (of varying experience with HTTP). Developers tend to prefer more detailed information when something fails than they prefer vague end-user friendly messages. We give developers what they need to debug failure cases. It's up to those developers to make something end-user friendly.

Failure scenarios also aren't uniform across all servers, applications, etc. The developer creating something for the end-user should strive to anticipate as many of those failure cases as possible and handle them in a way that appropriately and accurately describes the problem to those users.


But don't developer develop for users? I agree that during debugging the more detailed messages might be helpful, but in the end the developers should want to create programs that are easily used. Even when I develop programs for my own use, when I am actually using them I prefer error messages that I don't have to dissect.


We don't know who developers using requests are developing for. They can be developing for a wide range of other people. praw, github3.py, twython, etc. are all geared towards developers. httpie, python-keystoneclient, etc. are all geared towards some mix of developers and non-developers. Other projects use requests in other ways and expose errors much differently. Our only concern is providing the person directly using requests with as much information as we can.

How you provide users (whether you're the sole user or not) of your code with informative error messages is up to you.


I agree that the information should be available, but currently it is very difficult to access. Presumably it is accessible through the exception object itself, but how to access it is undocumented and seems to differ from exception to exception. The only thing I reliably have access to is the exception cast to a string, and there if I want to use it I have to parse the string. Preferably the components of that string would be available through attributes or methods of the exception.

In my experience, casting the exception to a string generally results is a reasonably easy to read summary of the error in a form that could be passed to the end user, and the components of that message are also available so that a tailored response is possible. For example, an IOError contains filename, strerror, errno, etc. to make it possible to roll your own message or craft a response.

The problem I am having with requests at the moment is that I cannot produce an error message suitable for end users. Casting the exception to a string produces something too complex, and I cannot find the things I would need to use to create an informative message. I am reduced to an error message that simply parrots the name of the exception:

``` python
    try:                                                                         
        payload = requests.get(url)
    except (requests.Timeout) as err:
        print('%s: connection timed out.' % url)
    except requests.ConnectionError as err:
        print('%s: connection error' % url)
    ...
```

My fear is that not only is the information I need not documented, but getting it is complicated and highly exception specific. This seems to be what @Lukasa is suggesting. Then, to produce good responses to exceptions I will have to understand all of those exceptions in detail and craft code that handles all the cases that could occur. And of course, most users would have to do the same thing, meaning that this code will be written over and over. Perhaps someday someone will create a requests-like package for interpreting the exceptions produced by requests. I guess I am suggesting that it would be good to build that into requests itself, or it it already exists, then give some examples of how to use it in the documentation.


For the Certbot project, we have a similar need: https://github.com/certbot/certbot/issues/4535. Specifically, when retries are enabled, all errors seem to get wrapped in "ConnectionError: HTTPSConnectionPool(host='acme-v01.api.letsencrypt.org', port=443): Max retries exceeded with url: /directory (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f020eee70d0>: Failed to establish a new connection: [Errno -2] Name or service not known',))". We can unwrap the error layers ourselves, but I think this is probably a common problem, and end-users would be better served by a library that provide more readable errors by default. And, having attempted to manually unwrap the error layers in our code, it is not immediately obvious how to do so.

I would also argue, in this specific case, that even a developer usually doesn't consider "Max retries exceeded" to be the core error, but the "Name or service not known." Reducing some of the layering in the string format would be useful. Note that I'm not proposing to change the actual exceptions thrown, since that would break existing code.

@jsha So, I think it'd be instructive to get an idea of what you're proposing here. Should we be removing the text "Max retries exceeded"? What do we do if retries *are* enabled, given that they will necessarily hide some errors?

Yep! My ideal error would be more like:

> ConnectionError: For https://acme-v01.api.letsencrypt.org/directory: Failed to establish a new connection: [Errno -2] Name or service not known

That involves stripping out a few layers of errors. I realize that this may hide errors: For instance, the first try might result in a connection refused, and the second try might result in a timeout. I think that's fine. For most error conditions, they will be the same, and in situations where they are different, it's usually non-deterministic.

If we wanted to really emphasize the retrying aspect, it would be reasonable to append "(tried N times)" after the message, but I think that is not super important.

I'd presume you would only want this change for the string representation of the error? Changing the internals so that the MaxRetryError is gone would be tricky and sketchy from a backward-compact perspective. 

Yep, agreed that changing the internals would likely break a lot of programs. It would be sufficient for our use case to just change the string representation.

Ok, so the likely thing to want to do is to override the `__str__` implementation of some of the relevant Requests exceptions, probably in the base class if possible, to spot `MaxRetryError` and strip it out. I'll happily review any PR that makes this change. :grin:

Working through an implementation now; will reference here.

Proposed starting to implementation: https://github.com/kennethreitz/requests/pull/4047