Connection pooling is tied to sessions

This leads to confusion, as demonstrated by #3201 (ping @thanatos), but it's not a documentation issue, it's a design issue. Connections and sessions are unrelated: a connection can be used by multiple sessions, and a session can span multiple connections; thus the library shouldn't tie them together.


Thanks for the feedback. It is appreciated and will be considered momentarily (then ignored) :)


Gentle +1 on this one.

The existing documentation isn't very clear (as another user mentioned in #3201, I find it quite contradictory, despite the assertion to the contrary). After reading these various GitHub issues I have some understanding as to how it works, and I would especially never have divined that it's possible for connections to be shared across sessions (as mentioned here) without them. I've never come across another language/HTTP library where this is the case, so I would never have intuited Requests' behavior here.

IMO, it would be beneficial for Requests to have a section in the documentation explaining the precise mechanics of connection pooling. I'd even go as far as to draft it if that would help.

> After reading these various GitHub issues I have some understanding as to how it works, and I would especially never have divined that it's possible for connections to be shared across sessions (as mentioned here) without them.

Connections *cannot* be shared across Sessions. Anything you read that gave you that impression is wrong. Sessions are the only place where we anchor connection pooling.

As to whether the library ties them together or not, that is absolutely the prerogative of this library. Despite the confident assertion of the OP, plenty of other libraries in other languages tie Sessions and connections together: consider `NSURLSession` from iOS, or Alamofire, or any number of others.

I have no problem with having some extra documentation, but again I don't think there's any lack of clarity here. Right at the top of the [Session objects](http://docs.python-requests.org/en/master/user/advanced/#session-objects) section of the documentation we say:

> The Session object allows you to persist certain parameters across requests. It also persists cookies across all requests made from the Session instance, and will use urllib3's connection pooling. So if you're making several requests to the same host, the underlying TCP connection will be reused, which can result in a significant performance increase (see HTTP persistent connection).

I don't know how much clearer we could make this without adding a hero graphic.

> I have no problem with having some extra documentation, but again I don't think there's any lack of clarity here. Right at the top of the Session objects section of the documentation we say:

Right. I don't think that in itself is overly unclear, but when considered along with the front page assertion of:

> Keep-alive and HTTP connection pooling are 100% automatic

I think that most developers (including myself and the original reporter of #3201) would find the documentation contradictory. I understand that Kenneth may have meant this to read that the porcelain-level APIs will pool within any given HTTP action, but when backend developers talk about connection pooling, we tend to be thinking about pooling that's a far reach more effective than that. Pooling within any single action is likely to give you only a very nominal performance advantage when using an HTTP library across ten of thousands of requests.

> I don't know how much clearer we could make this without adding a hero graphic.

I am sorry, but I honestly don't understand what this means. I hope that my original comment was worded constructively, but let me know if there's anything that I can correct there.

I think we're arguing past each other, and mostly disagreeing on what "100% automatic means". It seems that you and the original reporter of #3201 conclude that "100% automatic" means "applies to any possible usage of the API": that is, no matter how you use Requests you should get connection pooling unless you explicitly don't ask for it.

Ultimately I don't agree with that: to my reading, "100% automatic" means "without user intervention", and that aspect is *true*. But it does not mean that arbitrary usage of the API will obtain it.

Would the problem be resolved if we simply removed the "100%" from the documentation? If we removed the word "automatic"? I'm trying to understand what specifically in the sentence "Keep-alive and HTTP connection pooling are 100% automatic" contradicts the notion that you have to use a Session to get it.

Regardless, let's say that we accept your assertion that the documentation is contradictory. How would you resolve the contradiction? What documentation would you add/remove/change, and how?

> Would the problem be resolved if we simply removed the "100%" from the documentation? If we removed the word "automatic"? I'm trying to understand what specifically in the sentence "Keep-alive and HTTP connection pooling are 100% automatic" contradicts the notion that you have to use a Session to get it.

I don't think removing "100%" would clarify anything. The basic problem is that this sentence is perfectly ambiguous; you might interpret it one way while I might interpret it another.

I don't think an ambiguous sentence is the end of the world, but I do think that it should be clarified elsewhere. Talking about connection pooling under the session documentation is fine, but not wholly adequate.

> Regardless, let's say that we accept your assertion that the documentation is contradictory. How would you resolve the contradiction? What documentation would you add/remove/change, and how?

I'd suggest a short section on "Connection Pooling in Requests". It could talk about (1) how connections are pooled within a single top-level API request, and (2) how connection pooling across many requests can be achieved with the use of a session object. Pretty much exactly what we've been talking about here.

Anyway, I don't feel strongly enough about this to forcefully champion it through in the case that there's strong backpressure to corrections from this library's maintainers. I commented originally to re-raise that issues like this one and #3201 seem to be good evidence that people are having trouble interpreting the documentation, but there seems to be a very strong sentiment that this is user error, so I'll let it slide.

> Despite the confident assertion of the OP, plenty of other libraries in other languages tie Sessions and connections together: consider `NSURLSession` from iOS, or Alamofire, or any number of others.

Other libraries being designed the same way doesn't prove that the design is good.

(I'm not trying to reopen the issue, I'm just answering because I was mentioned.)