Issue 1:
I'm trying to request google search results and access the top few links. However, it always gets stuck on some sites. Ctrl-C is not in effect but Ctrl-Z does bring me to the terminal.

I have set a timeout for each request.

I have read messages that DNS is one of the sources of blocking and I patched this with an HTTP adapter.

It seems that the requests library is using blocking TCP connection(not sure), where the program has no control over the timeout if the underlying implementation does not return.

In another scenario, the connection is perfectly slow or the page is unexpected too big such that this connection keeps forever. This case is obviously not a bug but a vulnerability that can be used to anti-spider.

I would like to set a hard limit for my connection, regardless of DNS, TCP handshaking time, TLS handshaking time, or uploading/downloading time, and at any time I would like to exit, I just enter Ctrl-C. This can be done by using requests in another thread(POSIX thread)/process, ungracefully.

My program just stuck again.

Issue 2:
The request my_session.head(some_url, verify=True) reuses existing connections, including any that were established with verify=False.

Expected Result

I would expect that HTTP requests with verify=True would only reuse existing connections that were also established with verify=True.

Actual Result

From my test-case, it seems that the session will reuse an existing connection, irrespective of how that connection was established.

Reproduction Steps

The following code should demonstrate the problem.

import requests
from urllib3.exceptions import (MaxRetryError, SSLError)

def do_head_request_on(strategy, verify):
try:
r = strategy.head("https://wrong.host.badssl.com/",
verify=verify)
print("With verify %s,\tHTTP request made OK." % verify)
except requests.exceptions.ConnectionError as ce:
cause = ce.__context__
if type(cause) is MaxRetryError:
cause = cause.reason
if type(cause) is SSLError:
print("With verify %s,\tTLS problem" % verify)
else:
raise ce

requests.packages.urllib3.disable_warnings()

print("NOT using sessions:")
do_head_request_on(requests, True)
do_head_request_on(requests, False)
do_head_request_on(requests, True)

session = requests.Session()

print("\nUSING sessions:")
do_head_request_on(session, True)
do_head_request_on(session, False)
do_head_request_on(session, True)

Here is the output on my machine:

NOT using sessions:
With verify True, TLS problem
With verify False, HTTP request made OK.
With verify True, TLS problem

USING sessions:
With verify True, TLS problem
With verify False, HTTP request made OK.
With verify True, HTTP request made OK.

The issue is that the final request has verify=True against a known-bad endpoint; however, the request is successful.

System Information

$ python -m requests.help

{
"chardet": {
"version": "4.0.0"
},
"cryptography": {
"version": "37.0.4"
},
"idna": {
"version": "2.10"
},
"implementation": {
"name": "CPython",
"version": "3.9.2"
},
"platform": {
"release": "5.10.0-18-amd64",
"system": "Linux"
},
"pyOpenSSL": {
"openssl_version": "30000050",
"version": "22.0.0"
},
"requests": {
"version": "2.25.1"
},
"system_ssl": {
"version": "101010ef"
},
"urllib3": {
"version": "1.26.5"
},
"using_pyopenssl": true
}