Still cannot prevent requests library from encoding

I have the same problem as https://github.com/psf/requests/issues/5964, but the solution does not work for me.
I need to send unescaped URL requestes to a server. The purpose is to evoke response code 400 to test the firewall configuration.

## Expected Result
The following code should (according to https://github.com/psf/requests/issues/5964) send the unescaped URL:

```
import requests
s = requests.Session()

# Using Burp Suite Proxy to examine request being sent.
s.proxies = { 
    "http"  : "http://127.0.0.1:8080",
    "https" : "http://127.0.0.1:8080",
}
# Do not verify certificate.
s.verify = False

# Solution from https://github.com/psf/requests/issues/5964
base_url = 'https://www.example.com/search'
query = '?date_range=2017-01-01|2017-03-01'
req = requests.Request('GET', base_url)
p = req.prepare()
p.url += query
resp = s.send(p)
print(resp.request.url)
```


## Actual Result

The `print(resp.request.url)` prints `https://www.example.com/search?date_range=2017-01-01|2017-03-01` as expected.
But what is really transmitted is the escaped URL, `https://www.example.com/search?date_range=2017-01-01%7C2017-03-01` as seen in this Burp Suite screenshot: 
![image](https://user-images.githubusercontent.com/4678285/165955894-9a8d73e0-98a9-48a3-abac-6ce2c652c138.png)


## Reproduction Steps

Use the python code above and a way of choice to capture the actual response being sent (e.g. Burp Suite  proxy).
You can also use curl to send an actually unescaped response:

`curl "https://www.example.com/search?date_range=2017-01-01|2017-03-01" -x 127.0.0.1:8080 --insecure`

![image](https://user-images.githubusercontent.com/4678285/165958290-a6dfbf45-a620-453a-9054-84cba0b4f5a2.png)



## System Information

    $ python -m requests.help

```json
{
  {
  "chardet": {
    "version": "3.0.4"
  },
  "charset_normalizer": {
    "version": "2.0.12"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.10"
  },
  "platform": {
    "release": "5.14.0-1034-oem",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.27.1"
  },
  "system_ssl": {
    "version": "1010106f"
  },
  "urllib3": {
    "version": "1.25.8"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": false
}
```




Hmm, so this appears to actually be a breakage introduced by this PR in urllib3 1.26.0. Requests is still forwarding the unencoded URL but [urllib3 now unilaterally enforces encoding](https://github.com/urllib3/urllib3/commit/691679f738ae098ce024aeae90cfdd8013b8e1e7#diff-2d5453550bf56a4666a88e9262248cea2f8ef2338695b5300b6c9a550647ee0eR631) on paths. The immediate fix would be to downgrade to urllib3 1.25.11 but that's not a great idea long term.

I don't know if there's a good way to fix this now given the amount of time it's been in place. We could potentially not override the provided url and only use the parsed version where needed for the scheme. [This line](https://github.com/urllib3/urllib3/commit/691679f738ae098ce024aeae90cfdd8013b8e1e7#diff-2d5453550bf56a4666a88e9262248cea2f8ef2338695b5300b6c9a550647ee0eR651) would need to be reverted but I don't know if we've enforced the scheme will always be present at this point.

@sethmlarson what are your thoughts here? urllib3 has become more strict with what it will emit which is generally a positive but has broken some portions of the PreparedRequests workflow over the last handful of years. Is this a use case urllib3 is willing to support?

@nateprewitt Yes, we should be supporting this through HTTPConnectionPool.request(). I think we lost this behavior sometime but it'd be good to restore it in cases where it's safe to do so.

This problem may be solved by hooking urllib3 like this:

```python
import requests
import urllib3.util.url as urllib3_url


def hook_invalid_chars(component, allowed_chars):
    # handle url encode here, or do nothing
    return component

urllib3_url._encode_invalid_chars = hook_invalid_chars

s = requests.Session()

s.verify = False

base_url = 'http://127.0.0.1:8080'
query = '?date_range=2017-01-01|2017-03-01'
req = requests.Request('GET', base_url)
p = req.prepare()
p.url += query
resp = s.send(p)
print(resp.request.url)
```

![图片](https://user-images.githubusercontent.com/11925471/176877894-8b209d02-0210-48f3-8fc8-b9db8ee03991.png)

System Information

```json
{
  "chardet": {
    "version": "3.0.4"
  },
  "charset_normalizer": {
    "version": "2.0.12"
  },
  "cryptography": {
    "version": "2.8"
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.10"
  },
  "platform": {
    "release": "4.4.0-19041-Microsoft",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010106f",
    "version": "19.0.0"
  },
  "requests": {
    "version": "2.26.0"
  },
  "system_ssl": {
    "version": "1010106f"
  },
  "urllib3": {
    "version": "1.25.8"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": true
}
```

@LudiusMaximus
Hope you're doing well. Did you find a solution?

The solution provided by @nateprewitt on #5964 doesn't work for me either (I tested).

@Lukasa provides a prepared requests example as well on #1454 at [this point in the discussion](https://github.com/psf/requests/issues/1454#issuecomment-20832874)

Based on the contribution guidelines I put the details for my situation on Stack Overflow since whether this is a bug/limitation is a gray area.

In my case if the device on the other end was my web server, I'd fix it - it isn't so I'm looking for a way to interface with it.

I would greatly appreciate help toward a code solution to overcome this roadblock. Thank you!

https://stackoverflow.com/questions/77255960/python-requests-prevent-url-encoding

hi, +1 on this being an issue. I tried LyleMi's solution and it did not work even though the url seems well formed.

> hi, +1 on this being an issue. I tried LyleMi's solution and it did not work even though the url seems well formed.

@abaruah117
TLDR: You'll either need to override items within `urllib3` or use the lower-level `urllib` directly (credit to Tomi on Stack Overflow). The first one is probably frowned upon and the latter with `urllib` could be painful (requires manually handling many aspects that `requests` handles for us).

(I haven't entirely chosen my path to overcome the URL roadblock I ran into here. My task to interact with an older system is on-hold until I have time.)

More Details:

It is my _impression_ that based on minimized comments (including Lyle's) and the lack of an official solution that this issue is deemed a problem the `requests` project won't be fixing (as the code causing this behavior exists in `urllib3`). Ludius' example included a pipe and mine included curly braces -- both of those do not appear to show up as "reserved" characters in the RFCs.

You might consider updating your reply with an example of the URL and format you're having issues with.

I have a question over on Stack Overflow where a really helpful person (Tomi) decided to poke at the issue. The code forcing the URI encoding is within `urllib3` and not directly in `requests` so it isn't something `requests` will resolve (my opinion).

In [RFC 2396](https://datatracker.ietf.org/doc/html/rfc2396), pipe and curly braces (among some other symbols) are considered "unwise" to use in URLs. And [RFC 3986 in section 2](https://datatracker.ietf.org/doc/html/rfc3986#section-2) does not show pipe or curly braces as reserved, but then again 2396 is obsoleted or superseded by 3986. I suppose more than anything those and RFCs that obsolete even those two intend to explicitly convey the **allowed characters** and leave the prohibited/forbidden ones for inference (exception being the reserved characters that are explicitly called out).

Edit: grammar

+1 I totally understand following standards in MOST cases but there should be some way for a developer to override the default behavior. This is making things hard for us security folks. 

> +1 I totally understand following standards in MOST cases but there should be some way for a developer to override the default behavior. This is making things hard for us security folks.

@superswan 
While it can be overridden the changes won't be in `requests`, but instead `urllib3`.

There are ugly hack solutions over on Stack Overflow. _No warranties and could break unexpectedly._
https://stackoverflow.com/questions/77255960/python-requests-prevent-url-encoding

My testing was via netcat (nc) for Stack Overflow post so hopefully this works when I get back to running it against that live legacy device.

I hope this helps you as it helped me.

@mjbear thank you. I recognize that the issue originates from urllib3. I wanted to express my concerns since this appears to be one of the few places engaging in active discussion. I have opted to use urlopen() instead.

However, this issue also impacts the Requests library, resulting in the loss of some of its enhanced features such as error handling, header manipulation, and timeouts due to the workaround. In past versions it was possible to override this behavior from requests directly. This change has made it challenging to work with legacy code and manage use cases that fall outside of standard practices.

As you are testing your firewall configuration you may be interested in my approach to unencode characters or even inject the payload right before the request is send, so its never modified:


```
import urllib3
import requests

def custom_make_request(self, conn, method, url, timeout=urllib3.connectionpool._Default, chunked=False, **httplib_request_kw):
    url = url.replace("FUZZ", "TEST!/&%$=)()")
    new_headers = {}
    for header, value in httplib_request_kw['headers'].items():
        new_header = header.replace("FUZZ", "TEST!/&%$=)()")
        new_value = value.replace("FUZZ", "TEST!/&%§")
        new_headers[new_header] = new_value
    httplib_request_kw['headers'] = new_headers
    return self._original_make_request(conn, method, url, timeout=timeout, chunked=chunked, **httplib_request_kw)

# Create a reference to the original _make_request method
urllib3.connectionpool.HTTPConnectionPool._original_make_request = urllib3.connectionpool.HTTPConnectionPool._make_request
# Replace the _make_request method with the custom_make_request function
urllib3.connectionpool.HTTPConnectionPool._make_request = custom_make_request

response = requests.get('http://<YOUR_TEST_SERVER>/FUZZ')
```