gabrielfalcao/HTTPretty — pull request #145 by luqmaan (2014-01-26 20:56:41+00:00)
Title: Fix bugs that occur when using custom schema/port/regex

HTTPretty does not behave correctly when using regex matching, HTTPS and custom ports. When this scenario is triggered, a timeout/max retries exceeded error occurs.

To duplicate run:

```python
@httpretty.activate
def exceed_max_retries_with_custom_port_and_https():
HTTPretty.register_uri(
HTTPretty.GET,
re.compile('https://api.yipit.com:1234/v1/deal;brand=(?P\w+)'),
body='meow'
)
uri = 'https://api.yipit.com:1234/v1/deal;brand=gap?first_name=chuck&last_name=norris'
response = requests.get(uri)
return response.content
```

Cause
--

The combination of a regex URI, custom port, and HTTPS causes HTTPretty to get stuck at https://github.com/gabrielfalcao/HTTPretty/blob/2e814635fff916d3a8c246ca010245362266c89f/httpretty/core.py#L323 and eventually raise this error:

```
ConnectionError: HTTPSConnectionPool(host='api.yipit.com', port=1234): Max retries exceeded with url: /v1/deal;brand=gap?first_name=chuck&last_name=norris (Caused by : [Errno 36] Operation now in progress).
```

This error happens because URI schema's are reconstructed incorrectly during the URI matching.

This should fail (http != https), but it does not!

```python
@httpretty.activate
def broken_reconstruction_of_uri_schema():
uri = 'api.yipit.com:1234/v1/deal'
HTTPretty.register_uri(HTTPretty.GET,
'https://' + uri,
body=lambda method, uri, headers: [200, headers, uri]
)
response = requests.get(uri)
expect(response.text).to.equal('http://' + uri) # incorrect!
```

Solution
--

To correct the internal confusion between HTTP and HTTPS ports, we need to separate the two in our DEFAULT/POTENTIAL PORTS lists. When URIMatcher encounters a non-regex URI it uses URIInfo.from_uri to add
the URIs port to the known ports. This behavior is now added for regex URIs.

We now use the DEFAULT_PORTS lists in HTTPretty.reset() to reset the POTENTIAL_PORTS lists. Also, to avoid using the global keyword, we do an in-place reset with intersection_update.

To prevent getting stuck at `self.truesock.connect(self._address)` and raising the ConnectionError after ~60 seconds, moved `self.truesock.settimeout(0)` immediately before the call to `self.truesock.connect(self._address)`. Now, a `urllib2.URLError` is raised immediately.

The test `test_httpretty_bypasses_a_unregistered_request` previously expected a non-registered URI to respond instead of raise an exception. This is conceptually flawed, as a non-registered URI should not respond! The test now expects that `urllib2.URLError` to be raised.

Added the following tests:

- test_httpretty_should_work_with_non_standard_ports
- test_httpretty_reset_by_switching_protocols_for_same_port
- test_httpretty_should_allow_registering_regexes_with_port_and_give_a_proper_match_to_the_callback