kennethreitz/requests — Issue #265: HTTP request uses full URL, should be path only
Author: pennersr
Created: 2011-11-14 14:30:31+00:00
State: open

Possibly related to #174: these are links that work just fine in a browser or `curl`/`wget`, but `requests` seems to have trouble with them:

* http://www.rankingz.com
* http://my.jibecompany.com

Example:

>>> import requests
>>> requests.get('http://www.rankingz.com')
Traceback (most recent call last):
File "", line 1, in
File "/Users/pennersr/src/ep/opt/python2.6/src/requests/requests/api.py", line 78, in get
return request('get', url, **kwargs)
File "/Users/pennersr/src/ep/opt/python2.6/src/requests/requests/api.py", line 65, in request
prefetch=prefetch
File "/Users/pennersr/src/ep/opt/python2.6/src/requests/requests/sessions.py", line 187, in request
r.send(prefetch=prefetch)
File "/Users/pennersr/src/ep/opt/python2.6/src/requests/requests/models.py", line 432, in send
self._build_response(r)
File "/Users/pennersr/src/ep/opt/python2.6/src/requests/requests/models.py", line 245, in _build_response
request.send()
File "/Users/pennersr/src/ep/opt/python2.6/src/requests/requests/models.py", line 424, in send
raise ConnectionError(e)
requests.exceptions.ConnectionError: Max retries exceeded for url: http://www.rankingz.comhttp/www.rankingz.com

Examining the HTTP headers reveals...:

GET http://www.rankingz.com HTTP/1.1
Host: www.rankingz.com
Accept-Encoding: identity, deflate, compress, gzip
Accept: */*
User-Agent: python-requests/0.8.0

HTTP/1.1 301 Moved Permanently
Server: Apache
X-Powered-By: PHP/5.3.2-1ubuntu4.10
X-Pingback: http://www.rankingz.com/xmlrpc.php
Expires: Wed, 11 Jan 1984 05:00:00 GMT
Last-Modified: Mon, 14 Nov 2011 14:20:48 GMT
Cache-Control: public
Pragma: no-cache
Location: http://www.rankingz.comhttp/www.rankingz.com
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Type: text/html; charset=UTF-8
Content-Length: 20
Date: Mon, 14 Nov 2011 14:20:48 GMT
X-Varnish: 875757742
Age: 0
Via: 1.1 varnish
Connection: keep-alive

... whereas `curl` shows:

GET / HTTP/1.1
User-Agent: curl/7.22.0 (x86_64-apple-darwin11.1.0) libcurl/7.22.0 OpenSSL/1.0.0e zlib/1.2.5 libidn/1.22
Host: www.rankingz.com
Accept: */*

HTTP/1.1 200 OK
Server: Apache
X-Powered-By: PHP/5.3.2-1ubuntu4.10
X-Pingback: http://www.rankingz.com/xmlrpc.php
Last-Modified: Mon, 14 Nov 2011 14:25:32 GMT
Vary: Cookie,Accept-Encoding
Expires: Mon, 14 Nov 2011 14:26:52 GMT
Pragma: public
Cache-Control: public
Content-Type: text/html; charset=UTF-8
X-Cacheable: NO: Vary: Cookie
Content-Length: 25914
Date: Mon, 14 Nov 2011 14:25:32 GMT
X-Varnish: 875757934
Age: 0
Via: 1.1 varnish
Connection: keep-alive

`requests` should be using "GET /" instead of "GET http://www.rankingz.com".