Returns empty with Google over HTTPS

``` python
 import requests

In [2]: r = requests.get('https://google.com')

In [3]: r.text
Out[3]: u''

In [4]: r = requests.get('http://google.com')

In [5]: r.text
Out[5]: u'<!doctype html><html itemscope="itemscope"...
```


Facing similiar issue with Twitter too

``` python
In [13]: r = requests.get('https://twitter.com')

In [14]: r.text
Out[14]: u''

In [15]: r = requests.get('http://twitter.com')

In [16]: r.text
Out[16]: u''

In [17]: r.history
Out[17]: (<Response [301]>,)

In [18]: r.history[0].headers['location']
Out[18]: 'https://twitter.com/'
```

Because Twitter always redirects to https:// I am unable to retrieve any response from non-htttps twitter either.


@drepo This works just fine on my machine. I'm inclined to suggest that you've got an SSL-related problem. What OS are you using?


Fedora 18 (3.9.6-200.fc18.x86_64)


Two follow-up questions: where did you get the version of Python you're using (e.g. did you built it yourself?), and what version of OpenSSL do you have installed?


$ python -V
Python 2.7.3
$ openssl
OpenSSL> version
OpenSSL 1.0.1e-fips 11 Feb 2013


Python is OS distro default not compiled.


That's strange, the default OS distro should be fine with SSL. Out of interest, did you `pip install` requests or did you get it from `yum`? (Also, what version are you running? `import requests; print requests.__version__`)


``` python
In [8]: requests.__version__
Out[8]: '1.2.3'
```

Requests is installed from pip.


Wow, that's bizarre. Let me spin up an F18 VM and see what I can see. It'll take a little while.


``` python
In [1]: import httplib2

In [2]: h = httplib2.Http()

In [3]: r, c = h.request('https://google.com')

In [4]: c
Out[4]: '<!doctype html><html itemscope="itemscope" itemtype="http://schema.org/WebPage">...
```

Working using httplib2!


So quite likely not an SSL issue.


I disagree. =) We do some funky stuff with SSL, differently to what httplib2 does. Additionally, seeing as you only see it on HTTPS, there's really only one conclusion. =)


BTW, the status_code on all failed https requests using (requests) is always 200.


Twitter sends a content length header, what do you get?

``` py
r = requests.get('https://twitter.com')
print r.headers['content-length']
```

Are you using pyOpenSSL/pyasn1?

```
$ pip freeze | egrep 'pyOpenSSL|pyasn1'
pyOpenSSL==0.13
pyasn1==0.1.7
```


@t-8ch Response to your questions:

``` shell
# pip freeze | egrep 'pyOpenSSL|pyasn1'
pyOpenSSL==0.13
```

``` python
In [1]: import requests

In [2]: r = requests.get('https://twitter.com')

In [3]: r.headers['content-length']
---------------------------------------------------------------------------
KeyError 
```


It really doesn't look like you're getting any data on that connection.


And still it is taking horrible long (>40secs) to respond.


Can you try with python3?


@t-8ch Let me try.


Ok, I cannot reproduce this on a fresh Fedora install, which means it's either a problem with your system setup or your network. I'm going to let @t-8ch take the lead on the system setup end because he knows it better than me. =)


@Lukasa Thanks for the help. I am setting up a virtenv for py3 and trying it out as per @t-8ch's sugesstion.


@t-8ch Using python3, the problem persists

``` shell
Python 3.3.0 (default, Sep 29 2012, 22:07:38) 
[GCC 4.7.2 20120921 (Red Hat 4.7.2-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> r = requests.get('https://google.com')
>>> r.text
''
>>> requests.__version__
'1.2.3'
>>> 
```


Hm, I can't reproduce on my VM.

What does the following do:

``` py
import requests
import ssl
import socket
import time

HOST = b'google.com'

print('--> plain socket')
sock = socket.create_connection((HOST, 443))
ssl_sock = ssl.wrap_socket(sock)
ssl_sock.send(b'GET / HTTP/1.1\nHost: ' + HOST + b'\n\n')
time.sleep(1)
print(ssl_sock.recv(1000))

print('--> urllib3')
http = requests.packages.urllib3.PoolManager()
r = http.request('GET', b'https://' + HOST + b'/')
print(r.data)
```


@t-8ch 
I messed up my Fedora 18 installation and did a clean install of F19; the problem persists. (Note: also persist on Mint 15). Here is the output of your code:

``` shell
--> plain socket
HTTP/1.1 301 Moved Permanently
Location: https://www.google.com/
Content-Type: text/html; charset=UTF-8
Date: Mon, 15 Jul 2013 08:54:18 GMT
Expires: Wed, 14 Aug 2013 08:54:18 GMT
Cache-Control: public, max-age=2592000
Server: gws
Content-Length: 220
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="https://www.google.com/">here</A>.
</BODY></HTML>

--> urllib3
Traceback (most recent call last):
  File "temp.py", line 16, in <module>
    http = requests.packages.urllib3.PoolManager()
AttributeError: 'module' object has no attribute 'packages'
```


That's interesting, it looks like your Requests install is no good. How are you able to put requests through that at all?


I do not know if its bad (used pip only). But observed a new thing, Non HTTPS HEAD requests are also failing in the same vein.

``` python
In [4]: r = requests.head('http://google.com')

In [5]: r
Out[5]: <Response [200]>

In [6]: r.text
Out[6]: u''
```


HEAD requests are not supposed to return any content in their body, so this is expected.
@Lukasa is right. Your installation of requests is seriously broken.
What do `requests.__file__` and `requests.__dict__` contain?


Post a fresh install, I usually use an init script to setup/install the tools and requests is a part of that script. This time, post installation, I only installed requests and it is working:

``` python
>>> import requests
>>> r = requests.get('https://google.com')
>>> r.text
u'<!doctype html><html itemscope="itemscope" it
```

It seems there is something in that installation chain which is breaking requests. I will have to investigate further to figure out, my preliminary guess is that some oauth library is breaking it. 


Ok, I'm glad you have somewhere to start looking. In the meantime, as this is not a Requests bug, I'm going to close this issue. Feel free to keep commenting here if you need further assistance!


As and when I do figure out the situation, I will post it here. BTW, Thanks @Lukasa and @t-8ch, you have been quite helpful. :)