TooManyRedirects getting page with 200 status code

I'm not sure why but requests raises `TooManyRedirects` on a `.get()` of a page that has a 200 status response.

URL: 
http://shorouknews.com/news/view.aspx?cdate=25112012&id=04c4889a-893c-49a2-8fea-5e2acb25dfc3

``` python
import requests

url = "http://shorouknews.com/news/view.aspx?cdate=25112012&id=04c4889a-893c-49a2-8fea-5e2acb25dfc3"
r = requests.get(url)
print r.url
print r.headers
```

Output:

```
Traceback (most recent call last):
  File "/home/inactivist/src/python/requests-redirects-test.py", line 4, in <module>
    r = requests.get(url)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 65, in get
    return request('get', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/safe_mode.py", line 39, in wrapped
    return function(method, url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 51, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 241, in request
    r.send(prefetch=prefetch)
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 648, in send
    self._build_response(r)
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 241, in _build_response
    raise TooManyRedirects()
TooManyRedirects
```

Let's see what lynx tells us about the page:

```
$ lynx -dump -head "http://shorouknews.com/news/view.aspx?cdate=25112012&id=04c4889a-893c-49a2-8fea-5e2acb25dfc3"
HTTP/1.1 200 OK
Server: cloudflare-nginx
Date: Sun, 25 Nov 2012 21:15:02 GMT
Content-Type: text/html; charset=utf-8
Connection: close
Cache-Control: private
Vary: Accept-Encoding
Set-Cookie: ASP.NET_SessionId=w3vuvi5lwsip2luzqxzoc31t; path=/; HttpOnly
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Set-Cookie: __cfduid=dc85a7b4e254397782142f2d681d5ca071353878100; expires=Mon, 2
3-Dec-2019 23:50:00 GMT; path=/; domain=.shorouknews.com
```
## Version Info

Requests 0.14.2, installed via `pip install requests`
Python 2.7.1
Ubuntu 11.04 32 bit.


Bizarrely, I can't repro this. Using OS X, but otherwise the exact same setup. @sigmavirus, can you repro this?


Using lynx you don't send the same request - `User-Agent` is different for example. Set `allow_redirect=False` and see the first response you get using Requests.


I'm not on Ubuntu but I have access to a few I can test on but they're not 11.04 they're 10.04 which are kept very much up-to-date. I'll also test on my slackware machines. 


I literally just remembered that I have an Ubuntu box in my house. On 12.04, Python 2.7.3, I _can_ repro this. So definitely a real bug.


So, preliminary investigation reveals that on OS X I am not redirected when I access that URL. On Ubuntu, I get a redirect to `/mobile/news/view.aspx?cdate=25112012&id=04c4889a-893c-49a2-8fea-5e2acb25dfc3` (for those not following, that's a redirect to the mobile version of that page). Then, every subsequent access redirects me to the same page.

The only difference between the two requests as far as I can see is the User Agent header. On Linux: `python-requests/0.14.2 CPython/2.7.3 Linux/3.2.0-32-generic`. On OS X: `python-requests/0.14.2 CPython/2.7.2 Darwin/12.2.0`.

My guess is that the website is at fault here, and is using the user-agent to redirect to the mobile version of a page based on having Linux in it (assuming that it's Android). That doesn't explain why it then enters a redirect loop, but clearly it does.


I get a redirect for:

> requests.get(url, allow_redirects=False, headers={'User-Agent': 'gene'})

but not for:

> requests.get(url, allow_redirects=False, headers={'User-Agent': 'gen'})

Just an odd website.  Redirects to mobile site, then keeps on redirecting to the same url.


Yeah, that's correct. Requests isn't misbehaving here, the site is.

To fix it, change your user agent to whatever you want, e.g.:

``` python
import requests

r = requests.get('http://shorouknews.com/news/view.aspx?cdate=25112012&id=04c4889a-893c-49a2-8fea-5e2acb25dfc3', headers={'User-Agent': 'python-requests/0.14.2 CPython/2.7.2'})
```


Closed as "not a bug".


@Lukasa : I'm not surprised the web site is misbehaving (I should have probed more thoroughly)  --  thanks for the mitigation tip.


@inactivist: No worries, it's not always easy to probe. Besides, your bug report was awesome: thorough and contained tons of information. Thanks!


@Lukasa: Thanks for the positive feedback.  I aims to please!

I'm definitely a requests newbie, but so far I love it.  And I have to say, the response here (on a Sunday, no less!) was freaking awesome.  Thanks, @all!