scrapy/scrapy — pull request #78 by darkrho (2012-01-09 18:27:43+00:00)
Title: settings: moved redirect middleware execution before httpcompresion mw.

This avoids the IOError "Not a gzipped file" when the website returns
a 301/302 with 'Content-Encoding: gzip' where actually the body is not
gzip-encoded.

To reproduce:
```
$> scrapy shell http://www.laloberinto-shop.com/checkout_shipping.php
2012-01-09 14:13:11-0400 [scrapy] INFO: Scrapy 0.15.1 started (bot: bot)
...
2012-01-09 14:13:11-0400 [default] INFO: Spider opened
2012-01-09 14:13:12-0400 [scrapy] ERROR: Shell error
...
File "/opt/projects/Insophia/scrapy_git/scrapy/core/downloader/middleware.py", line 46, in process_response
response = method(request=request, response=response, spider=spider)
File "/opt/projects/Insophia/scrapy_git/scrapy/contrib/downloadermiddleware/httpcompression.py", line 20, in process_response
decoded_body = self._decode(response.body, encoding.lower())
File "/opt/projects/Insophia/scrapy_git/scrapy/contrib/downloadermiddleware/httpcompression.py", line 36, in _decode
body = gunzip(body)
File "/opt/projects/Insophia/scrapy_git/scrapy/utils/gz.py", line 14, in gunzip
chunk = f.read(8196)
File "/usr/lib/python2.7/gzip.py", line 252, in read
self._read(readsize)
File "/usr/lib/python2.7/gzip.py", line 287, in _read
self._read_gzip_header()
File "/usr/lib/python2.7/gzip.py", line 181, in _read_gzip_header
raise IOError, 'Not a gzipped file'
exceptions.IOError: Not a gzipped file
```

After the fix:
```
$> scrapy shell http://www.laloberinto-shop.com/checkout_shipping.php
2012-01-09 14:15:14-0400 [scrapy] INFO: Scrapy 0.15.1 started (bot: bot)
...
2012-01-09 14:15:14-0400 [default] INFO: Spider opened
2012-01-09 14:15:14-0400 [default] DEBUG: Redirecting (302) to from
2012-01-09 14:15:15-0400 [default] DEBUG: Redirecting (302) to from
2012-01-09 14:15:17-0400 [default] DEBUG: Crawled (200) (referer: None)
```