ported to code to use w3lib.encoding (work in progress, many tests failing yet)
Adapt response encoding detection to pass test cases
Merge pull request #51 from lostsnow/master

scrapyd: support bind to a specific ip address
added minor clarification on cookiejar request meta key usage
bumped required w3lib version to 1.1, after refactoring encoding detection to use the new w3lib.encoding module
Merge pull request #97 from scrapy/w3lib_encoding

Ported scrapy to use w3lib.encoding
removed unused import
Dropped support for Python 2.5. See: http://blog.scrapy.org/scrapy-dropping-support-for-python-25
change  start_request to return a generator instead of a list, in order
to allow to modify start_urls triggered by spider_opened signal
replace list by generator also in start_requests method of Sitemap
spider
Merge pull request #98 from kalessin/start_requests

This will break any spider that extends `start_requests` and expect a `list` as return value.

In the other side:

* [Docs](http://doc.scrapy.org/en/latest/topics/spiders.html#scrapy.spider.BaseSpider.start_requests) says that return value is **iterable** not list: 
* Scrapy core already support consuming start_requests generator on demand so we can avoid problems like #47
* it allows extensions to change starting requests on `spider_opened` signal
Restored scrapy.utils.py26 module for backwards compatibility, with a deprecation message. This is needed because the module was used a lot by users and the change causes too much trouble
test that default start_request return value type is a generator. refs #98
bumped required python-w3lib version in debian/control
Added CLOSESPIDER_ERRORCOUNT to scrapy/default_settings.py
Removed deprecated setting: CLOSESPIDER_ITEMPASSED
removed backwards compatibility code for old scrapy versions
Removed (very old and possibly broken) backwards compatibility support for Twisted 2.5
updated FormRequest tests to use HtmlResponse instead of Response, as it makes more sense
removed some obsolete remaining code related to sqlite support in scrapy