scrapy/scrapy — pull request #228 by marcospri (2013-01-21 14:51:59+00:00)
Title:    Check for None request before sending them to crawl

make_request_from_url on InitSpider returns None for the requests scheduled for postinit.

When using InitSpider and start_urls make_request_from_url from InitSpider returns None for each url after the first (scheduling them instead of returning right away), added check on engine.py/_next_request

Tried with this sample spider:
```python
from scrapy.http import Request
from scrapy.contrib.spiders.init import InitSpider

class DmozSpider(InitSpider):
name = "dmoz"
allowed_domains = ["dmoz.org"]
start_urls = [
"http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
"http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
]

def init_request(self):
return Request("http://www.dmoz.org", callback=self.initialized)

def parse(self, response):
filename = response.url.split("/")[-2]
open(filename, 'wb').write(response.body)
```