scrapy/slybot — pull request #59 by andresp99999 (2013-02-06 16:38:04+00:00)
Title: Add allowed_domains variable to spider definition

The allowed_domains settings allow us to control the domains filtered by the OffsiteMiddleware. If not then it does the previous behavior (extract allowed_domains from start_urls)