scrapy/scrapy — pull request #303 by redapple (2013-05-13 16:51:27+00:00)
Title: Optparse-stype options in Spiders (command line crawl)

This is a proposition to define options in spiders similar to what Django has for management commands.

It's basically an alternative to the `-a key=val` technique, but add `optparse`'s conversions and boolean support.

**Note: I only tested this for `scrapy crawl`**

Define `option_list` as a list of options in your spider.
Options are then available in spider in `self.cmdopts`.

Example usage:

```python
...
from scrapy.utils.options import make_option

class MyOptionsSpider(scrapy.spider.BaseSpider):
...
option_list = [
make_option("-I", "--counter", dest="counter", type="int",
help="set a cool option", default=None)
]

def parse(self, response):
...
print "counter:", self.cmdopts.counter
```
Then,
```
scrapy crawl MyOptions -I 899
```

**CAUTION: spider name has to be first parameter after `scrapy crawl`**