ericholscher/django-crawler — pull request #6 by chrisv2 (2013-02-08 17:03:42+00:00)
Title: add '--exclude' option

patch which adds a '--exclude' option, consisting of comma-separated list of patterns to exclude from crawl. excluding stuff like ^/admin can significantly speed up the crawl.