Skip static/media paths as it doesn't makes sense to crawl them
Merge pull request #26 from fabrique/skip_static

[crawler] Skip static/media paths as it doesn't makes sense to crawl them