A 200 hundred lines node.js crawler that follows robots.txt convention. Possibility to store files for tests.