ekalinin/robots.js — pull request #5 by bingalls-compete (2012-12-12 16:24:10+00:00)
Title: Fixed a bug with relative redirects and added unit tests

There was an issue with redirects where if it was relative, it wouldn't have the full url to fetch and would throw an exception. I added a url.resolve call to cover this.

I also added a suite of unit tests to cover the parser.read functionality, and it should be pretty easy to extend in the future. It mocks up http and https so there shouldn't be any external calls.