ecdeveloper/node-web-crawler — pull request #6 by josephdpurcell (2013-07-19 18:43:14+00:00)
Title: Handle invalid links and empty responds bodies

The code fixes in 147abbc were required for me to crawl sites that have invalid links or links to URLs with empty response bodies without killing node. The solution I implemented was to ignore the links that are invalid and links that have empty response bodies. I don't know if that is the best behavior, but first and foremost I think it is important that the code work out of the box.

In addition, this upgrades jQuery to v1.9 and Bootstrap to the latest, v2.3.2. (Not sure how you feel about upgrading those, if you don't think it's a good idea for some reason please let me know.)