gjtorikian/html-proofer — pull request #24 by gjtorikian (2013-12-29 18:32:08+00:00)
Title: Speedup, Redux

This branch introduces a few significant changes and optimizations:

* The ability to pass along any of Typhoeus' options from `HTML::Proofer.new`
* A cleanup of the test suite
* All external URL checks are collected, and the tests are run _after_ everything else has run. This lets you pass in a new option, `disable_external`, which won't trigger these tests (in case you only want to run them on a CI or something)
* All external URL checks are now [`HEAD` request](http://ochronus.com/http-head-request-good-uses/). This is way, way faster, since unlike `GET`, we don't download the page contents, nor do we care for them. A `HEAD` request is good enough for checking a URL's existence.

Unfortunately, using `HEAD` introduced some weird bug from `curl` (I'm on version 7.30). For an example, run this command from the terminal:

```
curl -I -X HEAD https://help.github.com/changing-author-info
```

You'll get back a `404`, even though the browser sees it as a `301`, then a `200`. This seems to affect a wide variety of sites.

The workaround is to execute a `GET` request when you see a `404`. If _this_ is a failure, then the page really does not exist.

Although you're under no obligation to, since a lot of the code has changed, it'd be nice for some :eyes:. /cc @benbalter @afeld @parkr