Ignore todo.txt
Issue HEAD request and check content-type

Prevents fetching and parsing URLs which are not HTML and therefore will
not contain links - especially important to avoid this for large binary
files.