added some null checking
removed head request method
added 404 error checking
Redirect urls are now not followed, rather it becomes a two-step process and are logged as two urls
added poor link naming to the accessibility helper
updated the is hash tag function
added new input label ignores, changed input labels to look for input ids instead of name's
now respects robots txt