fixes #19 HasRobotsNoFollow does not support "none"
Merge branch 'master' of github.com:sjdirect/abot