Meta robots

This commit allows the code to honor the meta attributes for nofollow and noindex. The system is separately configurable to honor nofollow and/or noindex. The default configuration is to honor noindex, but to ignore nofollow.
In other tests I have simply assumed that this test file is loaded and have been treating it like our "main" test config file so we don't have to keep loading it everywhere. In tests that need different config I think we want to use `defer` to reset the config like you did below. Can we just change this to be an `init()` function and not have all the `readConfig()` calls?