make doindexfrom/butdontindexfrom into sets
test that some pages have the correct search-engine-indexability
bugfix: make rr resources be potentially indexable (fixes a test)

We'd been checking "if path in route_metadata" before adding
any rewritable-resource routes to route_metadata.  Fix: put the
reachability checking code after adding them.

Note: Redirects generated by trailing-slash-redirection
still aren't indexable, and I'm not sure in what cases they should
be indexable (same as target page? only when linked to directly?
never?).
Create a feature to obfuscate email addresses against spammers
remove old commented code
Use the validator's JSON output.

This is more robust than parsing the GNU text output with regexps.
allow obfuscated-mailto-address validation error

+ # This validation error is deliberate in an attempt to reduce
+ # spambot email harvesting.  The nonconformance appears not to
+ # cause issues in common browsers.
add optional spampoison link to HTML template

This ought to be able to be specified in the 'head' variable, but
Pandoc put a <p> around <link> and <p> don't go in <head>.
Don't give resources HTTP Link rel=canonical

Since our resources have their names chosen by exact contents,
and rel=canonical refers to only identity/approximate contents,
pointing the rel=canonical to the path with contents hash
wasn't really accurate.  Further, I don't foresee issues with
omitting this rel=canonical that including it would prevent.