added support for disabling indexing by google using SiteSetting.allow_index_in_robots_txt = false
Merge branch 'master' of github.com:discourse/discourse
rel nofollow, on by default to protect forums from spam etc. we should consider lifting it at high trust by default.
Merge branch 'master' of github.com:discourse/discourse