openstreetmap/openstreetmap-website — pull request #761 by stefanb (2014-06-16 19:04:49+00:00)
Title: Updated robots.txt for new routes (no more /browse)

Updated for changes done in
3cd5f45e08d977d04a778ab8802f71df85edc314 and
61bb31ebdddc9e7c317a1c6784a68874d5a0ef80

Test search queries listing the pages that were not suppose to be indexed by robots:
https://www.google.si/search?q=site%3Awww.openstreetmap.org+inurl%3Auser+inurl%3Ahistory
https://www.google.si/search?q=site%3Awww.openstreetmap.org+inurl%3Anode+inurl%3Ahistory