localwiki/localwiki — pull request #303 by philipn (2012-07-04 03:07:18+00:00)
Title: Fix history crawling

Needs review.

I can't figure out how to add that meta tag into the feeds right now. But this should do the trick. I tested this with the google webmaster tools:

https://scruzwiki.org/
https://scruzwiki.org/map/Lighthouse_Neighborhood_Park/_history/2008-07-12%2002:43:13.300000
https://scruzwiki.org/map/Lighthouse_Neighborhood_Park
https://scruzwiki.org/Lighthouse_Neighborhood_Park_history/
https://scruzwiki.org/map/Lighthouse_Neighborhood_Park/_history/_feed
https://scruzwiki.org/Recent_Changes/_feed/
https://scruzwiki.org/Recent_Changes/_feed

Url	Googlebot	Googlebot-Mobile
https://scruzwiki.org/
Allowed

Allowed
https://scruzwiki.org/map/Lighthouse_Neighborhood_Park/_history/2008-07-12%2002:43:13.300000
Blocked by line 2: Disallow: /*/_history/*

Blocked by line 2: Disallow: /*/_history/*
https://scruzwiki.org/map/Lighthouse_Neighborhood_Park
Allowed

Allowed
https://scruzwiki.org/Lighthouse_Neighborhood_Park_history/
Allowed

Allowed
https://scruzwiki.org/map/Lighthouse_Neighborhood_Park/_history/_feed
Blocked by line 2: Disallow: /*/_history/*

Blocked by line 2: Disallow: /*/_history/*
https://scruzwiki.org/Recent_Changes/_feed/
Blocked by line 8: Disallow: /*/_feed/

Blocked by line 8: Disallow: /*/_feed/
https://scruzwiki.org/Recent_Changes/_feed
Blocked by line 5: Disallow: /*/_feed$

Blocked by line 5: Disallow: /*/_feed$