django/djangoproject.com — pull request #43 by dlo (2012-09-07 03:30:39+00:00)
Title: disallow search engine indexing of trunk documentation

The end result of this data being indexed confuses newbies and is the source of
an increasing number of invalid tickets in trac. More specifically:

1. User searches to find info on a specific feature.
2. Said feature is backwards incompatible with previous versions of Django.
3. User does not know better, assumes Django has a bug.
4. Invalid bug is filed in trac.

Hopefully, removing the dev documentation pages from Google will help solve
this issue, since those who want to read the dev docs can just click through to
them by using the version navigation at the bottom of the docs pages.

BTW, I talked with Alex earlier today about this--he suggested redirecting users
as well from dev -> 1.4 (or whatever the latest version is). I initially thought it was
a good idea, but I thought about it and realized you would have to at least add
some sort of referrer check if this was server side. I then went down a rabbit hole
of whether it would be ok to do something like

def conditional_documentation_redirect(request):
if not request.META['REFERER'].startswith("http://docs.djangoproject.com"):
return HttpResponseRedirect #... and so on

This felt wrong to me because if that sort of check were in place, users would
no longer be able to permalink to dev documentation. Anyways, I'll post a ticket
on trac with this PR.