Updated robots.txt to include some more CKAN pages to not be indexed.

We really don't want /qa/linkchecker to be indexed as it will just end up generating emails for me, and the resource_cache should *never* be indexed as we don't want them to turn up in a search engine as they are often (deliberately) out of date pages.
Merge pull request #125 from rossjones/master

Updated robots.txt to include some more CKAN pages to not be indexed.