Fix bug in fetcher, as robots.Test require the requestURI
Allow TestBasicRobotsDisallow to run, since code properly checks robots
Merge pull request #109 from LongsightGroup/robots-requesturi

Fetcher robots.Test needs URL.RequestURI, not string(URL)
Merge branch 'master' into TRN-135-BugFix