Bunch of efforts, inspect robots.txt, attempt to use cookies to allow crawl
Merge branch 'master' of https://github.com/iamliamc/grcourt