Also adding main/prod to disallow

Note that the root dir will still be crawled.