switch to third-party robots.txt library