An easy to use robots.txt parser