Feed text content instead of binary to robots.txt parser.