PPLAN-1493: Added robot.txt test