Added 'Disallow' rules to 'RobotsTxt' class, 'Allow' to be dealt with another time if needed!