Correctly parse robots.txt with repeated user-agent or no trailing \n