Fixing exception handling for Robots and sitemap parsing