Features/rt94833 (updates and tests)

Hi there!

I'm really looking forward to seeing this branch reach CPAN! The robots.txt patch is a great idea, and having a way to prevent/override sitemap routes is a personal favourite of mine :)

However, I understand your concerns, so I started writing a bunch of tests to make sure everything works as expected. I have added 41 tests and they all pass. I had to update some stuff as I wrote them, though, and even fixed some weird behaviour with the excluded routes, and made sure to document everything.

Hope this helps! Thanks again for building such a great plugin!