Add failing test with bad message

We incorrectly state that some errors were "Caused by redirect" when they aren't, for example if you retry and get a 500 a few times in a row. 

This adds a failing test to demonstrate this. Not sure how you'd like to handle it, I don't have great suggestions at the moment but I'll keep looking.
If you ever catch yourself writing `except Exception:`, there's a 98% chance you're doing something wrong. :)