Spider: Major improvements to error tracking and support for 3xx and 4xx errors