Not considering undefined response.request.urls as redirects
Sending whether or not the page was crawled to the onRedirect method when a redirect occurs
Executing finish callback so page was a redirect and already crawled (would otherwise not happen)
Rearranged some tests