mikeal/request — Issue #98: Inline port numbers in URLs cause request to spawn multiple callback chains which loop "endlessly"
Author: rahrens
Created: 2011-10-26 10:57:34+00:00
State: open

I've been using request for a small project where we crawl some pages on a site I work on. We noticed an issue where request was entering an "endless" loop when attempting to follow a redirect.

Further investigation showed that

* request wasn't just looping until it hit options.maxRedirects. Instead it was looping until it hits options.maxRedirects, _options.maxRedirects + 1 times over!_
* The issue occurs at the point when request fetches http://app.compete.com/login from our server and receives a 301 response redirecting it to https://app.compete.com:443/login
* It's this second url, with an inline port which is actually confusing request. In fact, you don't even need to follow the redirect in order to observe the behavior -- you can just try to request https://app.compete.com:443/login directly.

I've attached some code demonstrating this behavior here: https://gist.github.com/1316008

I've also attached a sketch-of-a-patch. It appeared that by passing the same object around everywhere as options to callbacks that various callbacks were being run multiple times and that multiple chains of processing were running. Stopping that behavior by copying only the necessary properties into a new Object of properties given to the _http_ or _https_ module caused the problem to cease -- even when the same Agent is passed through.

To be honest, I'm submitting this in a gist not in a pull request because I don't really have a thorough enough understanding of _why_ this fixes things. That means I view it as a piece of diagnostic information not a viable fix in its own right.