postmodern/spidr — Issue #31: Infinite path loop
Author: ethicalhack3r
Created: 2012-02-01 08:57:29+00:00
State: open

Hi,

It seems that when $_SERVER['REQUEST_URI'] or similar is used AND the web server is configured to return custom error pages (including 200 statuses), Spidr ends up in an infinite loop.

In this particular case the problem URL is in a POST form action element, but I don't think it matters where the URL appears.

Eventually ends up with pages like so:

http://www.example.com/dir/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/js/somefile.js

I'm not sure how this could be solved, the depth option may help cut down on the false positive URLs but wouldn't solve the problem.

Thanks,
Ryan