fixed ability to spider multiple urls from the
same IP at the same time. Also respects
sameIpWait constraints.