more spider fixes. still need more speedups
when spidering multiple spiders on same ip.