avoid waiting for the spider to be idle before popping the redis queue

This will increase the throughput of your spiders significantly.