Spider algorithm changed:
- refresh http context every 500 requests (or even less)
- refresh page every 500 requests
- now update routes every 3 minutes

And, yes, there are some vars now in the Spider object. It was completely necessary evil.