fix retrying error forever pointlessly
when msg4 request is corrupt
fix long-standing core when getting linkinfo
from a collection that got nuked.
try to fix spiders not going.
try to fix another core.
computing link text is slow for some reason.
needs to be looked at, but take out for now
for custom crawls to speed things up.
fix core from corrupt spider request
fix a core when deleting a coll while
saving its doledb. fix it right actually.
fix core trying to roundstart=1 a coll
that does not exist
fix sockets never closing bug when client
deletes then queries a collection.
fix another spiderdb corruption core
do not do any simplified redirects for custom
crawls, just like bulk jobs.
fix oops
Merge branch 'diffbot-testing' into testing