added diffbot retry rules.
added maxTotalSpiders parm for
all colls to follow.
tried to fix msg 0x00 socket jam up.