kenshinx/second-spider — pull request #2 by jetz (2013-06-18 03:53:45+00:00)
Title: add time control for a long-time spider

If spider is blocked because of some unforeseen reasons, it should also be stopped after a fixed time, so adding timer for spider is necessary, here it can crawl at most 6 hours.