try to fix spider bug.
Merge branch 'diffbot' of github.com:gigablast/open-source-search-engine into diffbot