re-enable spidering if off because we missed
a crawlinfo reply from udp timed out.
Merge branch 'diffbot-testing' of github.com:gigablast/open-source-search-engine into diffbot-testing
simple fix for &token=... searches
if no &token= or &c= then use default collnum for searching
add some debug msgs
fix stack smash core when title is huge.
Merge branch 'diffbot-testing' of github.com:gigablast/open-source-search-engine into diffbot-testing
ask all hosts for ALL crawl info for ALL collections
when we first startup. pass in 'f' flag. use 'i' normally
to indicate 'incremental' to only send crawlinfos that changed.
fix for bad crawl info stats
turn off stack smash detection so it will get a seg fault and
save and dump core when stack gets smashed.
Merge branch 'diffbot-testing' into testing