make &s=20000 search much faster by skipping
generation of first 20000 summaries if
deduping is off, site clustering is off and
gigabit generation are off (&dr=0&sc=0&dsrt=0).
turn gigabits off on load for all customcrawls(diffbot)
infinite loop fix for injecting
only send localcrawlinfo if it has
been updated significantly since last time.
should remove the sluggishness/missedhearbeats
from host #0 on neo.
fix type:json bug to not return non-diffbot reply docs.
upped query max words for large client bool query.
Merge branch 'diffbot-testing' of github.com:gigablast/open-source-search-engine into diffbot-testing
fix core
update crawl info once per sec again
now that we only send if localCrawlInfo
has changed.
try to remove the sluggishness from
all hosts... should really reduce load.
remove debug log comments
fix infinite loop. why wasn't this pulled in
from master?
Merge branch 'testing' into diffbot-testing

Conflicts:
	PageInject.cpp
Merge branch 'diffbot-testing' into diffbot

Conflicts:
	PageInject.cpp