parm updates
bulk api post truncation fix
parm updates
fix restart crawl
more bug fixes parmdb
parmdb fixes
fix cores
log cleanups mostly.
took out disk page cache,
kinda buggy... need to fix at some point.
rebuild url filters was not getting
called when some relavent parms were
updated.
fixes for getUrlFilterNum so
it looks at "hadReply" bit
in SPiderRequest when getting
diffbot api url.
final fixes for parms
wtf i did not modify these files.
Merge branch 'diffbot' of github.com:gigablast/open-source-search-engine into diffbot

Conflicts:
	CollectionRec.cpp
	Collectiondb.cpp
	Collectiondb.h
	Hostdb.h
	Parms.cpp
	main.cpp
fix big post of 70MB of urls
Merge branch 'diffbot' of github.com:gigablast/open-source-search-engine into diffbot
fix oom doleiptable using too much mem
so bulk job went oom
fix doledb empty logic
try to fix spiders from petering out.
reset doledb next keys and empty flags
every 3 minutes.
more fixing stuck spiders
minor print format fix