checkpoint for new parm logic
to allowing syncing with newly added or deleted
collections even if a host was dead when collection
was added/deleted. also added parm change request
queueing.
Merge branch 'diffbot' of github.com:gigablast/open-source-search-engine into diffbot

Conflicts:
	Process.cpp
another parm overhaul checkpoint
minor comments update.
Merge branch 'diffbot' of github.com:gigablast/open-source-search-engine into diffbot
new parm stuff almost compiling.
got this new parm shit compiling
test commit
parmdb overhaul. support collection add/del
sync when host comes back online. use udp not tcp.
host #0 can now handle a new incoming request while
a parm change is currently outstanding.
all missed "command" parms will be received when a dead host
comes back online, too, like a tight merge for instance.
does not use msg4, uses msg3e and msg3f for syncing and
sending parms.
do not accept msg4 add requests until in sync with host 0
move CollectionRec stuff into Collectiondb files
for simplicity.
fix core in addExistingColl()
clean up logging so i can see what's going on
some parm overhaul bug fixes
parmdb fixes
more parm overhaul fixes
Merge branch 'diffbot' into diffbot-testing

Conflicts:
	CollectionRec.cpp
	CollectionRec.h
	Collectiondb.cpp
	Pages.cpp
	Parms.cpp
bring in changes from live beta from ~/github.
limit spiders to 50, not 500 to prevent oom.
resume killed merges that had num files shrunk even
if down to one file. show collnum in spider queue.
remove back-to-back whitespace, and make all space
a ' ' for getting the doc checksum for deduping.
Merge branch 'diffbot' into diffbot-testing

Conflicts:
	iana_charset.cpp
	iana_charset.h
Merge branch 'master' into diffbot