Merge branch 'master' into topic/jsiwek/file-analysis
Merge remote-tracking branch 'origin/master' into topic/bernhard/sqlite

Conflicts:
	src/threading/AsciiFormatter.cc
make work with newer AsciiFormatter.
fix small bug with vectors and sets.

On a first glance - this kind of seems to work. On mac-os you need
a newer than the system-installed sqlite - the hanging problem only
occurs with that one...
add sqlite tests and fix small vector/set escaping bugs
FileAnalysis: move unique file handle string generation to script-layer

And add minimal integration with HTTP analyzer.
FileAnalysis: small tweak to file handle generation.

For files that go over a single connection, add connection start time
to handle, so the file id will always differ even if the same connection
parameters are later used to transfer a file (same one or different).
Merge remote-tracking branch 'origin/master' into topic/seth/metrics-merge

Conflicts:
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/scripts.base.frameworks.notice.cluster/manager-1.notice.log
	testing/btest/Baseline/scripts.base.frameworks.notice.suppression-cluster/manager-1.notice.log
	testing/btest/Baseline/scripts.base.protocols.ftp.gridftp/notice.log
FileAnalysis: decentralize unique file handle generator callbacks.

The framework now cycles through callbacks based on a table indexed
by analyzer tags, or the special case of service strings if a given
analyzer is overloaded for multiple protocols (FTP/IRC data).  This
lets each protocol script bundle implement the callback locally and
reduces the FAF's external dependencies.
Added Exec, Dir, and ActiveHTTP modules.
Checkpoint
FileAnalysis: buffer input that can't get unique file handle immediately

A retry happens on every new input and also periodically based on a
timer.  If a file handle is returned at those times, the input is
forwarded for analysis, else it keeps retrying until a timeout
threshold.
FileAnalysis: change file handle -> file id mapping process.

They're now actually directly related via a hash function that will
produce the same results among different instances in a cluster.
Support for cleaning up threads that have terminated.

Once a BasicThread leaves its run() method, a thread is now marked for
cleaning up, and the ThreadMgr will soon join it to release the OS
resources.

Also, adding a function Log::remove_stream() that remove a logging
stream, stopping all writer threads that are associated with it.

Note, however, that removing a *filter* from a stream still doesn't
clean up any threads. The problem is that because of the output paths
potentially being created dynamically it's unclear if the writer
thread will still be needed in the future. We could add clean writers
up with timeouts, but that doesn't sound great either. So for now, the
only way to sure clean up logging threads is to remove the entire
stream.

Also note that cleanup doesn't work with input threads yet, which
don't seem to terminate (at least in the case I tried).
FileAnalysis: improve HTTP integration.

Seems to be working with various examples I tested, now including
multipart/byteranges.
change raw reader to use basic c io instead of fdstream encapsulation class.

preparation for future changes.
replace popen with fork and exec.

Note for future: eof only gets sent when the parent closes its in-pipe
after forking.
Streaming reads from external commands work without blocking anything.
allow sending data to stdin of child process
Changing semantics of thread stop methods.

PrepareStop() is now SignalStop() and just signals a thread that it
should terminate. After that's called, WaitForStop() (formerly Stop())
wait for it to actually finish processing.

When stopping writers during operation, we now no longer wait for them
to finish.