yaml: add eve flow and netflow entries

Added, commented out by default.

Bug #1257.
output-lua: add script-dir config param

Add 'scripts-dir' config directive that is prepended to the script
names to form a path. If ommited or empty, script are opened from
the CWD.
output-lua: add config to yaml

Disabled by default.
output-lua: set proper callbacks for HTTP

Enable the relevant HTTP callbacks.

Bug #1287
flow: fix flow logging at shutdown

Move all flows from the hash to the recycler at shutdown.

Bug #1260
flow json log: add 'shutdown' as flow end reason

When engine shuts down all flows in the hash are logged out. They
may not have timed out yet. So they are forced. Log the reason to
be 'shutdown'.
MPM: build sid list from MPM matches

Pmq add rule list: Array of uint32_t's to store (internal) sids from the MPM.

AC: store sids in the pattern list, append to Pmq::rule_id_array on match.

Detect: qsort rule_id_array after it was set up by the MPM. Rule id's
(Signature::num) are ordered, and the rule's with the lowest id are to
be inspected first. As the MPM doesn't fill the array in order, but instead
'randomly' we need this sort step to assure proper inspection order.

**TODO**:

Currently we use a new array 'rule_id_array'. However, in a few places,
the 'pattern_id_array' is used in a similar way (proto detect and perhaps
smtp?). So we could convert those to use the rule_id_array and remove the
pattern_id_array completely.
Detect: create per sgh non-MPM rule array

Array of rule id's that are not using MPM prefiltering. These will be
merged with the MPM results array. Together these should lead to a
list of all the rules that can possibly match.
Add negated MPM to non-MPM array (HACK)

Treat negated MPM sigs as if non-MPM, so we consider them always.

As MPM results and non-MPM rules lists are now merged and considered
for further inspection, rules that need to be considerd when a pattern
is absent are caught in the middle.

As a HACK/workaround this patch adds them to the non-MPM list. This
causes them to be inspected each time.

**TODO**
This will likely perform bad if there are a lot of rules using negated
MPM's. In the ET/ETpro rulesets those rules are quite rare though.
Replace build match array with new filter logic

Use MPM and non-MPM lists to build our match array. Both lists are
sorted, and are merged and sorted into the match array.

This disables the old match array building code and thus also bypasses
the mask checking.

**TODO**
See if we should reintroduce parts of the mask and other prefilter
logic.

**TODO**
Having a zero sized list is quite common for both lists, so we can
probably add some optimization for this.
Add detect debug code

Add some expensive code to print the input and output of the
lists merging function.

**TODO**
Remove before merging this branch.
detect: move checks from prefilter to rule detect

Move the prefilter checks to the main detect loop.

**TODO**
Review/test to what extend this is necessary.
AC: use local bit array (HACK)

Use a local pattern bit array to making sure we don't match more than
once, in addition to the pmq bitarray that is still used for results
validation higher up in the rule matching process.

Why: pmq->pattern_id_bitarray is currently sometimes used in a
'stateful' way, meaning that for a single packet we run multiple
MPM's on the same pmq w/o resetting it.

The new bitarray is used to determine wherther we need to append the
patterns associated 'sids' list to the pmq rule_id_array.

It has been observed that MPM1 matches for PAT1, and MPM2 matches for
PAT1 as well. However, in MPM1 PAT1 doesn't have the same sids list.
In this case MPM2 would not add it's sids to the list, leading to missed
detection.

**TODO**

Consider alternative solutions: e.g. make sure that each MPM runs
with it's own pmq. Then merge the pmq's using PmqMerge. Might perform
better than the stack alloc'd + memset'd bitarray in use now.
Fix AC-tile for new pattern ID array.
Implement MPM opt for b2g, b3g, wumanber

Found problems in b2gm and b2gc, so those are removed.
Fix b2g unit test failures for new mpm opt

The sid data was not getting copies into the hash item. Should really
work more like b3g where hash item just stores a pattern index.

Also copy the SIDs into the output match array so they get reported.
Remove ac-bs and ac-gfbs MPM

The headers on these don't say how they are different, but appear to use
relative state values instead of direct state index for next state table.

It is not clear they have been kept up to date with AC.

Didn't want to update these to use new SID opt.
Optimize DetectPrefilterMergeSort
Fix clang warning

Clang doesn't seem to like defining a function within a function.
fixup gfbs include