packet handling: fix release function

Extended data were freed before the release function was called.
The result was that, in AF_PACKET IPS mode, the release function
was only sending void data because it the content of the extended
data is the content of the packet.

This patch updates the code to have the freeing of extended data
done in the cleaning function for a packet which is called by the
release function. This improves consistency of the code and fixes
the bug.
dns: improve response name parsing

Improve parsing of names with multiple pointers following each other.
byte-extract: switch to pcre_copy_substring

Fixes memory leaks (str_ptr was never freed).
Fixes cppcheck warning:
    [src/detect-byte-extract.c:268]: (error) Uninitialized variable: str_ptr

Bug #1046.
byte-test: switch to pcre_copy_substring

Fixes cppcheck:
[src/detect-bytejump.c:360]: (error) Uninitialized variable: str_ptr
[src/detect-bytejump.c:372]: (error) Uninitialized variable: str_ptr
[src/detect-bytejump.c:375]: (error) Uninitialized variable: str_ptr

Bug #1046.
stream: fix midstream syn/ack setup

If midstream is enabled and the first packet is the syn/ack packet from
the 3whs, initialized server.last_ack to the packets seq.

This fixes tracking the session.
protocol detection: midstream handling update

If a TCP session is midstream, we may end up with a case where the
start of an HTTP request is missing. We won't detect HTTP based on
the request. However, the reply is fine, so we detect HTTP anyway.
This leads to passing the incomplete request to the htp parser.

This has been observed, where the http parser then saw many bogus
requests in the incomplete data. This is not limited to HTTP.

To counter this case, a midstream session MUST find it's protocol
in the toserver direction. If not, we assume the start of the
request/toserver is incomplete and no reliable detection and parsing
is possible. So we give up.
proto detect: add cutoff for unbalanced traffic

If we're getting a lot of data in one direction and the proto for this
direction is unknown, proto detect will hold up segments in the segment
list in the stream. They are held so that if we detect the protocol on
the opposing stream, we can still parse this side of the stream as well.
However, some sessions are very unbalanced. FTP data channels, large
PUT/POST request and many others, can lead to cases where we would have
to store many megabytes worth of segments before we see the opposing
stream. This leads to risks of resource starvation.

In this patch, a cutoff point is enforced. If we've stored 100k in one
direction and we've seen no data in the other direction, we give up.

If we've given up, the applayer_proto_detection_skipped event is set.
    app-layer-event: applayer_proto_detection_skipped;
stream: detect data gap at stream start

In midstream mode we may encounter a case where the data we is beyond
the isn, but below last_ack. This means we're missing some data, that
is already acked so it won't be retransmitted. Therefore, we can
conclude it's a data gap.
stream: improve midstream reassembly gap detection

The reassembly gap detection makes use of the window. However, in
midstream mode the window size we use is unreliable, as we have to
assume window scaling is in place. This patch improves midstream
handling of those cases.
protocol detection: handle very unbalanced case

Some traffic is very unbalanced. For example a 4 bytes request
followed by 12mb of response data. If the 4 bytes don't lead to
the protocol being detected, then app layer processing won't
start, but it will not give up either. It will just wait for more
data. This leads to piling up data on the opposite side.

Observed:

TS: 4 bytes. PP failed (bit set), PM has not given up (bit not set).
    This makes sense as max_depth is not yet reached.

TC: 12mb. PP and PM failed (bits set).

As ts-PM never gives up, we never consider proto detect complete,
so all segments in either direction are still kept in the session.

This patch adds a cutoff for this case:
- IF for TS we have PP bit set, PM not set, AND
- we have TC both bits set, AND
- proto is unknown, AND
- TC data is 100k already, THEN
- give up on protocol detection.

The same for the opposite direction.
stream: flags cleanup

Stream flags are 16bit, but notation is still 8bit. Clean this up to
avoid confusion.
stream: improve StreamTcpPruneSession

Check if a segment is done for the app-layer using
StreamTcpAppLayerSegmentProcessed instead of the flag directly so the
gap case works better.
stream: fix raw reassembly flag issue

AppLayer reassembly correctly only flags a segment as done when it's
completely used in reassembly. Raw reassembly could flag a partially
used segment as complete as well. In this case the segment could be
discarded early. Further reassembly would miss data, leading to a
gap. Due to this, up to 'window size' bytes worth of segments could
remain in the session for a long time, leading to memory resource
pressure.

This patch sets the flag correctly.
stream: update GAP detection

Change GAP detection logic. If we encounter missing data before
last_ack, we know we have missed data. The receiving host has ack'd
it already, so a retransmission of the missing data is highly
unlikely.
stream: improve retransmission handling

Improve retransmission handling in the CLOSE_WAIT state.

Bug #1180.
json-file: check pointer before using

A check was missing to see if ht_ud was not null before using the
pointer. This should be rare, but it can happen.

Reported-by: Will Metcalf
json-file: improve error handling

If the functions getting uri, etc fail. Return "unknown" instead of
NULL pointer. This improves consistency.
DAG: Sync dag packet and drop counts to live device on exit for better exit
logging.
DAG: Consistent code style.
DAG: Pull some raw values out into defines.