afmongodb: Fix a deadlock at deinit time.

There is a race condition between the writer thread and stop_threads
(via deinit): if deinit sends the signal just at the wrong time, then
the worker thread willl never wake up, as the main thread is already
waiting for a join to finish.

The fix is to lock the queue_mutex before signalling, that eliminates
the race. This patch does just that.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
afsocket: handle DNS failures as if it was a connection failure

Instead of preventing the startup completely.

Reported-By: Heiko Gerstung <heiko@am-anger-1.de>
Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
afinter: Make the parent class LogSrcDriver, instead of LogDriver.

The parent of AFInterSourceDriver should be LogSrcDriver, because
that's the struct we pass down to it anyway, and other parts of the
code will cast it back to LogSrcDriver, too.

But if its parent is LogDriver, as it was, we were corrupting the
later parts of the original LogSrcDriver, with values from
AFInterSourceDriver.

Many thanks to Peter Eisenlohr <peter.eisenlohr@inform-software.com>
for initial debugging, and for pinpointing mark messages as the
triggers.

Reported-by: Peter Czanik <czanik@balabit.hu>
Signed-off-by: Gergely Nagy <algernon@balabit.hu>
Merge remote-tracking branch '3.3/master'
test_value_pairs: fixed heap corruption & memory leak

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
afinter: fixed a compilation warning

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
cfg-tree: fixed memory leaks in config compilation routines

The objects hash and the name for rules were leaked when processing
the configuration. This patch fixes those.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
cfg-lexer: fixed a memory leak that caused the linked list that holds generator plugins to be leaked

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
cfg-grammar: fixed memory leak when parsing "block" definitions

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
use on-demand LogMessage cloning to implement inmutable messages on parallel log paths

When processing a message, syslog-ng tries hard to avoid copying messages
along the line. The semantics provided by the syslog-ng configuration
is that parallel branches of the log processing pipeline are independent,
e.g. change on one of the branches may not affect the other one.

Earlier, this was implemented by the LogMultiplexer object, before
posting a message to a branch it checked if the branch may change
the message by inspecting the PIF_CLONE flag of the head element. This
didn't handle all cases (e.g. when a rewrite was applied between two
destinations).

This patch delegates this clone-when-needed operation right before the
actual change happens, this also allows some performance improvements
(since the clone may not always be needed).

I introduced the notion of "write-protection" to LogMessage instances,
when an msg is write-protected no changes can be applied. If a LogPipe
needs write access, it needs to acquire that by calling
log_msg_make_writable().

Write-protection is enabled on a LogMessage when it is assumed not to
change. Such an occassion is when it is put on a LogQueue of a destination
which processes messages asynchronously. A similar case is when the
db-parser() puts the message in its correllation state.

Before any writes can be applied, a call to log_msg_make_writable()
needs to be made, which creates a clone if the current LogMessage
is write-protected, or leaves the message be if it isn't.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
preparations for 3.4.0alpha2
configure.in: Add --disable-smtp support.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
afsql: Move the database connection to a separate function

To make the code clearer, move the code to connect to a database out
of afsql_dd_insert_db() into afsql_dd_connect().

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
[PATCH (3.4) 2/3] afsql: Simplify insert_db()'s table validation

To make things a tiny bit easier, move the final error handling out of
_insert_db() into a separate function: afsql_dd_insert_fail_handler().

Furthermore, change afsql_dd_validate_table() to allocate the GString
for the table name itself, and resolve the template too, if so need
be.

The above two allows us to simplify afsql_dd_insert_db() a little:
when validating the table, we can immediately bail out, without a
goto, since we only need to free up the bare minimum.

And in the end, a monstrous if (success) {} else {} branch is replaced
by something a lot shorter, and clearer.

All of these together make the flow of afsql_dd_insert_db() easier to
understand and follow.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
afsql: Move the INSERT construction out of insert_db()

To improve the flow of the afsql_dd_insert_db() function, moved the
code to construct the insert statement out into a new function:
afsql_dd_construct_query().

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
afsmtp: Add SCS_SMTP to lib/stats.h

Add SCS_SMTP to lib/stats.h, so we get proper smtp stats too.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
cfg-tree: handle empty channel {} blocks

Earlier an empty channel {} was considered a compilation error, these
are now handled as a do-nothing pipe.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
logmsg: publish log_msg_set_tag_by_id_onoff()

This function used to be private, now it is published as it is needed by
set-tag() and clear-tag().

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
rewrite: added set-tag() and clear-tag() ops

This patch implements two rewrite primitives, set-tag() takes a tag
and associates it with the log message. clear-tag() does the reverse thing.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
json-parser(): added marker() option

This patch implements a marker() option for json-parser(). Marker is
a string prefix that can be used to identify a message to be parsed. If
the marker is set and the incoming message doesn't start with it,
parsing is not performed and the message is dropped.

If the input indeed matches the marker, json parsing commences with the
string _after_ marker. White space between the marker and the json
payload is ignored.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>