Link to SSL-using modules to libsyslog-ng-crypto too

Since OpenSSL needs some initialization to work properly, link afql to
libsyslog-ng-crypto to do that. If we don't do this, then loading
afsql (or any other module that uses OpenSSL) before anything that uses
libsyslog-ng-crypto (like afsocket) will fail and result in a
segmentation fault.

Affected modules are: afsql, afmongodb and afsmtp.

Reported-by: Imre Lazar <imre@balabit.hu>
Signed-off-by: Gergely Nagy <algernon@balabit.hu>
syslogformat: Only set PROGRAM to kernel when the message is local

When the message is not local, do not override - or set - the value of
PROGRAM, there's other ways to do that. Doing an unconditional setting
breaks setups where kernel messages are forwarded between syslog-ng
instances:

The first syslog-ng sees "<4>Some message..." on /proc/kmsg, sets
PROGRAM to "kernel", and sends it over on tcp as "<4>kernel: Some
message...". The second syslog-ng sees that, from the facility it sees
that it's a kernel message, and sets PROGRAM to kernel again.

But when a message is from the kernel, we don't parse the rest, we don't
extract the program, we treat the rest of the message as MESSAGE, so in
this case, we'd end up with "<4>kernel: kernel: Some message", and each
hop would add another "kernel: ".

If we only set this when the message is local, we avoid this. The
downside is, that on the remote end, PROGRAM will not be set.

Reported-by: Matyas Koszik <koszik@atw.hu>
Signed-off-by: Gergely Nagy <algernon@balabit.hu>
coverage: Removed running of firefox in coverage collector script

Because it prevents coverage generation on a non-desktop machine...
syslog-ng-ctl/Makefile.am: Include more files in make dist

make dist requires control-client.h and control-client-unix.c to be
included as well, add them to the appropriate variables.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
affile: affile_open_file() refactoring

This splits up the large affile_open_file() function into smaller, more
readable private functions, that makes it easier to understand the
control flow.

Signed-off-by: Budai Laszlo <lbudai@balabit.hu>
Signed-off-by: Gergely Nagy <algernon@balabit.hu>
redis: Simplify redis_worker_insert()

Instead of having a number of branches and formatting the Redis command
with redisCommand(), use redisCommandArgv(). This flattens the flow a
little and eliminates a needless string parsing & formatting step.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
redis: Turn self->command into a GString

Since we need the length of the command on every insert, turn it into a
GString, so we will always have the length readily available.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
cfg-tree: fix node flag propagation

This patch fixes the application of node flags during compilation. Earlier
the node flags were either applied to the first_pipe (normal log paths
lacking a source reference) or the last_pipe (log paths that did contain
source references).

The issue is that due to the compilation of junctions, the last_pipe may
not be reachable through LogPipe->pipe_next list, which is iterated in
log_multiplexer_init() when looking for LC_XXX flags. Rather they are
only reachable through an intermediate LogMultiplexer's next_hop list.

This causes the flags to be missed when evaluating the compiled structure.

This patch tries to apply flags to the first LogPipe ever encountered while
compiling the sequence and only resort to the last_pipe if we don't encounter
anything that would start with a sub_pipe_head.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
tests/functional: validate that no additional messages are delivered to a destination

This patch adds validation that no extra messages get delivered to
a destination.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
tests/functional: added tests for final flag after embedded log statements

The final flag was incorrectly processed for log statements where it was
specified after a set of embedded log statements.

This adds a testcase that validates that.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
logsource: use stats_instant_inc_dynamic_counter() for HOST counters

Instead of open coding the implementation, use the function.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
logsource: track time for PROGRAM counters

syslog-ng tracks counters for each $PROGRAM it receives at stats-level(3),
add a last-update timestamp too.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
afinter: add inclusion to mainloop.h

Earlier this include was acquired through stats.h, but that header got
rid off that.
mainloop: remove old orphanage based stats cleanup

Previously statistic counters were cleaned up when they were orphaned,
at reload time. This is not really good for anything as at high stats
levels, syslog-ng can consume a lot of memory for random $PROGRAM, $HOST
values. Pruning these at reload time doesn't really solve the problem.

In subsequent patches this cleanup scheme is being changed to a timer
based approach that regularly cleans up counters that were updated
a long time ago.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
stats: rename stats_instance_inc_dynamic_counter() to stats_register_and_increment_dynamic_counter()

To make it easier to understand its purpose.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
stats: add stats_prune_old_counters()

This patch adds a new function that cleans up counters that were updated
the given time interval ago. It only cleans up dynamic entries, e.g.
counters that relate to $HOST, $PROGRAM and $HOST_FROM.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
stats-timer: implement stats_lifetime option

This patch implements a periodic timer that cleans up stats counters
after a specific lifetime. It defaults to 10 minutes.

Every 10 minutes, syslog-ng will walk the set of registered counters
and drops all that are older than the specific lifetime.

The user configurability is coming in a separate patch.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
cfg: add stats-lifetime() global option

This patch implements stats-lifetime() option that controls how long a
dynamic stats counter is alive. The setting is not exact, as syslog-ng will
actively check the whole set of counters every once in a while, and then
prune those that are too old. It may happen that some of the counters live
a bit longer than the stats-lifetime() setting.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
stats: decouple stats from GlobalConfig

Use a separate StatsOptions structure and separate grammar rules to
parse them. The stats subsystem became independent of the GlobalConfig
class, it just happens to use a member.

This patch also removes stats-timer.c as the timer requires access
to the global StatsOptions instance.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
stats: publish StatsCounter struct

Only to be used internall by the stats implementation.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>