logsource: use stats_instant_inc_dynamic_counter() for HOST counters

Instead of open coding the implementation, use the function.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
logsource: track time for PROGRAM counters

syslog-ng tracks counters for each $PROGRAM it receives at stats-level(3),
add a last-update timestamp too.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
afinter: add inclusion to mainloop.h

Earlier this include was acquired through stats.h, but that header got
rid off that.
mainloop: remove old orphanage based stats cleanup

Previously statistic counters were cleaned up when they were orphaned,
at reload time. This is not really good for anything as at high stats
levels, syslog-ng can consume a lot of memory for random $PROGRAM, $HOST
values. Pruning these at reload time doesn't really solve the problem.

In subsequent patches this cleanup scheme is being changed to a timer
based approach that regularly cleans up counters that were updated
a long time ago.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
stats: rename stats_instance_inc_dynamic_counter() to stats_register_and_increment_dynamic_counter()

To make it easier to understand its purpose.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
stats: add stats_prune_old_counters()

This patch adds a new function that cleans up counters that were updated
the given time interval ago. It only cleans up dynamic entries, e.g.
counters that relate to $HOST, $PROGRAM and $HOST_FROM.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
stats-timer: implement stats_lifetime option

This patch implements a periodic timer that cleans up stats counters
after a specific lifetime. It defaults to 10 minutes.

Every 10 minutes, syslog-ng will walk the set of registered counters
and drops all that are older than the specific lifetime.

The user configurability is coming in a separate patch.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
cfg: add stats-lifetime() global option

This patch implements stats-lifetime() option that controls how long a
dynamic stats counter is alive. The setting is not exact, as syslog-ng will
actively check the whole set of counters every once in a while, and then
prune those that are too old. It may happen that some of the counters live
a bit longer than the stats-lifetime() setting.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
stats: decouple stats from GlobalConfig

Use a separate StatsOptions structure and separate grammar rules to
parse them. The stats subsystem became independent of the GlobalConfig
class, it just happens to use a member.

This patch also removes stats-timer.c as the timer requires access
to the global StatsOptions instance.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
stats: publish StatsCounter struct

Only to be used internall by the stats implementation.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
stats: rename StatsCounter to StatsCluster

This patch renames StatsCounter to StatsCluster that better describes its
purpose, as it encapsulates a cluster of related counters.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
stats: add accessor functions for source/destination/tag names
stats: add stats_foreach_cluster and stats_foreach_cluster_remove() functions

To make the internal stats counter hash accessible by other stats related
modules.
stats: extract code to generate CSV to a separate module
stats: migrate csv to stats_get_direction_and_source_name()
stats: extract syslog facility/severity statistics to a separate module
stats: extract StatsCounterItem into a separate module
stats: extract StatsCluster into a separate file
stats: use stats_cluster_foreach_counter() in stats_log

and remove access to the underlying arrays, except from the accessor
functions.
stats: extract Log statistics message to a separate module