system-source: Make /proc/kmsg optional

In certain environments (vserver and OpenVZ come to mind), /proc/kmsg is
not readable, not even for root. On these systems, emit a warning, and
skip it, instead of aborting the startup.

Reported-by: Tamas Pal <folti@balabit.hu>
Signed-off-by: Gergely Nagy <algernon@balabit.hu>
cfg-args: split off from cfg-lexer.c

This is a preparatory patch that slices off functionality from cfg-lexer.c
to make it easier to write unit tests for.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
Signed-off-by: Gergely Nagy <algernon@balabit.hu>
cfg-lexer.h: added GError based error handling boilerplate

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
cfg-lexer: added cfg_lexer_free_token() function

The role of this function is to free any type of token returned by the
lexer. Do note however that the YYSTYPE struct itself is managed
by the caller, only its contents are freed.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
cfg-lexer: implement proper escape handling in backtick insertion

This patch implements escaping when inserting a value using
backticks. When backticks are used to insert values within string
and qstrings and the value itself is a syslog-ng.conf string literal
we need to inject its value, adapting the escaped characters.

While implementing this function, I also did:
 - moved to cfg_lexer_subst_args() to a separate module
 - added GError based error handling
 - unit tests

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>

Note: I removed the unit tests from the backport, due to them being
harde to backport aswell.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
logqueue: remove log_queue_assert_output_thread()

When explicit-commits is in effect, SQL destination attempts to commit
the INSERT SQL commands when the number of uncommitted inserts reaches
the limit set by flush_lines. However, when the input queue of the SQL
destination becomes empty, we do not want uncommitted INSERT commands
to hang in the air, so we register a callback to be notified when new
log messages become available and commit the last batch of them,
regardless of flush_lines settings. When this happens, then the implicit
condition log_queue_assert_output_thread() depended upon cannot be
satisfied, and not because of a programming mistake.

We could work around this limitation of that assertion, but the thing
is that this assertion is not very clean, and though it had caught a lot
of programming errors in the past, it may generate false positives in
case of destinations like SQL (and because of the lack of proper locking
around it, it may miss some real problems).

Long story short, we decided to remove it entirely, while ensuring that
all the methods of logqueue-fifo that assume to be running in the output
thread of simple log destinations are documented.

Signed-off-by: Attila M. Magyar <athos@balabit.hu>
Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
LogQueue: remove the ability to check for partially fulfilled batches

TBD
afsql: allocate locks/condvars at startup

Instead of doing this for all start_thread/stop_thread pairs, allocate
them once to avoid churn.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
sql: migrate to use the log_queue_check_items() API

This patch changes the SQL driver to use the log_queue_check_items()
API instead of its own locking around the LogQueue instance.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
afsql: flush queue at exit
smtp: migrate to use the log_queue_check_items() API

This patch changes the SMTP driver to use the log_queue_check_items()
API instead of its own locking around the LogQueue instance.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
mongodb: migrate to use the log_queue_check_items() API

This patch changes the MongoDB driver to use the log_queue_check_items()
API instead of its own locking around the LogQueue instance.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
amqp: migrate to use the log_queue_check_items() API

This patch changes the AMQP driver to use the log_queue_check_items()
API instead of its own locking around the LogQueue instance.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>