log_template_compile: function refactorized

This patch adds a LogTemplateCompiler that encapsulates the template
compilation process that is independent from actually formatting (or
evaluating) the template.  This reduces the number of parameters to parsing
functions and makes the code easier to follow.

The next possible step should be, that split the templates.c
file into more files.

This patch based on the log-template-refactor branch

Signed-off-by: Juhasz Viktor <jviktor@balabit.hu>
Add support for number unit suffixes

There are a number of cases in the configuration where one needs to set
a size (think buffer sizes), and counting zeroes is extremely annoying
and unintuitive. For this reason, we introduce number unit suffixes for
kilo-, mega- and giga-: K, M, G, respectively (case-insensitive,
optionally followed by a 'b' or 'B', for byte). The parser also
understands KiB, MiB, etc to indicate base-two units.

Anywhere where you can specify a number, you'll be able to specify one
with a unit suffix from now on:

 log-fifo-size(200M)

A function to parse longs already existed in basic-funcs (tf_parse_int),
which was moved to lib/parse-number.c now, and enhanced to recognise the
unit suffixes too.

This also closes #28.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
Signed-off-by: Gergely Nagy <algernon@balabit.hu>
templates: fixed compilation error

This patch loosens coupling between LogTemplateCompiler and
LogTemplate and fixes a compilation error at the same time.

Signed-off-by: Balazs Scheidler <bazsi@balabit.hu>
filter: Implement an in-list() filter

The in-list filter allows us to do simple file-based black- or
whitelisting. The syntax is:

 filter f_whitelist { in-list("/path/to/file.list", value("PROGRAM")); };

Implementing a blacklist is as simple as negating the in-list() filter.

The filter works by pulling in the whole file (one entry per line) and
putting it into a tree structure, and comparing against that on eval.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
basicfuncs: Add upper-case and lower-case functions

This adds two new template functions: $(upper-case) and $(lower-case)
which will upper- or lower-case their UTF-8 arguments. Based on code
from Chris Johnson <chris.johnson3@hp.com>

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
basicfuncs: Add a new $(delimit) template function

The $(delimit DELIMITERS NEW-DELIMITER HAYSTACK) template function takes
three arguments, and replaces all the delimiters from DELIMITERS in the
HAYSTACK with the NEW-DELIMITER.

For example, $(delimit x | fooxbar) would result in "foo|bar".

Based on code and ideas from Chris Johnson <chris.johnson3@hp.com>.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
basicfuncs: Add an $(env) template function

This new template function can be used to look up environment variables
and insert them into the string.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
lib/: Install header files to their proper place

With header files in subdirs of lib/, they were getting installed to
${pkgincludedir} with the extra directory parts stripped. This patch
corrects that, so headers end up where they should.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
Mk/Makefile.am: Install merge-grammar.pl as executable

Instead of installing merge-grammar.pl as data, install it as a script
so it becomes executable.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
lib/merge-grammar.pl: Work from outside of the syslog-ng tree

Try opening cfg-grammar.y in the same directory the script is in, too,
so that the script will work when installed, and not only from the
syslog-ng source tree.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
Mk/lex-rules.am: Make it work when installed too

Instead of referencing $(top_srcdir)/lib, use $(syslog_ng_tools) instead
in Mk/lex-rules.am, so that this file can be reused by external projects
too.

Define $(syslog_ng_tools) to $(top_srcdir)/lib in the top-level
Makefile, so that the same file will continue working in the syslog-ng
source tree too.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
stomp: New STOMP destination driver

Based on initial work by Attila Nagy <bra@fsn.hu>, this destination adds
support for sending events to any STOMP-capable message queue. Similar
to AMQP and MongoDB, the driver uses value-pairs to transfer messages
across. An optional body can still be set, however.

destination d_stomp {
 stomp(
  host("localhost")
  port(61613)
  destination("/topic/syslog")
  routing-key("")      # optional, empty by default
  body("")             # optional, empty by default
  persistent(yes)
  ack(no)
  username("user")     # optional, empty by default
  password("password") # optional, empty by default
  value-pairs(scope(selected-macros, nv-pairs, sdata))
 );
};

The driver requires no new dependencies, an internal STOMP client
implementation is used.

Signed-off-by: Attila Nagy <bra@fsn.hu>
Signed-off-by: Viktor Tusa <tusa@balabit.hu>
Signed-off-by: Gergely Nagy <algernon@balabit.hu>
templates: Properly clear options defaults

In log_template_options_defaults(), clear all of the LogTemplateOptions
structure, not just the first few bytes.

Reported-by: Michal Privoznik <miso.privoznik@gmail.com>
Signed-off-by: Gergely Nagy <algernon@balabit.hu>
system-source: Fix the copyright headers

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
afsocket: Warn & auto-migrate when using /dev/log under systemd

We were emitting a warning when /dev/log was in use, yet, we were
running under systemd, but only when the config version was older than
3.4. Due to various circumstances, lets extend that so it warns on 3.5
too.

Requested-by: Peter Czanik <czanik@balabit.hu>
Signed-off-by: Gergely Nagy <algernon@balabit.hu>
filter: Allow in-list("FILE", value("MACRO"))

While the in-list("FILE", value("MACRO")) syntax was documented in the
commit message, it was not actually supported. This patch implements
that syntax too.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
afstomp: Make it depend on libsyslog-ng.la

For parallel builds, depend on ${MODULE_DEPS_LIBS}, to avoid a race
condition.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
value-pairs: Fix a crash when an invalid scope is specified

Terminate the list of scopes with empty values, so we reach the stop
condition instead of overflowing.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
templates: Initial support for on-error()

This implements low-level support for the on-error() setting, making it
possible to set it globally in options{}, and making inheritance work,
so that it can be overridden later on.

Valid values are: drop-message, drop-property and
fallback-to-string (this latter one is only valid for type hinting,
though!), optionally prefixed by "silently-".

Signed-off-by: Gergely Nagy <algernon@balabit.hu>
value-pairs: Add support for the on-error() setting

value-pairs should implement on-error handling, so make it possible to
configure the behaviour on a per-instance basis. The default value is
inherited from the global setting.

Signed-off-by: Gergely Nagy <algernon@balabit.hu>