commands: External command handling system overhaul

HUGE PATCH WARNING: This is a significant changeset in a single commit.
I'm deeply sorry for this, but the (previous) state of this TU makes it
non-trivial to make incremental changes to achieve big changes. Coupled
with the fact that the branch was in a state of decay (I haven't had the
time to touch it for a few months), it was quite the challenge to merge
with Naemon's new indentation rules and other changes. Again, apologies.

This patch-set is the first step towards getting a sane command
handling system. It is not finished, but it should in its current form
be a significant improvement to the one it replaces.

Patch overview:

* Proper single-point parsing, and verification of external commands,
 including arguments and their respective types:
    a) Proper error reporting on violation of the above
    b) Automatic resolution of referenced objects
* Wireframe for supporting different parse modes
* On-the-fly creation of custom external commands
    a) Default parameters
    b) In-core command descriptions
* Better error messages
* Relevant ABI/API backwards compatibility preserved
* Future changes

Detailed information:

** Proper single-point parsing, and verification of external commands,
 including arguments and their respective types

Parsing of external commands was previously very much ad-hoc and deeply
in violation of DRY, leading to various errors that should be trivial to
avoid. Each command handling routine reinventing the parsing lead to
minor variations in how arguments were parsed and resolved as well as
errors stemming from simply parsing the wrong argument.

This patch instead introduces a generic (but not too generic),
batteries-included parsing system, which verifies the type of arguments
based on typing information specified at the time of their creation
(more on this later).
This leads to a consistent approach to error handling/reporting and, in
general, a more pleasant way of working with external commands.

Also included is automatic resolution of referenced objects, such as
hosts, services, timeperiods, etc. This is something that was previously
also very ad-hoc, leading to inconsistencies in error reporting, should
you reference an object that doesn't exist or in some other way, was
"bad".

Of course, this patch introduces a number of new error codes/messages to
reflect the state of execution.

** Wireframe for supporting different parse modes
We have, historically, been tremendously locked-in to the syntax of
external commands. While this may not seem like a huge problem to most
people (especially those comfortable with the Nagios-style syntax of
semicolon separators and what-not), it is hardly optimal for a number of
reasons. For example, by using a position-dependant syntax, we require
users to _always_ specify _every_ argument, even if that means simply
specifying whatever seems more "default". A better approach would likely
be to adopt a key:value-like syntax, and allow commands to have default
values for unspecified parameters (which we now also have, see more
below). This would also ease integration with external systems/syntaxes
and commonly used datastructures such as JSON/maps/dictionaries.

This patch, however does not include such a parse mode, since it should,
in my opinion, be discussed further among the developers what such a
mode should entail.

** On-the-fly creation of custom external commands
Naturally, to be able to support type checking of parameters, we need to
be able to specify their types. This patch includes an API that allows
developers to create and register their commands with the external
command subsystem. Created commands need information such as a name, a
description and a list of types/arguments as well as a
developer-specified callback. It is also possible to specify default
values for arguments. In addition, for backwards compatibility
reasons, the system supports specifying a "command code",
which is simply a unique integer macro as used in many of the
pre-existing command handlers.

To increase ease-of-use, I've also includes a number of helper macros
and a trivial syntax for declaring parameters at creation-time.

** Better error messages
This work was already started a while back, and some of it has already
been merged. It is still not perfect, and still needs more effort, but
at least it's something.

** Relevant ABI/API backwards compatibility preserved
Nagios has historically not cared much about which functions are
exported (non-static) outside of TUs. Because of this, it is nigh
impossible to make significant changes without breaking _some_ sort of
API. I have, however, tried very hard (and of course, after getting some
input on the matter from AE) to only change/remove things that
sane people would actually have used. It is truly my belief that this
patch will not break existing NEBs API-wise.

** Future changes
In the future, I hope to, among other things:

* Implement a key-value command syntax, as discussed above.
* Provide a reasonably opaque data structure in which to expose parse
 errors ("syntax error in position <n>", "parameter <n> has the wrong
 type", "parameter <name> is a <type> that does not exist"). This would
 allow, for example, query handlers to expose errors in a way that makes
 sense to their use case.
* Consolidate and deprecate superfluous commands. Today, we have a
large number of commands, which differ only in parameter count and
minor implementation details. For example, SCHEDULE_SVC_CHECK and
SCHEDULE_FORCED_SVC_CHECK are two different commands. This example
could easily be implemented as a SCHEDULE_SVC_CHECK command with a
'force' flag, default off. Again, there are a lot of commands like
this, and this is one of the less horrendous ones. I believe that doing
so would much reduce the complexity of the system, with all the benefits
that usually brings.

Thanks-to: Andreas Ericsson <ae@op5.se>
Thanks-to: Carl Helmertz <chelmertz@op5.com>
Signed-off-by: Anton Lofgren <alofgren@op5.com>
naemon: Register and initialize external command register

This patch enables the use of the external commands register.

Signed-off-by: Anton Lofgren <alofgren@op5.com>
commands/test: Test external command register

This patch adds and integrates tests for parsing of, resolution of and
execution of external commands.

I was initially inclined to write a test for every external command that
we currently support. However, I finally realised that that would take
me way too much time (as in, I would never finish it). Besides, as I've
mentioned earlier, I think we should start deprecating and consolidating
many of the commands that we currently support. And hey, it's better
than the 0 runnable test cases we previously had.

Signed-off-by: Anton Lofgren <alofgren@op5.com>
Remove dependencies in wproc, use wproc_run_callback

Designing the system to keep as few dependencies between the modules
in the system as low as possible is generally a good thing. That
makes each module do it's own thing without knowledge of the other
parts of the system. That means, each functionality has its own
distinct border, and debugging is mostly easier.

One part in this cleanup is wproc:s. wproc handles job types as
integers, and each job type can only be called from one other module.
But the wproc needs to have knowledege of the other module, because
it should be able to call back to that module depending on job type,
by using a switch statement in the wproc result handler.

Better structure is to use function pointers, and design a generic
callback, because then, the wproc module doesn't know anything at all
about the caller, and the explicit dependencies to other modules can
be reamoved.

Using function callbacks makes it having fewer special cases, and
makes it easier to set up test cases for.

Signed-off-by: Max Sikstrom <msikstrom@op5.com>
Fix memory leak when running callback worker job

Destroying a callback job didn't free the callback storage, causing a
memory leak. Fix that!

This wasn't an issue earlier, when callbacks wasn't used much at all
(only some broker modules used that on some user interactions)

Signed-off-by: Max Sikstrom <msikstrom@op5.com>
Notifications: Use wproc_run_callback

By using wproc_run_callback instead of a notify-specific job type, we
can loose the dependency from workers.c to notifications.

Signed-off-by: Max Sikstrom <msikstrom@op5.com>
Convert OCSP and OCHP to wproc_run_callback

Continue to reduce dependencies from wproc to other parts of the system.

Signed-off-by: Max Sikstrom <msikstrom@op5.com>
Convert event handlers to wproc_run_callback

This reduces the knowledege of the surrounding environment in the wproc
module further. That makes it easier to write tests for later.

Signed-off-by: Max Sikstrom <msikstrom@op5.com>
Convert perfdata to wproc_run_callback

performance data doesn't need to handle the result, but convert it to
wproc_run_callback, so that's the only used interface for wproc.

Signed-off-by: Max Sikstrom <msikstrom@op5.com>
Cleanup wproc, when we don't use types anymore

Since the earlier cleanup, all worker calls results in callback jobs,
there's no need to have any job types anymore. Cleanup all type-related
code in wproc, and merge structs, so job contains callback and data
pointers directly

Signed-off-by: Max Sikstrom <msikstrom@op5.com>
naemon/macros.c: removing double check for variable

we already check a few lines before if input_buffer is or not
valid.

as a plus, if a fail during strdup() occurs we return ERROR

Signed-off-by: Ricardo Maraschini <ricardo.maraschini@gmail.com>
naemon/macros.c: using continue to avoid uneede `else'

if we are not `inside a macro' we just copy the current string
to output buffer and keep it going.

Signed-off-by: Ricardo Maraschini <ricardo.maraschini@gmail.com>
naemon/macros.c: avoid to call grab_macro_value_r() if escaped $ found

bringing comparsion of macro with "" a little up. this way we
avoid to call grab_macro_value_r() with a parameter that we
know it is not a macro.

in other words, if strcmp(temp_buffer,"") is false, we know
that macro contains $$ so we just concatenate it to the output
buffer and keep rocking on.

Signed-off-by: Ricardo Maraschini <ricardo.maraschini@gmail.com>
naemon/macros.c: grab_macro_value_r() returns only OK or ERROR

grab_macro_value_r() and it's inner functions only return OK
or ERROR(macro could not be found).

in the last case we just copy the macro as is to the output
buffer as the previous version has being doing since remote
ages.

Signed-off-by: Ricardo Maraschini <ricardo.maraschini@gmail.com>
t-tap/test_macros.c: added tests for escaped $ and invalid macros

added two extra tests to validate if $$ is being escaped to $
and if an invalid macro is copied as is to the output buffer

Signed-off-by: Ricardo Maraschini <ricardo.maraschini@gmail.com>
Remove "extern" from method prototypes

extern is the default behaviour for method prototypes that isn't
declrared as static. Explicitly specify extern in the header files just
adds to the noise for the person reading the header files.

Signed-off-by: Max Sikstrom <msikstrom@op5.com>
Merge branch 'catharsis-external-commands'

This branch currently clashes hard with the new retention code - both
with how they read the retention file, and with how the tests actually
work - so this is just a very, very early start for completing the
merge.

Signed-off-by: Robin Sonefors <robin.sonefors@op5.com>

Conflicts:
	naemon/commands.c
	t-tap/Makefile.am
	t-tap/smallconfig/naemon.cfg
t-tap: Use more abs and build in Makefile

As for the build, the source files are now copied to the builddir, so
start there when looking for them to get help from the nspath functions
to avoid hardcoded paths.

As for the abs, this means I can re-run the tests to debug them, without
cd:ing into the t-tap directory.

Signed-off-by: Robin Sonefors <robin.sonefors@op5.com>
test_config: Adjust to sane TESTS_ENVIRONMENT

Now that TESTS_ENVIRONMENT has already copied the files in place, I
don't need to copy anything, and I really don't need to copy any strings
for finding retention data.

Though, this trick relies on nspath_absolute finding a config_file_dir,
a global variable which is set inside main(). Of course. I'll just close
my eyes and get it over with.

Signed-off-by: Robin Sonefors <robin.sonefors@op5.com>
test_commands: Load config using default.c functions

This means config and retention can be resolved compile time - not only
while writing the test, nor only when running it - assuming
config_file_dir is set. So also config_file_dir.

This still crashes, but it loads the config.

Signed-off-by: Robin Sonefors <robin.sonefors@op5.com>