Initial stab at logstash package update
Multiple ID
Update naming clashes
Added split config files
Added support for multiple redis servers
Handle migration from app_skeleton logstash-1.2.2

We need to stop it from running via upstart, and ensure any old
process is killed.

Also:

* ensure logstash process cannot write to its own config files, and
* ensure conf.d and patterns dirs are cleaned of unexpected files.
Ensure tests pass in Vagrant
README update
Add tests for 200_filter_all config

Many of the tests in syslog_input_spec.rb do not need to be
tested via the syslog input - we just need to ensure that
the data provided to a filter-only config object is as it would
be if the syslog filter had parsed the original event.

This sets up the structure for moving most of the syslog_input_spec
tests out of there, and just ensuring that the syslog input is
doing what we expect.
Simplify type=>syslog tests via a direct filter test suite

The hacking to get a syslog input handler in place is excessive for
most of our tests, when all we need to know is that the filter configuration
has received the message from the input in the right form.

Migrate the syslog_input_test test cases into a specific test based
around the 200_filter_all config.
Fix apparmor events overwriting 'type' field

'kv' filter needs to be used with care, it can easily overwrite
core fields - like 'type' in this case.

Flesh out the tests with other apparmor data too.
Fix syslog '@message' mutate config bug.

We were incorrectly writing syslog_message field to @message, thinking
we were overwriting 'message' field that the message is actually stored
in (at least at v1.4.x).

This corrects this issue, in two ways:

* Leave syslog_message intact - it has semantic meaning.
* * Do not overwrite the source 'message' field -- we will want to use that for reloading.
*
handle nginx timestamp_msec

There is a weirdness trying to test that we are getting an ISO8601+msec
value out of @timestamp, hence the .to_json test. If a clearer way of doing
this exists, PR!
Rearrange type-specific filters into own files

The '200_filters_all.conf' was actually a container for filters
acting on 'syslog' and 'audit' type messages.

Split these into their own config files, and leave 200_filters_all
available for filters acting early (200) on all types.
Add a uuid in @logstash_uuid, use for ES doc id.

Set a UUID for each message, and make it the _id for each message fired
into ElasticSearch.

This in turn can be used to:

* Reload data into ES
* Multi-path messages, so overwrites of duplicates do not matter.

In the process, extract the elasticsearch flush_size parameter into a pillar
value, so we can play with tuning it. '1' is extremely inefficient.

WARNING:

This appears to add significant time to the tests and probably start-up
time of logstash -- this appears to be due to SecureRandom.uuid blocking
for a few minutes, I suspect due to entropy gathering.

The filter can be disabled by removal of '200_filter_all' from the pillar data.
Replace broken uuid filter with a ruby one-liner
Handle haproxy log messages via syslog
Allow specification of _spec.rb files to test
Fix up audit events; add tests

Whaddayaknow, you miss adding a test for some preexisting code, and it
turns out to be broken.

Add a test for type->audit messages from /var/log/audit/audit.log

Add a CATCHALL grok parser, as there were quite a few log lines not
passing the existing ones (o/ _grokparsefailure)
Remove legacy indexer.conf template