LOGSTASH-1886: intercept exception raised by march_hare

Since ae867bc64, the march_hare version we use raises an error when the
network link is broken while the socket is idle. This case must be
handled by logstash to initiate the reconnection process.
Merge pull request #1665 from mfournier/LOGSTASH-1886-2nd_try

LOGSTASH-1886: intercept exception raised by march_hare
Support almost-ISO8601 patterns in date filter ISO8601 config
fix rebase
Closes #769
Add Plugin manager to Logstash

The plugin manager allows users to install, uninstall, update and list plugins for logstash
filter/date fix locale config
Closes #1664
String has to be surrounded by quotes
prevent overwrite of event host field
Closes #1668
Dynamic include_keys and exclude_keys for kv filter
Closes #1618
Change credentials default to [], use empty array

closes /elasticsearch/logstash/#1619
Removed nil check
Closes #1661
Merge @joekiller's Kafka plugin to logstash-core
Add documentation and tests
Use scala version 2.9.2 and Kafka 0.8.1.1
Closes #1472
Prevents event.sprintf from converting floats to sci notation

This allows sprintf to correctly print floats up to 15 decimal
places, which should be enough for most. Without this, very big
or very small floats will be converted to scientific notation.

closes #1670
Dont add kafka metadata by default
Closes #1691
Merge remote-tracking branch 'upstream/master'
Add recurse method for doing breadth-first traversal of the AST

This will be used by the filter flush compiler

Add generation of a flush lambda for each filter.

This allows filters to flush and have any generated events proceed
downward through the config as you would expect, respecting any
branches or future plugins.

Die on IOError which occurs when reading from a closed STDIN

Make filter_flusher invoke the new (and correct, I hope!) way to flush.

- On shutdown, we will also flush all filters.
- The flusher thread will terminate if we are shutting down.

Clarify the comment

Fix comment generation in the code to avoid newlines.

Add 'max_age' setting to multiline for flushing.

This setting chooses how long (in seconds) an event is considered to be
fresh before it will be automatically flushed.

This is useful for:
* slow log sources to get the 'last event' flushed,
* transaction-id-style events that have no obvious "end" event and also
  are mixed among other-id events in the same stream.

Also:
- Make filters have no teardown by default.
- Remove 'enable_flush' since it is not needed anymore; flush is always
  enabled.

refactor flush

new spool filter and specs, mainly for testing flushing

turn off unrelated test error for now

fix the flush logic, fix the flush compiles code to not include output section

synchronize cross-thread  access to @pending

refactor for performance and readability

synchronize cross-thread access to @spool

unused code

input:udp removed boggus ShutdownSignal handling, morphed loop do into while true, cosmetic reformat

use transcient events and not exceptions for in-flow pipeline signaling

inline flushing into  filterworker

removed now unnecessary flushing thread safety

fix conditionals bug for new events generated by filters & specs

spec for issue #793

performance tweeks

simplify filter handling of events and new_events

this removes unecessary duplication when treating the original event as
a special case (different from new_events generated by a filter).
Also, since @filter_func only outputs non-cancelled events, some checks
were also removed.

Move multiple filter specs to a filter_chains file

append events generated by a filter using unshift instead of insert

closes #793, closes #1429, closes #1431, closes #1548
Fixes finterprint filter handling of timestamp field

Fixes #1572
Merge github.com:elasticsearch/logstash