logstash/logstash — pull request #157 by webbob (2012-06-14 16:48:49+00:00)
Title: new filters crush and smash

This pull request is for new filter files, crush.rb and smash.rb. I wrote these to support pulling JSON-formatted data out of JBoss logs and sending the results to a MongoDB instance for business-oriented analysis.

The "crush" filter is configured with two strings, "leftstop" and "rightstop" which correspond to the left end and right end of the part of the log line which is to be preserved. Everything between and including leftstop and rightstop is sliced out of the event's message field and preserved, the rest of the message is discarded.

Default values of leftstop and rightstop are both an empty string, which matches the start and end of the log line.

The "smash" filter replaces the contents of the event's message field with an arbitrary string set in the filter configuration, including an empty string. Our configuration ended up putting two copies of the original log line in the final message field, which was not useful after we'd JSON parsed the message into separate fields. Rather than thinking too much about the problem, we just smashed the message field.