logstash/logstash — pull request #888 by bpezan (2013-12-18 20:46:43+00:00)
Title: enable automatically mapping logstash fields to riemann event fields.

this patch allows automatic mapping from logstash fields to riemann event fields via a new config directive (map_fields). it recurses through all key value pairs in the event that don't start with '@', unless '@fields' is present, and maps them to a riemann field containing all parent keys separated by a dash and the deepest value. it also subs out any underscores with dashes in the resulting field name as i think it falls in-line with the riemann philosophy(?). here's an example of a logstash event transformed into a riemann event.

logstash_event = {"@timestamp":"2013-12-10T14:36:26.151+0000","@message":"log message","@fields"{"first_level_field":"first_level_string","first_level_hash":{"second_level_field":"second_level_string","second_level_hash":{"third_level_field":"third_level_string"},"second_level_array":["first_array_string","second_array_string"]}}}

riemann_event = {:host host.domain.com, :time 1386686186, :description log message, :first-level-field first_level_string, :first-level-hash-second-level-field second_level_string, :first-level-hash-second-level-hash-third-level-field third_level_string, :first-level-hash-second-level-array ["first_array_string","second_array_string"]}

it can be used in conjunction with or independent of the riemann_event option. when used with the riemann_event any duplicates keys receive their value from riemann_event instead of the logstash event itself.

byron