Graylog2/graylog2-server — pull request #67 by webbob (2011-12-28 16:09:32+00:00)
Title: create additional fields by parsing JSON in short_message for key-value pairs

In this fork I added code to fully parse a JSON object included as the GELF short_message section and turn the keys and values into GELF additional fields.

If a JSON key-value pair is "foo":"bar", the result is an additional field "_foo" with value "bar". Only string, array, and number JSON values are handled at this time.

In the case of a key "foo" with array value "baz", individual elements of the array are returned as field "_foo_1 " with value baz[1], "_foo_2" with value baz[2], etc.. If short_message is a JSON array, the values are returned with keys "_root_1", "_root_2", etc..

JSON parsing can be switched on or off with a setting in the graylog2-server configuration file. The location of a "skip list" file can optionally be configured in the graylog2-server configuration file. If the JSON parser encounters a key which literally matches a string in the skip list, the key and its value are ignored.

Source code changes for this extension to graylog2-server are 1 new file for the parser, a change to the parser hook to support the new parser (in addition to the Drools-based parser), and modification to Main to support new configuration file entries and JSON parser initialization. All the new code is in Java.