inliniac/suricata — pull request #875 by regit (2014-03-05 20:45:07+00:00)
Title: output-json: update timestamp format

This patch allows user to set the format to use. The new
YAML variable 'timestamp-format' can be set to 'iso' or
'legacy' (default value). If set to 'legacy' nothing change
but if set to 'iso' then it updates the timestamp format
to use a ISO 8601 comptabile string. This allow tools parsing
the output to easily detect adn/or use the timestamp.

In the EVE JSON output, the value of the timestamp key has been
changed to 'timestamp' (instead of 'time'). This allows tools
like Splunk to detect the timestamp and use it without configuration.

Logstash configuration is simple:

input {
file {
path => [ "/usr/local/var/log/suricata/eve.json" ]
codec => json
type => "suricata-log"
}
}

filter {
if [type] == "suricata-log" {
date {
match => [ "timestamp", "ISO8601" ]
}
}
}

In splunk, auto detection of the fle format is failling and it seems
you need to define a type to parse JSON in
$SPLUNK_DIR/etc/system/local/props.conf:

[suricata]
KV_MODE = json
NO_BINARY_CHECK = 1
TRUNCATE = 0

Then you can simply declare the log file in
$SPLUNK_DIR/etc/system/local/inputs.conf:

[monitor:///usr/local/var/log/suricata/eve.json]
sourcetype = suricata

In both cases the timestamp are correctly imported by
the tools.

Redmine ticket: https://redmine.openinfosecfoundation.org/issues/1123

PR builds:
- PR build: https://buildbot.suricata-ids.org/builders/regit/builds/117
- PR pcaps: https://buildbot.suricata-ids.org/builders/regit-pcap/builds/56