elasticsearch/logstash — pull request #1657 by suyograo (2014-08-22 20:19:18+00:00)
Title: Use optimized grok library

Tested using `%{COMBINEDAPACHELOG}` grok format. Almost 3x throughput increase --

With grok version `0.10.12`
``` JSON
{
"@version" => "1",
"@timestamp" => "2014-08-04T23:19:55.993Z",
"message" => "Suyogs-MacBook-Pro.local",
"events.count" => 445142,
"events.rate_1m" => 8388.367317089305,
"events.rate_5m" => 7234.09457935761,
"events.rate_15m" => 6983.090272699621,
"tags" => [
[0] "metric"
]
}
```
With grok version `0.11.0` and associated LS changes

``` JSON
{
"@version" => "1",
"@timestamp" => "2014-08-05T00:29:43.977Z",
"message" => "Suyogs-MacBook-Pro.local",
"events.count" => 2965949,
"events.rate_1m" => 20580.377595661972,
"events.rate_5m" => 17600.49616640097,
"events.rate_15m" => 16340.908759448706,
"tags" => [
[0] "metric"
]
}
```
LS Config:
``` javascript
input {
generator {
message => '31.184.238.164 - - [24/Jul/2014:05:35:37 +0530] "GET /logs/access.log HTTP/1.0" 200 69849 "http://8rursodiol.enjin.com" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.12785 YaBrowser/13.12.1599.12785 Safari/537.36" "www.dlwindianrailways.com"'
#count => 1
}
}
filter {
grok {
match => {"message" => "%{COMBINEDAPACHELOG}"}
}
metrics { meter => "events" add_tag => "metric" }
}
output {
# only emit events with the 'metric' tag
if "metric" in [tags] {
stdout {
codec => "rubydebug"
}
}
}
```