fluent/fluentd — pull request #151 by y-ken (2013-07-25 13:54:32+00:00)
Title: add auto_type_convert option for parser.rb

## summary

I have add auto type conversion option for `parser.rb`.
It is not cool to convert type (cast) by manually. it have to update whenever log format has changed.

## configuration sample

```

type tail
format ltsv
time_format %d/%b/%Y:%H:%M:%S %z
path /var/log/httpd/access_log
tag debug.apache.access
auto_type_convert yes

```
## sample results

the difference seems in status, size, response_time.

```diff
$ diff -u before.json after.json
--- before.json	2013-07-25 22:46:45.000000000 +0900
+++ after.json	2013-07-25 22:47:09.000000000 +0900
@@ -1,3 +1,2 @@
-2013-07-25 01:12:48 +0900 debug.apache.access:
-{"domain":"127.0.0.1","host":"127.0.0.1","server":"127.0.0.1","ident":"-","user":"-","method":"GET","path":"/","protocol":"HTTP/1.1","status":404,"size":198,"referer":"-","agent":"Mozilla","response_time":398,"cookie":"-","set_cookie":"-"}
-
+2013-07-25 01:13:12 +0900 debug.apache.access:
+{"domain":"127.0.0.1","host":"127.0.0.1","server":"127.0.0.1","ident":"-","user":"-","method":"GET","path":"/","protocol":"HTTP/1.1","status":"404","size":"198","referer":"-","agent":"Mozilla","response_time":"398","cookie":"-","set_cookie":"-"}
```

## the implemention logic benchmark result

I have benchmarked the type conversion speed because it is very important to parse as fast as possible.
The result, `Integer(value) rescue false` is faster than `value.to_i.to_s` implemention.

```
benchmark ken$ bundle exec ruby test.rb
user system total real
0.000000 0.000000 0.000000 ( 0.000051)
0.000000 0.000000 0.000000 ( 0.000021)
benchmark ken$ bundle exec ruby test.rb
user system total real
0.000000 0.000000 0.000000 ( 0.000032)
0.000000 0.000000 0.000000 ( 0.000020)
benchmark ken$ bundle exec ruby test.rb
user system total real
0.000000 0.000000 0.000000 ( 0.000033)
0.000000 0.000000 0.000000 ( 0.000019)
benchmark ken$ bundle exec ruby test.rb
user system total real
0.000000 0.000000 0.000000 ( 0.000032)
0.000000 0.000000 0.000000 ( 0.000030)
```

#### benchmark code

```ruby
require 'benchmark'

def convert_type1(record)
record.each do |index,value|
if value == value.to_i.to_s
record[index] = value.to_i
elsif value == value.to_f.to_s
record[index] = value.to_f
end
end
return record
end

def convert_type2(record)
record.each do |index,value|
if (Integer(value) rescue false)
record[index] = value.to_i
elsif (Float(value) rescue false)
record[index] = value.to_f
end
end
return record
end

record = {"int"=>"123","float"=>"12.34"}
n = 1000000
Benchmark.bm do |x|
x.report { convert_type1(record) }
x.report { convert_type2(record) }
end
```