fluent/fluent-plugin-s3 — pull request #36 by suzukaze (2013-11-02 15:17:11+00:00)
Title: Add Snappy compression option.

#6: Implement.

Install gem:
gem install snappy

Add `store_as snappy` in config:
```text

type s3

aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET/KEY
s3_bucket YOUR_S3_BUCKET_NAME
s3_endpoint s3-ap-northeast-1.amazonaws.com
s3_object_key_format %{path}%{time_slice}_%{index}.%{file_extension}
path logs/
buffer_path /var/log/fluent/s3

store_as snappy # snappy compression option

time_slice_format %Y%m%d-%H
time_slice_wait 10m
utc

```

test:
22-55_0_jun-MacBook.local.snappy file in S3 is decompressed and the decompressed text saved hoge.txt.

Input a log:
```text
echo '{"json":"message"}' | fluent-cat debug.test
```

snappy-inflate.rb:
```ruby
require 'snappy'

file = File.open(ARGV[0])
str = file.read
puts "str=#{str}"

str2 = Snappy.inflate(str)
puts "str2=#{str2}"

file.close

open("hoge.txt", "w") {|f| f.write str2}
```

Execute a decompressed ruby program:
```text
ruby snappy-inflate.rb 22-55_0_jun-MacBook.local.snappy
```

hoge.txt:
```text
2013-11-02T22:55:17+09:00	debug.test	{"json":"message"}
```