fluent/fluent-plugin-s3 — pull request #33 by suzukaze (2013-09-25 12:01:26+00:00)
Title: This patch stores logs in reduced redundancy.

This patch uses the code of the following article :

http://developer.smartnews.be/blog/2013/09/02/an-effective-log-management-technique-which-uses-fluentd-and-s3/
https://github.com/moaikids/fluent-plugin-s3/commit/183708f6a9a5c2eb12e8d64a9f02233c25cb3a9d

For example, this feature is used and set up fluent.conf:

type s3

aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET/KEY
s3_bucket YOUR_S3_BUCKET_NAME
s3_endpoint s3-ap-northeast-1.amazonaws.com
s3_object_key_format %{path}%{time_slice}_%{index}.%{file_extension}
path logs/
buffer_path /var/log/fluent/s3

time_slice_format %Y%m%d-%H
time_slice_wait 10m
reduced_redundancy true
utc

A default value of reduced_redundancy is false.

I test the following tests:

* reduced_redundancy is true => Logs is stored in reduced redundancy.
* reduced_redundancy is false => Logs is stored in standard.
* reduced_redundancy is none => Logs is stored in standard.