logstash/logstash — pull request #182 by derekgr (2012-08-09 18:14:35+00:00)
Title: Add a Cassandra output plugin

Apache Cassandra is a distributed key-value store based on DynamoDB. It's similar to Riak in many ways. This PR adds an output plugin for logstash to Cassandra. You can specify in the config where your events get logged, and provide a fast way to access them with wide-rows as secondary indexes.

Here's an example config for storing access log lines that have been grokked into CLF. The event will have its auth, timestamp, request, and response attributes inserted as named Cassandra columns, and the UUIDs of the records will go into wide columns in two tables, one to quickly lookup events by time, and one to lookup events by user and time.

```
output {
cassandra {
type => "example"
nodes => [ "host1", "9160", "host2", "9160" ] # etc
keyspace => "logstash"
table => "logstash"
event_schema => [
'user', 'auth',
'request_at', 'timestamp',
'query', 'request',
'http_status', 'response'
]
index_tables => [ "logstash_timeline", "user_timeline" ]
index_table_keys => [
"logstash_timeline", "logstash_%{+YYYYMMdd}"
"user_timeline", "user_%{auth}_%{+YYYYMM}"
]
}
}
```

One thing I wasn't sure of was changing the Event class. Since the Cassandra UUID generator will Do The Right Thing with Ruby Time objects, I added a method to get at an event's parsed timestamp, since I couldn't find one that didn't return it pre-formatted into a number or a string.

Let me know what you think.