logstash/logstash — pull request #768 by lucabelluccini (2013-11-09 21:59:10+00:00)
Title: CustomRedis Output Plugin(Draft implementation v0) + MsgPack Codec (Fix)

The PR consists in:
* Fix MsgPack Codec - It was throwing "no to_msgpack method for *", I replaced the code with MsgPack.pack
* Implement a "custom" redis plugin (I am developing it to perform some stuff for a project) - It allows to use LUA SCRIPTS to extend the Redis API and offload some processing; It also allows to use different codecs other than the hardcoded json

Here it is an example of the config for the customredis output plugin:
```
customredis {
data_type => "list"
key => "ourkey"
batch => true
codec => msgpack {}
workers => 5
luascript => "

local globalKey = 'gcxid'
local globalKeys = {}
globalKeys['oid'] = true
globalKeys['vc'] = true
local singleKeys = {}
singleKeys['trxnb'] = true
singleKeys['@timestamp'] = true
singleKeys['st'] = true
singleKeys['sn'] = true
singleKeys['cr'] = true
singleKeys['tpfk'] = true

for index, raw_item in ipairs(ARGV) do
local item = cmsgpack.unpack(raw_item)
local output = {}
output['single'] = {}
output['global'] = {}
for key, value in pairs(item) do
if globalKeys[key] ~= nil then
output['global'][key] = value
end
if singleKeys[key] ~= nil then
output['single'][key] = value
end
end
output['key'] = ((item[globalKey] ~= nil) and item[globalKey] or 'UNKNOWN')
redis.call('SET', 'g:' .. output['key'], cmsgpack.pack(output['global']))
redis.call('RPUSH', 's:' .. output['key'], cmsgpack.pack(output['single']))
redis.call('ZADD', 'to_be_expired', 10, KEYS[1])
end
return 'OK'
"
}
```

Feel free to open a branch and I'll submit other PRs on that branch.
Sorry for the quality of the code, but it's the first time I'm coding in Ruby.

Best regards,
Luca