logstash/logstash — pull request #329 by alexdean (2013-01-24 20:25:10+00:00)
Title: redis output can lose messages in batch mode

Scenario:
1. @flush_thread starts writing to redis with `@redis.rpush`

https://github.com/logstash/logstash/blob/619f7f8504c64904ae0680400c3c36affc343fc0/lib/logstash/outputs/redis.rb#L199
1. While @flush_thread is waiting on redis, the main thread adds a new message to `@pending`

https://github.com/logstash/logstash/blob/619f7f8504c64904ae0680400c3c36affc343fc0/lib/logstash/outputs/redis.rb#L153
1. @flush_thread deletes this message with `@pending.delete`

```
in @flush_thread in main thread

@pending.each do |k, v|
@redis.rpush(k, v)
@pending[event.sprintf(@key)] << event.to_json
@pending.delete(k) ^^-- this message is deleted from @pending without ever being sent to redis
end

```

## re: attached test ##

if you comment out the `@pending_mutex.synchronize` call in `receieve`, this test will usually fail, but not 100% of the time.

to make it reliably fail when the sychronize block is disabled, have this script executing in another shell while you run the tests

```
while [ true ]
do
pkill -STOP redis
echo 'stopped'
sleep 1
pkill -CONT redis
echo 'running'
sleep 1
done
```