logstash/logstash — pull request #340 by enjoyjava (2013-01-29 23:12:02+00:00)
Title: Fix for elasticsearch output concurrency issue

Hi,

We experienced an dead lock issue with ES output if the file input is set up to watch files (start from beginning for new files). ES hangs when it finishes the first file and starts to pull in records from the 2nd file.

I found out there are two issues in the trunk code:

1) increment_inflight_requests is called outside the synchronize block. As a result, two threads pass the max limit check.

2) The failure retry callback is made from the ES client worker thread. It will block the thread from being released back to the pool. As a result, ES requests cannot be scheduled but they hold counters to inflight_requests. It leads to a deadlock im my test.

The fix is to spawn a new thread for the failure retry.

Please merge https://github.com/enjoyjava/logstash/commit/90a4f19bad3c97c66263ff4eaa3169190da03ee8.