aphyr/riemann — pull request #140 by gmanika (2013-01-29 23:22:11+00:00)
Title: Update netty to 3.6.1, fixes leaking thread pools

I believe this could fix issue #89.

I send data to riemann using lots of very short TCP connections. I was getting this message very very frequently, and stream processing would often halt after this:

_WARNING: You are creating too many MemoryAwareThreadPoolExecutor instances. MemoryAwareThreadPoolExecutor is a shared resource that must be reused across the application, so that only a few instances are created._

I tried to debug the problem a bit. It seemed to be related to the fact that TCP sockets were not getting closed by riemann, and thus get stuck in a CLOSE_WAIT state on the server and FIN_WAIT2 on the client. Interestingly I could only reproduce this behavior on Linux and not on OS X.

This seems to be a netty bug. I tried updating it to the latest version. Connections no longer get stuck in CLOSE_WAIT, the message no longer appears and riemann has been processing streams continually for some time now. Seems to work.