belaban/JGroups — pull request #89 by WIgor (2013-07-24 22:55:41+00:00)
Title: Don't block tcp queue

Hi

I've found yet another problem with blocking queue in TCP. I tested it with TCP behind RELAY2 - but I suppose when we are using TCP without RELAY2 problem will be same.

It's easy to reproduce - just set send_queue_size to small value (I setted it to 2) and break connection between nodes (I used two VM and switched off network card on one of them). In our production it is reproduced with larger queue size ~ 2000.

This queue is blocking on put and will not released because thread that work in Sender.run method will be interrupted when connection is broken, and no one will take messages from this queue instance. When connection will be restored one more instance of Sender will be created. But it doesn't release threads that stuck with put().

I tested this in 3.3 branch. May be something simmilar will be in master - but I have not tested it there yet.