INT-3433 commit

INT-3433: Fixed NIO deadlock (or timout if the CallerBlocksPolicy is used) with Bounded ThreadPoolTaskExecutor.

JIRA: https://jira.spring.io/browse/INT-3433

I have signed and agree to the terms of the SpringSource Individual
Contributor License Agreement.
My understanding is when a server is overwhelmed with data from clients, it has only 3 options:
1. Crash due to out of memory (either because it cannot allocate more heap space or can't create more threads).
2. Drop messages.
3. Throttle clients to be able to catch up with them and continue as usual.

I think in most cases option 3 is preferable. As far as I know, the only way to throttle clients is to stop (or slow down) reading from sockets which decreases the TCP Sliding Window and forces clients to slow down. Please correct me if I'm wrong, because I'm not a big expert in NIO/TCP.

As far as the selector immediately re-running the task, I did a test and saw that the selector doesn't try to read only from one socket when the read from it is skipped, but rather reads from all available sockets indiscriminately. Hence for some sockets, the selector tries to read from, assembler threads are available and processing continues.

I agree that it causes waste of CPU cycles when the selector tries to read from sockets for which assembler threads are not available and performance degrades, but my point is that when the pool is exhausted performance is already poor and the CPU usage is probably close to 100%, so my solution doesn't make the situation much worse. If load spikes are rare the application should quickly recover from it and if excessive load continues for a long period of time, the only proper solutions are to increase the thread pool size or a hardware upgrade / more servers.

However it would be good if you can come up with a solution that minimizes the performance impact.