apache/incubator-flink — pull request #76 by uce (2014-07-21 11:50:30+00:00)
Title: [FLINK-998] Close network connections when idle

1. **Close idle TCP connections**

Tasks enqueue envelopes to be dispatched via NettyConnectionManager. The
first task to enqueue an envelope triggers the establishment of a new
TCP connection to the target task manager. Usually, there are multiple
connections between the same task managers in order to avoid deadlocks.

This commit addresses the following problem: idle connections were never
closed. This meant that the number of TCP connections was monotonically
increasing over time. Idling connections are now closed after a
configured amount of time, see `taskmanager.net.closeAfterIdleForMs`.

2. **Enqueue envelopes directly**

TCP connections are handled by Netty's IO loop. Every enqueue operation
needs to ensure that the envelope to be enqueued is handed over to the
respective IO thread in a thread-safe manner. Previously, this was done
via Netty's user events.

This commit addresses the following problem: every enqueue operation
resulted in the creation of an IO event loop task to hand over the
envelope. Envelopes are now directly added to a `ConcurrentLinkedQueue`,
which improves the throughput of enqueue operations. The
user event is now only fired to trigger the processing if the queue was
empty.

3. **Added unit tests for OutboundConnectionQueue and adapted existing
tests to test concurrent enqueue and close**

4. **Removed unused arguments to NettyConnectionManager (low and high
water mark)**