npahucki/vert.x — pull request #4 by wangjunbo924 (2013-10-28 07:25:59+00:00)
Title: remove closed connections from pool to avoid potential memory leak.

remove closed connections from pool to avoid potential memory leak.

# issue summay
The issue happens when adapters respond a non-success http code.(Perhaps not all non-success code , but I’m sure 404 will 100% re-produce this issue) . In this case an closed connection will be referenced by the connection pool which leads to: 1) the connection pool holds more and more closed connections. It’s a memory leak. The memory keeps increasing. 2) every time a new request try to select a connection, it will iterate all connections. As the pool size increases, more and more CPU time are wasted. 3) By default a closed connection won’t be used. However, under high load, it’s easily the count of in-use connections reaches the threshold: 10 by default. Then vert.x will try to assign an closed connection to request. And since its’ closed the request will try to pick up another connecton(The first one can be released now). Probably vertx assign another closed connection. It’s an infinite loop which cause “StackOverFlowError”.

# how to fix
The key point is to remove closed connections from pool in time so as to avoid memory leak.

# Tests
Run the following cases locally using Jmeter and local ABus/ JavaEE-based Adapter():
1. normal cases with Adata/Event/Content
2. The adapter respond a non-success http code in 10% samples; normal cases in 90% samples.
3. The adapter respond before timeout in 10% samples; normal cases in 90% samples
4. non-success codes in all samples
5. timeout in all samples.