Cleanup connection refused tests
Actually ignore IOException in closeQuietly
Fix connect timeout test on Linux

The server socket backlog is actually a hint and might be ignored
entirely. On Linux, the minimum backlog size is 16. So, we set the
minimum backlog and keep connecting until it fails with a timeout.