Do not hold connection lock while closing channels