Fix ungraceful connection termination goroutine leak

Fixes ungraceful termination leak discovered by @erikh