Try to correctly handle cases where closing a peer connection doesn't work (doesn't result in the OS giving us the EOF we expect)
Fix one place the peer wasn't in the list we expected