Tweak to CloseConnection

Will now not block if there is no connection