Actually destroy the connection when calling disconnect