Make sure we clean up correctly after failed thrift connection