[fix] Remove the event listeners attached to the socket when it gets closed