Remove the event listeners attached to the socket when it gets closed

Just revert it if the server starts crashing because we removed the error listeners too early :smile: