Call invalidate_fd in disconnect_and_raise