AIS-Bench/AIS-Bench huggingface dataset:
World Model Implanting for Test-time Adaptation of Embodied Agents
Minjong Yoo 1 Jinwoo Jang 1 Sihyung Yoon 1 Honguk Woo 1
Abstract
In embodied AI, a persistent challenge is enabling
agents to robustly adapt to novel domains with-
out requiring extensive data collection or retrain-
ing. To address this, we present a world model
implanting framework (WorMI) that combines
the reasoning capabilities of large language mod-
els (LLMs) with independently learned, domain-
specific world models through test-time com-
position.
By allowing seamless implantation
and removal of the world models, the embod-
ied agent’s policy achieves and maintains cross-
domain adaptability. In the WorMI framework,
we employ a prototype-based world model re-
trieval approach, utilizing efficient trajectory-
based abstract representation matching, to incor-
porate relevant models into test-time composition.
We also develop a world-wise compound attention
method that not only integrates the knowledge
from the retrieved world models but also aligns
their intermediate representations with the rea-
soning model’s representation within the agent’s
policy. This framework design effectively fuses
domain-specific knowledge from multiple world
models, ensuring robust adaptation to unseen do-
mains. We evaluate our WorMI on the Virtual-
Home and ALFWorld benchmarks, demonstrating
superior zero-shot and few-shot performance com-
pared to several LLM-based approaches across a
range of unseen domains. These results highlight
the framework’s potential for scalable, real-world
deployment in embodied agent scenarios where
adaptability and data efficiency are essential.
1. Introduction
In recent years, policies powered by large language models
(LLMs) have demonstrated remarkable success in embodied
1Department
of
Computer
Science
and
Engineering,
Sungkyunkwan University. Correspondence to: Honguk Woo
<hwoo@skku.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR ...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000090. Conference bucket: ICML.
Branches: Efficiently Seeking Optimal Sparse Decision Trees via AO*
Ayman Chaouki 1 Jesse Read 1 Albert Bifet 2 3
Abstract
Decision Tree (DT) Learning is a fundamental
problem in Interpretable Machine Learning, yet it
poses a formidable optimisation challenge. Prac-
tical algorithms have recently emerged, primarily
leveraging Dynamic Programming and Branch &
Bound. However, most of these approaches rely
on a Depth-First-Search strategy, which is ineffi-
cient when searching for DTs at high depths and
requires the definition of a maximum depth hyper-
parameter. Best-First-Search was also employed
by other methods to circumvent these issues. The
downside of this strategy is its higher memory
consumption, as such, it has to be designed in a
fully efficient manner that takes full advantage of
the problem’s structure. We formulate the prob-
lem within an AND/OR graph search framework
and we solve it with a novel AO*-type algorithm
called BRANCHES. We prove both optimality and
complexity guarantees for BRANCHES and we
show that it is more efficient than the state of
the art theoretically and on a variety of experi-
ments. Furthermore, BRANCHES supports non-
binary features unlike the other methods, we show
that this property can further induce larger gains
in computational efficiency.
1. Introduction
Black-box models are ill-suited for contexts where decisions
carry substantial ramifications. In healthcare for example,
misdiagnoses can delay crucial treatments and lead to severe
outcomes. Likewise, in the criminal justice system, black-
box models can obscure biases and result in discriminatory
rulings. Such risks highlight the necessity of adopting inter-
pretable models in sensitive domains.
Decision Trees (DTs) are highly interpretable due to their
1LIX, Ecole Polytechnique, IP Paris 2AI Institute, University
of Waikato 3LTCI, T´el´ecom Paris, IP Paris. Correspondence to:
Ayman Chaouki <chaoukiayman2@gmail.com>.
Proceedings of the 42 nd International Conference on Mach...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000091. Conference bucket: ICML.
Distillation Scaling Laws
Dan Busbridge 1 Amitis Shidani 2 Floris Weers 1 Jason Ramapuram 1 Etai Littwin 1 Russ Webb 1
Abstract
We propose a distillation scaling law that esti-
mates distilled model performance based on a
compute budget and its allocation between the
student and teacher. Our findings mitigate the
risks associated with large-scale distillation by
enabling compute-optimal allocation for both the
teacher and student to maximize student perfor-
mance. We provide compute-optimal distillation
recipes for two key scenarios: when a teacher
already exists, and when a teacher needs train-
ing. In settings involving many students or an
existing teacher, distillation outperforms super-
vised learning up to a compute level that scales
predictably with student size. Conversely, if only
one student is to be distilled and a teacher also re-
quires training, supervised learning is generally
preferable.
Additionally, our large-scale study
of distillation increases our understanding of the
process and helps inform experimental design.
1. Introduction
The study of scaling laws (Hestness et al., 2017; Rosen-
feld et al., 2020; Kaplan et al., 2020; Hoffmann et al.,
2022) revealed that previously trained Language Models
(LMs) could have been more capable if they had followed
a compute optimal training paradigm, which determines
the model size and the number of training tokens that give
the best performing model under a given compute budget.
Many subsequent works have followed compute optimal
training (Dey et al., 2023; Muennighoff et al., 2023b).
The size of compute optimal models grows with compute
(Hoffmann et al., 2022), which makes them challenging
to use due to the growth in inference costs. In practice,
1Apple 2University of Oxford, UK. Work done during an
internship at Apple.
For a full breakdown of contributions
see Appendix J. Correspondence to:
Dan Busbridge <dbus-
bridge@apple.com>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouve...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000092. Conference bucket: ICML.
Diffusion Adaptive Text Embedding for
Text-to-Image Diffusion Models
Byeonghu Na1
Minsang Park1
Gyuwon Sim1
Donghyeok Shin1
HeeSun Bae1
Mina Kang1
Se Jung Kwon2
Wanmo Kang1
Il-Chul Moon1,3
1KAIST, 2NAVER Cloud, 3summary.ai
{byeonghu.na,pagemu,gkwlaks4886,tlsehdgur0,cat2507,kasong13}@kaist.ac.kr,
sejung.kwon@navercorp.com , {wanmo.kang,icmoon}@kaist.ac.kr
Abstract
Text-to-image diffusion models rely on text embeddings from a pre-trained text
encoder, but these embeddings remain fixed across all diffusion timesteps, limiting
their adaptability to the generative process. We propose Diffusion Adaptive Text
Embedding (DATE), which dynamically updates text embeddings at each diffusion
timestep based on intermediate perturbed data. We formulate an optimization
problem and derive an update rule that refines the text embeddings at each sampling
step to improve alignment and preference between the mean predicted image and
the text. This allows DATE to dynamically adapts the text conditions to the
reverse-diffused images throughout diffusion sampling without requiring additional
model training. Through theoretical analysis and empirical results, we show that
DATE maintains the generative capability of the model while providing superior
text-image alignment over fixed text embeddings across various tasks, including
multi-concept generation and text-guided image editing. Our code is available at
https://github.com/aailab-kaist/DATE.
1
Introduction
Text-to-image generation has recently received significant attention due to its capability to generate
realistic and semantically accurate images from textual prompts. This progress has been largely driven
by diffusion models [18, 53], particularly with large-scale models such as DALL·E [45] and Stable
Diffusion [47]. These models use pre-trained text encoders like CLIP [42] and T5 [43] to encode
prompts into embeddings, providing crucial semantic information to diffusion models. Notably, the
quality and semantic alignment of the gener...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000007. Conference bucket: NeurIPS.
Training-Free Safe Text Embedding Guidance for
Text-to-Image Diffusion Models
Byeonghu Na1
Mina Kang1
Jiseok Kwak1
Minsang Park1
Jiwoo Shin1
SeJoon Jun1
Gayoung Lee2
Jin-Hwa Kim2,3
Il-Chul Moon1,4
1KAIST, 2NAVER AI Lab, 3SNU AIIS, 4summary.ai
{byeonghu.na,kasong13,jskwak,pagemu,natu33,sjmathy,icmoon}@kaist.ac.kr,
{gayoung.lee,j1nhwa.kim}@navercorp.com
Abstract
Text-to-image models have recently made significant advances in generating realis-
tic and semantically coherent images, driven by advanced diffusion models and
large-scale web-crawled datasets. However, these datasets often contain inappropri-
ate or biased content, raising concerns about the generation of harmful outputs when
provided with malicious text prompts. We propose Safe Text embedding Guidance
(STG), a training-free approach to improve the safety of diffusion models by guid-
ing the text embeddings during sampling. STG adjusts the text embeddings based
on a safety function evaluated on the expected final denoised image, allowing the
model to generate safer outputs without additional training. Theoretically, we show
that STG aligns the underlying model distribution with safety constraints, thereby
achieving safer outputs while minimally affecting generation quality. Experiments
on various safety scenarios, including nudity, violence, and artist-style removal,
show that STG consistently outperforms both training-based and training-free base-
lines in removing unsafe content while preserving the core semantic intent of input
prompts. Our code is available at https://github.com/aailab-kaist/STG.
Warning: This paper contains model-generated content that may be disturbing.
1
Introduction
Recent advances in text-to-image models have received considerable attention for their ability to
generate realistic images that semantically align with given text prompts [30, 32, 34, 44]. These
advances have largely been driven by the development of diffusion models [9, 33] and the avail-
ability of large-scale datasets...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000008. Conference bucket: NeurIPS.
On topological descriptors for graph products
Mattie Ji
University of Pennsylvania
mji13@sas.upenn.edu
Amauri H. Souza
Federal Institute of Ceará
amauriholanda@ifce.edu.br
Vikas Garg
Aalto University
YaiYai Ltd
vgarg@csail.mit.edu
Abstract
Topological descriptors have been increasingly utilized for capturing multiscale
structural information in relational data. In this work, we consider various filtrations
on the (box) product of graphs and the effect on their outputs on the topological de-
scriptors - the Euler characteristic (EC) and persistent homology (PH). In particular,
we establish a complete characterization of the expressive power of EC on general
color-based filtrations. We also show that the PH descriptors of (virtual) graph prod-
ucts contain strictly more information than the computation on individual graphs,
whereas EC does not. Additionally, we provide algorithms to compute the PH dia-
grams of the product of vertex- and edge-level filtrations on the graph product. We
also substantiate our theoretical analysis with empirical investigations on runtime
analysis, expressivity, and graph classification performance. Overall, this work
paves way for powerful graph persistent descriptors via product filtrations. Code is
available at https://github.com/Aalto-QuML/tda_graph_product.
1
Introduction
Message-passing GNNs are prominent models for graph representation learning [19]. However, they
are bounded in expressivity by the WL hierarchy [33–35, 50] and cannot compute fundamental graph
properties such as cycles or connected components [10, 18]. Topological descriptors (TDs) such
as those based on persistent homology (PH) and the Euler characteristics (EC) can provide such
information and thus are being increasingly employed to augment GNNs, boosting their empirical
performance [7–9, 24, 48, 51].
In this work, we study two prominent types of topological descriptors based on PH [1, 13] and EC
[40, 42, 46] respectively. Both descriptors are defined with respect ...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000011. Conference bucket: NeurIPS.
Private Evolution Converges
Tomas Gonzalez
Carnegie Mellon University
tcgonzal@andrew.cmu.edu
Giulia Fanti
Carnegie Mellon University
gfanti@andrew.cmu.edu
Aaditya Ramdas
Carnegie Mellon University
aramdas@cs.cmu.edu
Abstract
Private Evolution (PE) is a promising training-free method for differentially private
(DP) synthetic data generation. While it achieves strong performance in some
domains (e.g., images and text), its behavior in others (e.g., tabular data) is less
consistent. To date, the only theoretical analysis of the convergence of PE depends
on unrealistic assumptions about both the algorithm’s behavior and the structure
of the sensitive dataset. In this work, we develop a new theoretical framework
to understand PE’s practical behavior and identify sufficient conditions for its
convergence. For d-dimensional sensitive datasets with n data points from a
convex and compact domain, we prove that under the right hyperparameter settings
and given access to the Gaussian variation API proposed in [33], PE produces an
(ε, δ)-DP synthetic dataset with expected 1-Wasserstein distance ˜O(d(nε)−1/d)
from the original; this establishes worst-case convergence of the algorithm as
n →∞. Our analysis extends to general Banach spaces as well. We also connect
PE to the Private Signed Measure Mechanism, a method for DP synthetic data
generation that has thus far not seen much practical adoption. We demonstrate the
practical relevance of our theoretical findings in experiments.
1
Introduction
Many modern machine learning applications rely on user data, making data privacy protection a
central concern. In this regard, differential privacy (DP) [17, 19] has become a de facto standard for
safeguarding sensitive information of individuals. Given a private dataset, many problems—including
regression [40, 12], deep learning [1], and stochastic optimization [6, 2]—can be performed in a DP
manner. While it is possible to design a different DP algorithm for each specific task, a reason...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000012. Conference bucket: NeurIPS.
Private Evolution Converges
Tomas Gonzalez
Carnegie Mellon University
tcgonzal@andrew.cmu.edu
Giulia Fanti
Carnegie Mellon University
gfanti@andrew.cmu.edu
Aaditya Ramdas
Carnegie Mellon University
aramdas@cs.cmu.edu
Abstract
Private Evolution (PE) is a promising training-free method for differentially private
(DP) synthetic data generation. While it achieves strong performance in some
domains (e.g., images and text), its behavior in others (e.g., tabular data) is less
consistent. To date, the only theoretical analysis of the convergence of PE depends
on unrealistic assumptions about both the algorithm’s behavior and the structure
of the sensitive dataset. In this work, we develop a new theoretical framework
to understand PE’s practical behavior and identify sufficient conditions for its
convergence. For d-dimensional sensitive datasets with n data points from a
convex and compact domain, we prove that under the right hyperparameter settings
and given access to the Gaussian variation API proposed in [33], PE produces an
(ε, δ)-DP synthetic dataset with expected 1-Wasserstein distance ˜O(d(nε)−1/d)
from the original; this establishes worst-case convergence of the algorithm as
n →∞. Our analysis extends to general Banach spaces as well. We also connect
PE to the Private Signed Measure Mechanism, a method for DP synthetic data
generation that has thus far not seen much practical adoption. We demonstrate the
practical relevance of our theoretical findings in experiments.
1
Introduction
Many modern machine learning applications rely on user data, making data privacy protection a
central concern. In this regard, differential privacy (DP) [17, 19] has become a de facto standard for
safeguarding sensitive information of individuals. Given a private dataset, many problems—including
regression [40, 12], deep learning [1], and stochastic optimization [6, 2]—can be performed in a DP
manner. While it is possible to design a different DP algorithm for each specific task, a reason...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000013. Conference bucket: NeurIPS.
Activation-Informed Merging of
Large Language Models
Amin Heyrani Nobari∗1, Kaveh Alim∗1, Ali ArjomandBigdeli2
Akash Srivastava3, Faez Ahmed1, Navid Azizan1
1Massachusetts Institute of Technology
2Stony Brook University
3MIT-IBM Watson AI Lab & Red Hat AI Innovation
Abstract
Model merging, a method that combines the parameters and embeddings of mul-
tiple fine-tuned large language models (LLMs), offers a promising approach to
enhance model performance across various tasks while maintaining computational
efficiency. This paper introduces Activation-Informed Merging (AIM), a technique
that integrates the information from the activation space of LLMs into the merging
process to improve performance and robustness. AIM is designed as a flexible,
complementary solution that is applicable to any existing merging method. It aims
to preserve critical weights from the base model, drawing on principles from con-
tinual learning (CL) and model compression. Utilizing a task-agnostic calibration
set, AIM selectively prioritizes essential weights during merging. We empirically
demonstrate that AIM significantly enhances the performance of merged models
across multiple benchmarks. Our findings suggest that considering the activation-
space information can provide substantial advancements in the model merging
strategies for LLMs with up to 40% increase in benchmark performance. Our code
is publicly available at https://github.com/ahnobari/ActivationInformedMerging.
1
Introduction
Foundation models are rapidly becoming the dominant force for building Artificial Intelligence (AI)
systems. In many cases, researchers build their machine learning models by starting from pre-trained
foundation models and fine-tuning (FT) these pre-trained models for some desired target task [27]. In
such a paradigm, numerous fine-tuned models are developed for various tasks. However, an important
opportunity is missed, as these fine-tuned task-specialized models typically operate in isolation
without leve...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000017. Conference bucket: NeurIPS.
Ascent Fails to Forget
Ioannis Mavrothalassitis∗
Pol Puigdemont∗
Noam Itzhak Levi∗
Volkan Cevher
LIONS, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
{ioannis.mavrothalassitis, pol.puigdemontplana, noam.levi}@epfl.ch
Abstract
Contrary to common belief, we show that gradient ascent-based unconstrained
optimization methods frequently fail to perform machine unlearning, a phenomenon
we attribute to the inherent statistical dependence between the forget and retain
data sets. This dependence, which can manifest itself even as simple correlations,
undermines the misconception that these sets can be independently manipulated
during unlearning. We provide empirical and theoretical evidence showing these
methods often fail precisely due to this overlooked relationship. For random forget
sets, this dependence means that degrading forget set metrics (which, for the oracle,
should mirror test set metrics) inevitably harms overall test performance. Going
beyond random sets, we consider logistic regression as an instructive example
where a critical failure mode emerges: inter-set dependence causes gradient descent-
ascent iterations to progressively diverge from the oracle. Strikingly, these methods
can converge to solutions that are not only far from the oracle but are potentially
even further from it than the original model itself, rendering the unlearning process
actively detrimental. A toy example further illustrates how this dependence can
trap models in inferior local minima, inescapable via finetuning. Our findings
highlight that the presence of such statistical dependencies, even when manifest
only as correlations, can be sufficient for ascent-based unlearning to fail. Our
theoretical insights are corroborated by experiments on complex neural networks,
demonstrating that these methods do not perform as expected in practice due to
this unaddressed statistical interplay.
1
Introduction
Machine learning models have become an integral part of modern r...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000019. Conference bucket: NeurIPS.
Private Evolution Converges
Tomas Gonzalez
Carnegie Mellon University
tcgonzal@andrew.cmu.edu
Giulia Fanti
Carnegie Mellon University
gfanti@andrew.cmu.edu
Aaditya Ramdas
Carnegie Mellon University
aramdas@cs.cmu.edu
Abstract
Private Evolution (PE) is a promising training-free method for differentially private
(DP) synthetic data generation. While it achieves strong performance in some
domains (e.g., images and text), its behavior in others (e.g., tabular data) is less
consistent. To date, the only theoretical analysis of the convergence of PE depends
on unrealistic assumptions about both the algorithm’s behavior and the structure
of the sensitive dataset. In this work, we develop a new theoretical framework
to understand PE’s practical behavior and identify sufficient conditions for its
convergence. For d-dimensional sensitive datasets with n data points from a
convex and compact domain, we prove that under the right hyperparameter settings
and given access to the Gaussian variation API proposed in [33], PE produces an
(ε, δ)-DP synthetic dataset with expected 1-Wasserstein distance ˜O(d(nε)−1/d)
from the original; this establishes worst-case convergence of the algorithm as
n →∞. Our analysis extends to general Banach spaces as well. We also connect
PE to the Private Signed Measure Mechanism, a method for DP synthetic data
generation that has thus far not seen much practical adoption. We demonstrate the
practical relevance of our theoretical findings in experiments.
1
Introduction
Many modern machine learning applications rely on user data, making data privacy protection a
central concern. In this regard, differential privacy (DP) [17, 19] has become a de facto standard for
safeguarding sensitive information of individuals. Given a private dataset, many problems—including
regression [40, 12], deep learning [1], and stochastic optimization [6, 2]—can be performed in a DP
manner. While it is possible to design a different DP algorithm for each specific task, a reason...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000020. Conference bucket: NeurIPS.