dataset-review/dataset-review huggingface dataset:
Locality Alignment Improves Vision-Language Models
ICLR 2025

Published as a conference paper at ICLR 2025
LOCALITY ALIGNMENT IMPROVES VISION-LANGUAGE
MODELS
Ian Covert, Tony Sun, James Zou∗, Tatsunori Hashimoto∗
Stanford University
{icovert, suntony, jamesz, thashim}@stanford.edu
ABSTRACT
Vision language models (VLMs) have seen growing adoption in recent years,
but many still struggle with basic spatial reasoning errors. We hypothesize that
this is due to VLMs adopting pre-trained vision backbones, specifically vision
transformers (ViTs) trained with image-level supervision and minimal inductive
biases. Such models may fail to encode the class contents at each position in
the image, and our goal is to resolve this with a vision backbone that effectively
captures both local and global image semantics. Our main insight is that we do
not require new supervision to learn this capability – pre-trained models contain
significant knowledge of local semantics that we can extract and use for scalable
self-supervision. We propose a new efficient post-training stage for ViTs called
locality alignment and a novel fine-tuning procedure called MaskEmbed that uses
a masked reconstruction loss to learn semantic contributions for each image patch.
We first evaluate locality alignment with a vision-only benchmark, finding that it
improves a model’s performance at patch-level semantic segmentation, especially
for strong backbones trained with image-caption pairs (e.g., CLIP and SigLIP). We
then train a series of VLMs with and without locality alignment, and show that
locality-aligned backbones improve performance across a range of benchmarks,
particularly ones that involve spatial understanding (e.g., RefCOCO, OCID-Ref,
TallyQA, VSR, AI2D). Overall, we demonstrate that we can efficiently learn local
semantic extraction via a locality alignment stage, and that this procedure benefits
VLM training recipes that use off-the-shelf vision backbones.
1
INTRODUCTION
Auto-regressive VLMs are an exciting new type of model that emerged in the last couple ...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000009. Conference bucket: ICLR.
Robust Simulation-Based Inference under Missing Data via Neural Processes
ICLR 2025

Published as a conference paper at ICLR 2025
ROBUST SIMULATION-BASED INFERENCE
UNDER MISSING DATA VIA NEURAL PROCESSES
Yogesh Verma, Ayush Bharti
Department of Computer Science, Aalto University
{yogesh.verma, ayush.bharti}@aalto.fi
Vikas Garg
YaiYai Ltd and Aalto University
vgarg@csail.mit.edu
ABSTRACT
Simulation-based inference (SBI) methods typically require fully observed data to
infer parameters of models with intractable likelihood functions. However, datasets
often contain missing values due to incomplete observations, data corruptions
(common in astrophysics), or instrument limitations (e.g., in high-energy physics
applications). In such scenarios, missing data must be imputed before applying any
SBI method. We formalize the problem of missing data in SBI and demonstrate that
naive imputation methods can introduce bias in the estimation of SBI posterior. We
also introduce a novel amortized method that addresses this issue by jointly learning
the imputation model and the inference network within a neural posterior estimation
(NPE) framework. Extensive empirical results on SBI benchmarks show that our
approach provides robust inference outcomes compared to standard baselines
for varying levels of missing data. Moreover, we demonstrate the merits of our
imputation model on two real-world bioactivity datasets (Adrenergic and Kinase
assays). Code is available at https://github.com/Aalto-QuML/RISE.
1
INTRODUCTION
Mechanistic models for studying complex physical or biological phenomena have become indispens-
able tools in research fields as diverse as genetics (Riesselman et al., 2018), epidemiology (Kypraios
et al., 2017), gravitational wave astronomy (Dax et al., 2021), and radio propagation (Bharti et al.,
2022a). However, fitting such models to observational data can be challenging due to the intractabil-
ity of their likelihood functions, which renders standard Bayesian inference methods inapplicable.
Simulation-based inference (SBI) methods (Cranmer et al., 2...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000010. Conference bucket: ICLR.
OptionZero: Planning with Learned Options
ICLR 2025

Published as a conference paper at ICLR 2025
OPTIONZERO: PLANNING WITH LEARNED OPTIONS
Po-Wei Huang1,2, Pei-Chiun Peng1,2, Hung Guei1, Ti-Rong Wu1†
1Institute of Information Science, Academia Sinica, Taiwan
2Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan
ABSTRACT
Planning with options – a sequence of primitive actions – has been shown ef-
fective in reinforcement learning within complex environments. Previous studies
have focused on planning with predefined options or learned options through ex-
pert demonstration data. Inspired by MuZero, which learns superhuman heuristics
without any human knowledge, we propose a novel approach, named OptionZero.
OptionZero incorporates an option network into MuZero, providing autonomous
discovery of options through self-play games. Furthermore, we modify the dy-
namics network to provide environment transitions when using options, allowing
searching deeper under the same simulation constraints. Empirical experiments
conducted in 26 Atari games demonstrate that OptionZero outperforms MuZero,
achieving a 131.58% improvement in mean human-normalized score. Our behav-
ior analysis shows that OptionZero not only learns options but also acquires strate-
gic skills tailored to different game characteristics. Our findings show promising
directions for discovering and using options in planning. Our code is available at
https://rlg.iis.sinica.edu.tw/papers/optionzero.
1
INTRODUCTION
Reinforcement learning is a decision-making process in which an agent interacts with environments
by selecting actions at each step to maximize long-term rewards. Actions, commonly referred to as
primitive action, advance the state by one step each. While this granularity allows precise control
at each time step, it can lead to inefficiencies in scenarios where predictable sequences of actions
are beneficial. For example, in a maze navigation task, it is more efficient to choose a sequence
of actions – such as following a strai...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000018. Conference bucket: ICLR.
Coreset Spectral Clustering
ICLR 2025

Published as a conference paper at ICLR 2025
CORESET SPECTRAL CLUSTERING
Ben Jourdan1, Gregory Schwartzman2, Peter Macgregor3, and He Sun1
1University of Edinburgh, UK
2Japan Advanced Institute of Science and Technology (JAIST), Japan
3University of St Andrews, UK
ben.jourdan@ed.ac.uk, greg@jaist.ac.jp, prm4@st-andrews.ac.uk,
h.sun@ed.ac.uk
ABSTRACT
Coresets have become an invaluable tool for solving k-means and kernel k-means clustering
problems on large datasets with small numbers of clusters. On the other hand, spectral clustering
works well on sparse graphs and has recently been extended to scale efficiently to large numbers
of clusters. We exploit the connection between kernel k-means and the normalised cut problem
to combine the benefits of both. Our main result is a coreset spectral clustering algorithm
for graphs that clusters a coreset graph to infer a good labelling of the original graph. We
prove that an α-approximation for the normalised cut problem on the coreset graph is an O(α)-
approximation on the original. We also improve the running time of the state-of-the-art coreset
algorithm for kernel k-means on sparse kernels, from ˜O(nk) to ˜O(n · min{k, davg}), where
davg is the average number of non-zero entries in each row of the n × n kernel matrix. Our
experiments confirm our coreset algorithm is asymptotically faster on large real-world graphs
with many clusters, and show that our clustering algorithm overcomes the main challenge faced
by coreset kernel k-means on sparse kernels which is getting stuck in local optima.
1
INTRODUCTION
Kernel k-means and spectral clustering are two popular algorithms which are capable of learning
non-linear decision boundaries between clusters. For this reason, they have been applied to many
practical problems in machine learning, including in the fields of medical research and network
science (Gönen & Margolin, 2014; Kuo et al., 2014; White & Smyth, 2005).
Given a data set X, both kernel k-means and spectral clustering ...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000023. Conference bucket: ICLR.
EmbedLLM: Learning Compact Representations of Large Language Models
ICLR 2025

Published as a conference paper at ICLR 2025
EMBEDLLM:
LEARNING
COMPACT
REPRESENTA-
TIONS OF LARGE LANGUAGE MODELS
Richard Zhuang∗Tianhao Wu∗
Zhaojin Wen
Andrew Li
Jiantao Jiao
Kannan Ramchandran
University of California, Berkeley
ABSTRACT
With hundreds of thousands of language models available on Huggingface to-
day, efficiently evaluating and utilizing these models across various downstream
tasks has become increasingly critical. Many existing methods repeatedly learn
task-specific representations of Large Language Models (LLMs), which leads to
inefficiencies in both time and computational resources. To address this, we pro-
pose EmbedLLM, a framework designed to learn compact vector representations
of LLMs that facilitate downstream applications involving many models, such as
model routing. We introduce an encoder-decoder approach for learning such em-
beddings, along with a systematic framework to evaluate their effectiveness. Em-
pirical results show that EmbedLLM outperforms prior methods in model routing
both in accuracy and latency. Additionally, we demonstrate that our method can
forecast a model’s performance on multiple benchmarks, without incurring ad-
ditional inference cost. Extensive probing experiments validate that the learned
embeddings capture key model characteristics, e.g. whether the model is special-
ized for coding tasks, even without being explicitly trained on them. We open
source our dataset, code and embedder to facilitate further research and applica-
tion: https://github.com/richardzhuang0412/EmbedLLM.
1
INTRODUCTION
Recent breakthroughs in Large Language Models (LLMs) (Vaswani et al., 2023) have led to the
creation of a vast array of models, each tailored for different use cases. These models, ranging from
small, specialized models to large, general-purpose systems (Hao et al., 2022), differ significantly
in their architecture, size, training data, and performance characteristics. For example, while some
models excel as conversational ...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000029. Conference bucket: ICLR.
CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferences
ICLR 2025

Published as a conference paper at ICLR 2025
CAKE: CASCADING
AND ADAPTIVE KV CACHE
EVICTION WITH LAYER PREFERENCES
Ziran Qin1,2∗, Yuchen Cao2, Mingbao Lin3†, Wen Hu2, Shixuan Fan2, Ke Cheng2,
Weiyao Lin1†, Jianguo Li2†
1Shanghai Jiao Tong University, 2Ant Group, 3Independent Researcher
ABSTRACT
Large language models (LLMs) excel at processing long sequences, boosting de-
mand for key-value (KV) caching. While recent efforts to evict KV cache have al-
leviated the inference burden, they often fail to allocate resources rationally across
layers with different attention patterns. In this paper, we introduce Cascading and
Adaptive KV cache Eviction (CAKE), a novel approach that frames KV cache
eviction as a “cake-slicing problem.” CAKE assesses layer-specific preferences
by considering attention dynamics in both spatial and temporal dimensions, allo-
cates rational cache size for layers accordingly, and manages memory constraints
in a cascading manner. This approach enables a global view of cache alloca-
tion, adaptively distributing resources across diverse attention mechanisms while
maintaining memory budgets. CAKE also employs a new eviction indicator that
considers the shifting importance of tokens over time, addressing limitations in
existing methods that overlook temporal dynamics. Comprehensive experiments
on LongBench and NeedleBench show that CAKE maintains model performance
with only 3.2% of the KV cache and consistently outperforms current baselines
across various models and memory constraints, particularly in low-memory set-
tings. Additionally, CAKE achieves over 10× speedup in decoding latency com-
pared to full cache when processing contexts of 128K tokens with FlashAttention-
2. Our code is available at https://github.com/antgroup/cakekv.
1
INTRODUCTION
Large language models (LLMs) (Zhao et al., 2023; Achiam et al., 2023; Dubey et al., 2024; An-
thropic, 2024; AI, 2024) have enhanced their long-text processing capabilities, improving perfor-
mance in mul...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000037. Conference bucket: ICLR.
Neural Interactive Proofs
ICLR 2025

Published as a conference paper at ICLR 2025
NEURAL INTERACTIVE PROOFS
Lewis Hammond∗
lewis.hammond@cs.ox.ac.uk
Sam Adam-Day∗
sam.adam-day@cs.ox.ac.uk
Department of Computer Science, University of Oxford, Oxford, United Kingdom
ABSTRACT
We consider the problem of how a trusted, but computationally bounded agent
(a ‘verifier’) can learn to interact with one or more powerful but untrusted agents
(‘provers’) in order to solve a given task. More specifically, we study the case in
which agents are represented using neural networks and refer to solutions of this
problem as neural interactive proofs. First we introduce a unifying framework based
on prover-verifier games (Anil et al., 2021), which generalises previously proposed
interaction protocols. We then describe several new protocols for generating neural
interactive proofs, and provide a theoretical comparison of both new and existing
approaches. Finally, we support this theory with experiments in two domains: a
toy graph isomorphism problem that illustrates the key ideas, and a code validation
task using large language models. In so doing, we aim to create a foundation for
future work on neural interactive proofs and their application in building safer AI
systems.
1
INTRODUCTION
Recent years have witnessed the proliferation of large machine learning (ML) systems (Villalobos
et al., 2022), useful for solving an increasingly wide range of tasks. Often, however, it can be difficult
to trust the output of these systems, raising concerns about their safety and limiting their applicability
in high-stakes situations (Amodei et al., 2016; Bengio et al., 2023; Hendrycks et al., 2023). At
the same time, traditional approaches in verification do not scale to today’s most powerful systems
(Seshia et al., 2022). There is thus a pressing need to identify new angles via which to gain such
assurances.
qƧǩƽΞƧǩ
UǩǙͭƧǩ
Figure 1: On receiving input x from distribution P the
agents exchange messages before the verifier decides on
an out...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000040. Conference bucket: ICLR.
Neural Interactive Proofs
ICLR 2025

Published as a conference paper at ICLR 2025
NEURAL INTERACTIVE PROOFS
Lewis Hammond∗
lewis.hammond@cs.ox.ac.uk
Sam Adam-Day∗
sam.adam-day@cs.ox.ac.uk
Department of Computer Science, University of Oxford, Oxford, United Kingdom
ABSTRACT
We consider the problem of how a trusted, but computationally bounded agent
(a ‘verifier’) can learn to interact with one or more powerful but untrusted agents
(‘provers’) in order to solve a given task. More specifically, we study the case in
which agents are represented using neural networks and refer to solutions of this
problem as neural interactive proofs. First we introduce a unifying framework based
on prover-verifier games (Anil et al., 2021), which generalises previously proposed
interaction protocols. We then describe several new protocols for generating neural
interactive proofs, and provide a theoretical comparison of both new and existing
approaches. Finally, we support this theory with experiments in two domains: a
toy graph isomorphism problem that illustrates the key ideas, and a code validation
task using large language models. In so doing, we aim to create a foundation for
future work on neural interactive proofs and their application in building safer AI
systems.
1
INTRODUCTION
Recent years have witnessed the proliferation of large machine learning (ML) systems (Villalobos
et al., 2022), useful for solving an increasingly wide range of tasks. Often, however, it can be difficult
to trust the output of these systems, raising concerns about their safety and limiting their applicability
in high-stakes situations (Amodei et al., 2016; Bengio et al., 2023; Hendrycks et al., 2023). At
the same time, traditional approaches in verification do not scale to today’s most powerful systems
(Seshia et al., 2022). There is thus a pressing need to identify new angles via which to gain such
assurances.
qƧǩƽΞƧǩ
UǩǙͭƧǩ
Figure 1: On receiving input x from distribution P the
agents exchange messages before the verifier decides on
an out...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000045. Conference bucket: ICLR.
Discrete Copula Diffusion
ICLR 2025

Published as a conference paper at ICLR 2025
DISCRETE COPULA DIFFUSION
Anji Liu1,2, Oliver Broadrick1, Mathias Niepert2, Guy Van den Broeck1
1Department of Computer Science, University of California, Los Angeles, USA
2Institute for Artificial Intelligence, University of Stuttgart, Germany
{liuanji,obroadrick,guyvdb}@cs.ucla.edu
mathias.niepert@ki.uni-stuttgart.de
ABSTRACT
Discrete diffusion models have recently shown significant progress in modeling
complex data, such as natural languages and DNA sequences. However, unlike
diffusion models for continuous data, which can generate high-quality samples in
just a few denoising steps, modern discrete diffusion models still require hundreds
or even thousands of denoising steps to perform well. In this paper, we identify
a fundamental limitation that prevents discrete diffusion models from achieving
strong performance with fewer steps – they fail to capture dependencies between
output variables at each denoising step. To address this issue, we provide a for-
mal explanation and introduce a general approach to supplement the missing de-
pendency information by incorporating another deep generative model, termed
the copula model. Our method does not require fine-tuning either the diffusion
model or the copula model, yet it enables high-quality sample generation with
significantly fewer denoising steps. When we apply this approach to autoregres-
sive copula models, the combined model outperforms both models individually
in unconditional and conditional text generation. Specifically, the hybrid model
achieves better (un)conditional text generation using 8 to 32 times fewer denois-
ing steps than the diffusion model alone. In addition to presenting an effective
discrete diffusion generation algorithm, this paper emphasizes the importance of
modeling inter-variable dependencies in discrete diffusion.1
1
INTRODUCTION
Discrete diffusion models have recently achieved significant progress in modeling complex data such
as natural lan...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000050. Conference bucket: ICLR.
Moral Alignment for LLM Agents
ICLR 2025

Published as a conference paper at ICLR 2025
MORAL ALIGNMENT FOR LLM AGENTS
Elizaveta Tennant
University College London
University of Bologna
l.karmannaya.16@ucl.ac.uk
Stephen Hailes
University College London
s.hailes@ucl.ac.uk
Mirco Musolesi
University College London
University of Bologna
m.musolesi@ucl.ac.uk
ABSTRACT
Decision-making agents based on pre-trained Large Language Models (LLMs)
are increasingly being deployed across various domains of human activity. While
their applications are currently rather specialized, several research efforts are un-
derway to develop more generalist agents. As LLM-based systems become more
agentic, their influence on human activity will grow and their transparency will de-
crease. Consequently, developing effective methods for aligning them to human
values is vital.
The prevailing practice in alignment often relies on human preference data (e.g., in
RLHF or DPO), in which values are implicit, opaque and are essentially deduced
from relative preferences over different model outputs. In this work, instead of
relying on human feedback, we introduce the design of reward functions that ex-
plicitly and transparently encode core human values for Reinforcement Learning-
based fine-tuning of foundation agent models. Specifically, we use intrinsic re-
wards for the moral alignment of LLM agents.
We evaluate our approach using the traditional philosophical frameworks of Deon-
tological Ethics and Utilitarianism, quantifying moral rewards for agents in terms
of actions and consequences on the Iterated Prisoner’s Dilemma (IPD) environ-
ment. We also show how moral fine-tuning can be deployed to enable an agent to
unlearn a previously developed selfish strategy. Finally, we find that certain moral
strategies learned on the IPD game generalize to several other matrix game en-
vironments. In summary, we demonstrate that fine-tuning with intrinsic rewards
is a promising general solution for aligning LLM agents to human values, and
it might repr...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000056. Conference bucket: ICLR.