AIS-Bench/AIS-Bench huggingface dataset:
Exploring Large Action Sets with Hyperspherical Embeddings
using von Mises-Fisher Sampling
Walid Bendada 1 2 Guillaume Salha-Galvan 3 Romain Hennequin 1
Th´eo Bontempelli 1 Thomas Bouabc¸a 1 Tristan Cazenave 2
Abstract
This paper introduces von Mises-Fisher explo-
ration (vMF-exp), a scalable method for explor-
ing large action sets in reinforcement learning
problems where hyperspherical embedding vec-
tors represent these actions. vMF-exp involves
initially sampling a state embedding representa-
tion using a von Mises-Fisher distribution, then
exploring this representation’s nearest neighbors,
which scales to virtually unlimited numbers of
candidate actions. We show that, under theoretical
assumptions, vMF-exp asymptotically maintains
the same probability of exploring each action as
Boltzmann Exploration (B-exp), a popular alter-
native that, nonetheless, suffers from scalability
issues as it requires computing softmax values
for each action. Consequently, vMF-exp serves
as a scalable alternative to B-exp for exploring
large action sets with hyperspherical embeddings.
Experiments on simulated data, real-world public
data, and the successful large-scale deployment of
vMF-exp on the recommender system of a global
music streaming service empirically validate the
key properties of the proposed method.
1. Introduction
Exploration is a fundamental component of the reinforce-
ment learning (RL) paradigm (Amin et al., 2021; McFar-
lane, 2018; Sutton & Barto, 2018). It allows RL agents to
gather valuable information about their environment and
identify optimal actions that maximize rewards (Amin et al.,
2021; Chiappa et al., 2023; Dulac-Arnold et al., 2015; Jin
et al., 2020; Ladosz et al., 2022; McFarlane, 2018; Reynolds,
1Deezer Research, Paris, France.
2LAMSADE, Universit´e
Paris Dauphine, PSL, Paris, France. 3SPEIT, Shanghai Jiao Tong
University, Shanghai, China. Correspondence to: Walid Bendada
<bendadaw@gmail.com>.
Proceedings of the 42 nd International Conference on M...
Published as a conference paper at ICLR 2025
MORAL ALIGNMENT FOR LLM AGENTS
Elizaveta Tennant
University College London
University of Bologna
l.karmannaya.16@ucl.ac.uk
Stephen Hailes
University College London
s.hailes@ucl.ac.uk
Mirco Musolesi
University College London
University of Bologna
m.musolesi@ucl.ac.uk
ABSTRACT
Decision-making agents based on pre-trained Large Language Models (LLMs)
are increasingly being deployed across various domains of human activity. While
their applications are currently rather specialized, several research efforts are un-
derway to develop more generalist agents. As LLM-based systems become more
agentic, their influence on human activity will grow and their transparency will de-
crease. Consequently, developing effective methods for aligning them to human
values is vital.
The prevailing practice in alignment often relies on human preference data (e.g., in
RLHF or DPO), in which values are implicit, opaque and are essentially deduced
from relative preferences over different model outputs. In this work, instead of
relying on human feedback, we introduce the design of reward functions that ex-
plicitly and transparently encode core human values for Reinforcement Learning-
based fine-tuning of foundation agent models. Specifically, we use intrinsic re-
wards for the moral alignment of LLM agents.
We evaluate our approach using the traditional philosophical frameworks of Deon-
tological Ethics and Utilitarianism, quantifying moral rewards for agents in terms
of actions and consequences on the Iterated Prisoner’s Dilemma (IPD) environ-
ment. We also show how moral fine-tuning can be deployed to enable an agent to
unlearn a previously developed selfish strategy. Finally, we find that certain moral
strategies learned on the IPD game generalize to several other matrix game en-
vironments. In summary, we demonstrate that fine-tuning with intrinsic rewards
is a promising general solution for aligning LLM agents to human values, and
it might repr...
Stochastically Dominant Peer Prediction
Yichi Zhang
DIMACS, Rutgers University
yz1636@dimacs.rutgers.edu
Shengwei Xu
University of Michigan, Ann Arbor
shengwei@umich.edu
David Pennock
DIMACS, Rutgers University
dpennock@dimacs.rutgers.edu
Grant Schoenebeck
University of Michigan, Ann Arbor
schoeneb@umich.edu
Abstract
Eliciting reliable human feedback is essential for many machine learning tasks,
such as learning from noisy labels and aligning AI systems with human prefer-
ences. Peer prediction mechanisms incentivize truthful reporting without ground
truth verification by scoring agents based on correlations with peers. Traditional
mechanisms, which ensure that truth-telling maximizes the expected scores in
equilibrium, can elicit honest information while assuming agents’ utilities are
linear functions of their scores. However, in practice, non-linear payment rules
are usually preferred, or agents’ utilities are inherently non-linear.
We propose stochastically dominant truthfulness (SD-truthfulness) as a stronger
guarantee: the score distribution of truth-telling stochastically dominates all other
strategies, incentivizing truthful reporting for a wide range of monotone utility func-
tions. Our first observation is that no existing peer prediction mechanism naturally
satisfies this criterion without strong assumptions. A simple solution—rounding
scores into binary lotteries—can enforce SD-truthfulness, but often degrades sensi-
tivity, a key property related to fairness and statistical efficiency. We demonstrate
how a more careful application of rounding can better preserve sensitivity. Further-
more, we introduce a new enforced agreement (EA) mechanism that is theoretically
guaranteed to be SD-truthful in binary-signal settings and, under mild assump-
tions, empirically achieves the highest sensitivity among all known SD-truthful
mechanisms.
1
Introduction
Information elicitation studies how to design mechanisms that incentivize honest and high-effort
human feedbac...
Structure Is All You Need: Structural Representation Learning on
Hyper-Relational Knowledge Graphs
Jaejun Lee 1 Joyce Jiyoung Whang 1
Abstract
Hyper-relational knowledge graphs (HKGs) en-
rich knowledge graphs by extending a triplet to
a hyper-relational fact, where a set of qualifiers
adds auxiliary information to a triplet. While
many HKG representation learning methods have
been proposed, they often fail to effectively utilize
the HKG’s structure. This paper demonstrates that
thoroughly leveraging the structure of an HKG
is crucial for reasoning on HKGs, and a purely
structure-based representation learning method
can achieve state-of-the-art performance on var-
ious link prediction tasks. We propose MAYPL,
which learns to initialize representation vectors
based on the structure of an HKG and employs
an attentive neural message passing consisting of
fact-level message computation and entity-centric
and relation-centric aggregations, thereby comput-
ing the representations based solely on the struc-
ture. Due to its structure-driven learning, MAYPL
can conduct inductive inferences on new entities
and relations. MAYPL outperforms 40 knowledge
graph completion methods in 10 datasets, com-
pared with different baseline methods on different
datasets to be tested from diverse perspectives.
1. Introduction
Knowledge graphs (KGs) provide a structured representa-
tion of human knowledge, which eases organizing the re-
lationships between concepts or objects (Ji et al., 2022).
In KGs, information is represented in the form of a
triplet (Toutanova & Chen, 2015; Bordes et al., 2013), e.g.,
(Finding Nemo, set in, Sydney). Since the sim-
ple triplet form oversimplifies information, hyper-relational
knowledge graphs (HKGs) have been recently proposed to
add auxiliary details by adding qualifiers to a triplet (Rosso
1School of Computing, KAIST, Daejeon, South Korea. Corre-
spondence to: Joyce Jiyoung Whang <jjwhang@kaist.ac.kr>.
Proceedings of the 42 nd International Conference o...
Lightweight Dataset Pruning without Full Training
via Example Difficulty and Prediction Uncertainty
Yeseul Cho * 1 Baekrok Shin * 1 Changmin Kang 1 Chulhee Yun 1
Abstract
Recent advances in deep learning rely heavily
on massive datasets, leading to substantial stor-
age and training costs. Dataset pruning aims to
alleviate this demand by discarding redundant ex-
amples. However, many existing methods require
training a model with a full dataset over a large
number of epochs before being able to prune the
dataset, which ironically makes the pruning pro-
cess more expensive than just training the model
on the entire dataset. To overcome this limitation,
we introduce the Difficulty and Uncertainty-
Aware Lightweight (DUAL) score, which aims
to identify important samples from the early train-
ing stage by considering both example difficulty
and prediction uncertainty. To address a catas-
trophic accuracy drop at an extreme pruning ra-
tio, we further propose a pruning ratio-adaptive
sampling using Beta distribution. Experiments
on various datasets and learning scenarios such
as image classification with label noise and im-
age corruption, and model architecture general-
ization demonstrate the superiority of our method
over previous state-of-the-art (SOTA) approaches.
Specifically, on ImageNet-1k, our method reduces
the time cost for pruning to 66% compared to
previous methods while achieving a SOTA 60%
test accuracy at a 90% pruning ratio. On CIFAR
datasets, the time cost is reduced to just 15% while
maintaining SOTA performance. Implementation
is available at github/dual-pruning.
1. Introduction
Advancements in deep learning have been significantly
driven by large-scale datasets. However, recent studies have
*Equal contribution
1Kim Jaechul Graduate School of AI,
KAIST, Seoul, South Korea. Correspondence to: Chulhee Yun
<chulhee.yun@kaist.ac.kr>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by...
(How) Do Language Models Track State?
Belinda Z. Li 1 Zifan Carl Guo 1 Jacob Andreas 1
Abstract
Transformer language models (LMs) exhibit
behaviors—from storytelling to code generation—
that seem to require tracking the unobserved state
of an evolving world. How do they do this? We
study state tracking in LMs trained or fine-tuned
to compose permutations (i.e., to compute the or-
der of a set of objects after a sequence of swaps).
Despite the simple algebraic structure of this prob-
lem, many other tasks (e.g., simulation of finite
automata and evaluation of boolean expressions)
can be reduced to permutation composition, mak-
ing it a natural model for state tracking in gen-
eral. We show that LMs consistently learn one of
two state tracking mechanisms for this task. The
first closely resembles the “associative scan” con-
struction used in recent theoretical work by Liu
et al. (2023) and Merrill et al. (2024). The second
uses an easy-to-compute feature (permutation par-
ity) to partially prune the space of outputs, and
then refines this with an associative scan. LMs
that learn the former algorithm tend to general-
ize better and converge faster, and we show how
to steer LMs toward one or the other with inter-
mediate training tasks that encourage or suppress
the heuristics. Our results demonstrate that trans-
former LMs, whether pre-trained or fine-tuned,
can learn to implement efficient and interpretable
state-tracking mechanisms, and the emergence
of these mechanisms can be predicted and con-
trolled. Code and data are available at https:
//github.com/belindal/state-tracking.
1. Introduction
Language models (LMs) are trained to model the surface
form of text. A growing body of work suggests that model
internals contain a latent, decodable state of the world—e.g.,
situations described by language and results of program
1MIT EECS and CSAIL. Correspondence to: Belinda Z. Li
<bzl@mit.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouve...
Physics-Informed Generative Modeling of Wireless Channels
Benedikt B¨ock 1 Andreas Oeldemann 2 Timo Mayer 2 Francesco Rossetto 2 Wolfgang Utschick 1
Abstract
Learning the site-specific distribution of the wire-
less channel within a particular environment of
interest is essential to exploit the full potential of
machine learning (ML) for wireless communica-
tions and radar applications. Generative model-
ing offers a promising framework to address this
problem. However, existing approaches pose un-
resolved challenges, including the need for high-
quality training data, limited generalizability, and
a lack of physical interpretability. To address
these issues, we combine the physics-related com-
pressibility of wireless channels with generative
modeling, in particular, sparse Bayesian genera-
tive modeling (SBGM), to learn the distribution of
the underlying physical channel parameters. By
leveraging the sparsity-inducing characteristics of
SBGM, our methods can learn from compressed
observations received by an access point (AP) dur-
ing default online operation. Moreover, they are
physically interpretable and generalize over sys-
tem configurations without requiring retraining.
1. Introduction
The accurate modeling of wireless channels is critical for
system design, network optimization, and performance eval-
uation in wireless communications and radar (Wang et al.,
2018; 2020; Yin & Cheng, 2016). There are two approaches
to wireless channel modeling: physically and analytically
oriented (Almers et al., 2007). The former integrates the
underlying physics in the description of wireless channels
and focuses on their accuracy and realism. In contrast, the
latter captures channel statistics, ignoring the underlying
physics, and provides an easy-to-use framework for evalua-
tion and system design schemes (Almers et al., 2007; Imoize
et al., 2021). Standardized channel models such as the
3GPP (3GPP, 2024a), COST (Liu et al., 2012) or WINNER
1Technical University of Munich...
On Measuring Long-Range Interactions in Graph Neural Networks
Jacob Bamberger * 1 Benjamin Gutteridge * 1 Scott le Roux * 1 Michael Bronstein 1 2 Xiaowen Dong 1
Abstract
Long-range graph tasks — those dependent on
interactions between ‘distant’ nodes — are an
open problem in graph neural network research.
Real-world benchmark tasks, especially the Long
Range Graph Benchmark, have become popular
for validating the long-range capability of pro-
posed architectures. However, this is an empirical
approach that lacks both robustness and theoret-
ical underpinning; a more principled characteri-
zation of the long-range problem is required. To
bridge this gap, we formalize long-range interac-
tions in graph tasks, introduce a range measure
for operators on graphs, and validate it with syn-
thetic experiments. We then leverage our measure
to examine commonly used tasks and architec-
tures, and discuss to what extent they are, in fact,
long-range. We believe our work advances efforts
to define and address the long-range problem on
graphs, and that our range measure will aid evalu-
ation of new datasets and architectures.
1. Introduction
Graphs have emerged as a rich data modality, capable of
modeling pairwise relationships between entities in diverse
applications. Graph neural networks (GNNs) have become
the dominant paradigm for learning from such data, with
message passing neural networks (MPNNs) (Gilmer et al.,
2020) standing out as the most widely used framework.
MPNNs operate by iteratively updating a node’s representa-
tion based on information aggregated from its local neigh-
borhood. This local message-passing mechanism has been
highly effective in a wide range of tasks, such as social net-
works (Monti et al., 2019), computational chemistry (Gilmer
et al., 2017) and recommendation systems (Fan et al., 2019).
*Equal contribution 1University of Oxford 2AITHYRA. Corre-
spondence to: Jacob Bamberger <jacob.bamberger@cs.ox.ac.uk>,
Benjamin Gutteridge <beng@robots.ox.ac.u...
Uni-MuMER: Unified Multi-Task Fine-Tuning of
Vision-Language Model for Handwritten
Mathematical Expression Recognition
Yu Li∗
Jin Jiang∗
Jianhua Zhu
Shuai Peng
Baole Wei
Yuxuan Zhou
Liangcai GaoB
Wangxuan Institute of Computer Technology, Peking University, Beijing, China
liyu@stu.pku.edu.cn
jiangjin@stu.pku.edu.cn
zhujianhuapku@pku.edu.cn
glc@pku.edu.cn
Abstract
Handwritten Mathematical Expression Recognition (HMER) remains a persistent
challenge in Optical Character Recognition (OCR) due to the inherent freedom
of symbol layouts and variability in handwriting styles. Prior methods have faced
performance bottlenecks by proposing isolated architectural modifications, making
them difficult to integrate coherently into a unified framework. Meanwhile, recent
advances in pretrained vision-language models (VLMs) have demonstrated strong
cross-task generalization, offering a promising foundation for developing unified
solutions. In this paper, we introduce Uni-MuMER, which fully fine-tunes a VLM
for the HMER task without modifying its architecture, effectively injecting domain-
specific knowledge into a generalist framework. Our method integrates three data-
driven tasks: Tree-Aware Chain-of-Thought (Tree-CoT) for structured spatial rea-
soning, Error-Driven Learning (EDL) for reducing confusion among visually simi-
lar characters, and Symbol Counting (SC) for improving recognition consistency
in long expressions. Experiments on the CROHME and HME100K datasets show
that Uni-MuMER achieves super state-of-the-art performance, outperforming the
best lightweight specialized model SSAN by 16.31% and the top-performing VLM
Gemini2.5-flash by 24.42% under zero-shot setting. Our datasets, models, and
code are open-sourced at: https://github.com/BFlameSwift/Uni-MuMER
1
Introduction
38.64
37.75
37.45
62.58
62.51
65.30 
72.62
64.26
69.14 
82.05
77.94
79.23
30
40
50
60
70
80
90
100
CROHME 2014
CROHME 2016
CROHME 2019
ExpRate(%)
SSAN(2025)
Qwen2.5VL Zero-Shot
Qwen2.5VL SFT(Vanilla)
Un...
Improving the Effective Receptive Field of Message-Passing Neural Networks
Shahaf E. Finder 1 2 Ron Shapira Weber 1 2 Moshe Eliasof 3 Oren Freifeld 1 2 4 Eran Treister 1 2
Abstract
Message-Passing Neural Networks (MPNNs)
have become a cornerstone for processing and an-
alyzing graph-structured data. However, their ef-
fectiveness is often hindered by phenomena such
as over-squashing, where long-range dependen-
cies or interactions are inadequately captured and
expressed in the MPNN output. This limitation
mirrors the challenges of the Effective Receptive
Field (ERF) in Convolutional Neural Networks
(CNNs), where the theoretical receptive field is
underutilized in practice. In this work, we show
and theoretically explain the limited ERF prob-
lem in MPNNs.
Furthermore, inspired by re-
cent advances in ERF augmentation for CNNs,
we propose an Interleaved Multiscale Message-
Passing Neural Networks (IM-MPNN) architec-
ture to address these problems in MPNNs. Our
method incorporates a hierarchical coarsening
of the graph, enabling message-passing across
multiscale representations and facilitating long-
range interactions without excessive depth or pa-
rameterization. Through extensive evaluations
on benchmarks such as the Long-Range Graph
Benchmark (LRGB), we demonstrate substantial
improvements over baseline MPNNs in captur-
ing long-range dependencies while maintaining
computational efficiency.
1. Introduction
Graph Neural Networks (GNNs) have emerged as power-
ful tools for solving a wide range of problems modeled
as graphs. Applications span diverse domains, including
combinatorial optimization (Cappart et al., 2023), particle
1Department of Computer Science, Ben-Gurion University, Is-
rael 2Data Science Research Center, Ben-Gurion University, Israel
3Department of Applied Mathematics and Theoretical Physics,
University of Cambridge, United Kingdom 4School of Brain Sci-
ences and Cognition, Ben-Gurion University, Israel. Correspon-
dence to: Shahaf E. Finder <finder...
Imagined Autocurricula
Ahmet H. Güzel∗
University College London AI Centre
Matthew T. Jackson
University of Oxford
Jarek L. Liesen
University of Oxford
Tim Rocktäschel
University College London AI Centre
Jakob N. Foerster
University of Oxford
Ilija Bogunovic
University College London AI Centre
Jack Parker-Holder
University College London AI Centre
Abstract
Training agents to act in embodied environments typically requires vast training
data or access to accurate simulation, neither of which exists for many cases in
the real world. Instead, world models are emerging as an alternative–leveraging
offline, passively collected data, they make it possible to generate diverse worlds
for training agents in simulation. In this work, we harness world models to gen-
erate “imagined” environments to train robust agents capable of generalizing to
novel task variations. One of the challenges in doing this is ensuring the agent
trains on useful generated data. We thus propose a novel approach IMAC (Imag-
ined Autocurricula) leveraging Unsupervised Environment Design (UED), which
induces an automatic curriculum over generated worlds. In a series of challeng-
ing, procedurally generated environments, we show it is possible to achieve strong
transfer performance on held-out environments having trained only inside a world
model learned from a narrower dataset. We believe this opens the path to utilizing
larger-scale, foundation world models for generally capable agents.
1
Introduction
Despite significant advancements in AI, we remain far from generally capable agents [1]. In the
2010s, progress towards this goal was driven by breakthroughs in deep reinforcement learning (RL,
[2]) where agents such as AlphaGo [3] showed it was possible to discover new knowledge beyond
human capabilities, like the famous move37. However, scaling RL in simulation to produce a more
general intelligence remains bottlenecked by the lack of sufficiently diverse and real-world simula-
tors [4, 5]. By contrast...
Published as a conference paper at ICLR 2025
CO-MOT: BOOSTING END-TO-END TRANSFORMER-
BASED MULTI-OBJECT TRACKING VIA COOPETITION
LABEL ASSIGNMENT AND SHADOW SETS
Feng Yan⇤, Weixin Luo⇤, Yujie Zhong, Yiyang Gan, Lin Ma†
Meituan Inc., China.
{yanfeng05, luoweixin, zhongyujie, ganyiyang}@meituan.com
forest.linma@gmail.com
ABSTRACT
Existing end-to-end Multi-Object Tracking (e2e-MOT) methods have not sur-
passed non-end-to-end tracking-by-detection methods. One possible reason lies
in the training label assignment strategy that consistently binds the tracked objects
with tracking queries and assigns few newborns to detection queries. Such an as-
signment, with one-to-one bipartite matching, yields an unbalanced training, i.e.,
scarce positive samples for detection queries, especially for an enclosed scenew
with the majority of the newborns at the beginning of videos. As such, e2e-MOT
will incline to generate a tracking terminal without renewal or re-initialization,
compared to other tracking-by-detection methods. To alleviate this problem, we
propose Co-MOT, a simple yet effective method to facilitate e2e-MOT by a novel
coopetition label assignment with a shadow concept. Speciﬁcally, we add tracked
objects to the matching targets for detection queries when performing the label
assignment for training the intermediate decoders. For query initialization, we ex-
pand each query by a set of shadow counterparts with limited disturbance to itself.
With extensive ablation studies, Co-MOT achieves superior performances without
extra costs, e.g., 69.4% HOTA on DanceTrack and 52.8% TETA on BDD100K.
Impressively, Co-MOT only requires 38% FLOPs of MOTRv2 with comparable
performances, resulting in the 1.4⇥faster inference speed. Source code is publicly
available at https://github.com/BingfengYan/CO-MOT.
1
INTRODUCTION
Multi-Object Tracking (MOT) is traditionally tackled by a series of tasks, e.g., object detection (Zhao
et al., 2024; Zou et al., 2023), appearance Re-ID (Zheng et al....