AIS-Bench/AIS-Bench huggingface dataset:
Published as a conference paper at ICLR 2025
NEURAL INTERACTIVE PROOFS
Lewis Hammond∗
lewis.hammond@cs.ox.ac.uk
Sam Adam-Day∗
sam.adam-day@cs.ox.ac.uk
Department of Computer Science, University of Oxford, Oxford, United Kingdom
ABSTRACT
We consider the problem of how a trusted, but computationally bounded agent
(a ‘verifier’) can learn to interact with one or more powerful but untrusted agents
(‘provers’) in order to solve a given task. More specifically, we study the case in
which agents are represented using neural networks and refer to solutions of this
problem as neural interactive proofs. First we introduce a unifying framework based
on prover-verifier games (Anil et al., 2021), which generalises previously proposed
interaction protocols. We then describe several new protocols for generating neural
interactive proofs, and provide a theoretical comparison of both new and existing
approaches. Finally, we support this theory with experiments in two domains: a
toy graph isomorphism problem that illustrates the key ideas, and a code validation
task using large language models. In so doing, we aim to create a foundation for
future work on neural interactive proofs and their application in building safer AI
systems.
1
INTRODUCTION
Recent years have witnessed the proliferation of large machine learning (ML) systems (Villalobos
et al., 2022), useful for solving an increasingly wide range of tasks. Often, however, it can be difficult
to trust the output of these systems, raising concerns about their safety and limiting their applicability
in high-stakes situations (Amodei et al., 2016; Bengio et al., 2023; Hendrycks et al., 2023). At
the same time, traditional approaches in verification do not scale to today’s most powerful systems
(Seshia et al., 2022). There is thus a pressing need to identify new angles via which to gain such
assurances.
qƧǩƽΞƧǩ
UǩǙͭƧǩ
Figure 1: On receiving input x from distribution P the
agents exchange messages before the verifier decides on
an out...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000040. Conference bucket: ICLR.
Published as a conference paper at ICLR 2025
NEURAL INTERACTIVE PROOFS
Lewis Hammond∗
lewis.hammond@cs.ox.ac.uk
Sam Adam-Day∗
sam.adam-day@cs.ox.ac.uk
Department of Computer Science, University of Oxford, Oxford, United Kingdom
ABSTRACT
We consider the problem of how a trusted, but computationally bounded agent
(a ‘verifier’) can learn to interact with one or more powerful but untrusted agents
(‘provers’) in order to solve a given task. More specifically, we study the case in
which agents are represented using neural networks and refer to solutions of this
problem as neural interactive proofs. First we introduce a unifying framework based
on prover-verifier games (Anil et al., 2021), which generalises previously proposed
interaction protocols. We then describe several new protocols for generating neural
interactive proofs, and provide a theoretical comparison of both new and existing
approaches. Finally, we support this theory with experiments in two domains: a
toy graph isomorphism problem that illustrates the key ideas, and a code validation
task using large language models. In so doing, we aim to create a foundation for
future work on neural interactive proofs and their application in building safer AI
systems.
1
INTRODUCTION
Recent years have witnessed the proliferation of large machine learning (ML) systems (Villalobos
et al., 2022), useful for solving an increasingly wide range of tasks. Often, however, it can be difficult
to trust the output of these systems, raising concerns about their safety and limiting their applicability
in high-stakes situations (Amodei et al., 2016; Bengio et al., 2023; Hendrycks et al., 2023). At
the same time, traditional approaches in verification do not scale to today’s most powerful systems
(Seshia et al., 2022). There is thus a pressing need to identify new angles via which to gain such
assurances.
qƧǩƽΞƧǩ
UǩǙͭƧǩ
Figure 1: On receiving input x from distribution P the
agents exchange messages before the verifier decides on
an out...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000045. Conference bucket: ICLR.
Published as a conference paper at ICLR 2025
DISCRETE COPULA DIFFUSION
Anji Liu1,2, Oliver Broadrick1, Mathias Niepert2, Guy Van den Broeck1
1Department of Computer Science, University of California, Los Angeles, USA
2Institute for Artificial Intelligence, University of Stuttgart, Germany
{liuanji,obroadrick,guyvdb}@cs.ucla.edu
mathias.niepert@ki.uni-stuttgart.de
ABSTRACT
Discrete diffusion models have recently shown significant progress in modeling
complex data, such as natural languages and DNA sequences. However, unlike
diffusion models for continuous data, which can generate high-quality samples in
just a few denoising steps, modern discrete diffusion models still require hundreds
or even thousands of denoising steps to perform well. In this paper, we identify
a fundamental limitation that prevents discrete diffusion models from achieving
strong performance with fewer steps – they fail to capture dependencies between
output variables at each denoising step. To address this issue, we provide a for-
mal explanation and introduce a general approach to supplement the missing de-
pendency information by incorporating another deep generative model, termed
the copula model. Our method does not require fine-tuning either the diffusion
model or the copula model, yet it enables high-quality sample generation with
significantly fewer denoising steps. When we apply this approach to autoregres-
sive copula models, the combined model outperforms both models individually
in unconditional and conditional text generation. Specifically, the hybrid model
achieves better (un)conditional text generation using 8 to 32 times fewer denois-
ing steps than the diffusion model alone. In addition to presenting an effective
discrete diffusion generation algorithm, this paper emphasizes the importance of
modeling inter-variable dependencies in discrete diffusion.1
1
INTRODUCTION
Discrete diffusion models have recently achieved significant progress in modeling complex data such
as natural lan...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000050. Conference bucket: ICLR.
Published as a conference paper at ICLR 2025
MORAL ALIGNMENT FOR LLM AGENTS
Elizaveta Tennant
University College London
University of Bologna
l.karmannaya.16@ucl.ac.uk
Stephen Hailes
University College London
s.hailes@ucl.ac.uk
Mirco Musolesi
University College London
University of Bologna
m.musolesi@ucl.ac.uk
ABSTRACT
Decision-making agents based on pre-trained Large Language Models (LLMs)
are increasingly being deployed across various domains of human activity. While
their applications are currently rather specialized, several research efforts are un-
derway to develop more generalist agents. As LLM-based systems become more
agentic, their influence on human activity will grow and their transparency will de-
crease. Consequently, developing effective methods for aligning them to human
values is vital.
The prevailing practice in alignment often relies on human preference data (e.g., in
RLHF or DPO), in which values are implicit, opaque and are essentially deduced
from relative preferences over different model outputs. In this work, instead of
relying on human feedback, we introduce the design of reward functions that ex-
plicitly and transparently encode core human values for Reinforcement Learning-
based fine-tuning of foundation agent models. Specifically, we use intrinsic re-
wards for the moral alignment of LLM agents.
We evaluate our approach using the traditional philosophical frameworks of Deon-
tological Ethics and Utilitarianism, quantifying moral rewards for agents in terms
of actions and consequences on the Iterated Prisoner’s Dilemma (IPD) environ-
ment. We also show how moral fine-tuning can be deployed to enable an agent to
unlearn a previously developed selfish strategy. Finally, we find that certain moral
strategies learned on the IPD game generalize to several other matrix game en-
vironments. In summary, we demonstrate that fine-tuning with intrinsic rewards
is a promising general solution for aligning LLM agents to human values, and
it might repr...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000056. Conference bucket: ICLR.
Published as a conference paper at ICLR 2025
CO-MOT: BOOSTING END-TO-END TRANSFORMER-
BASED MULTI-OBJECT TRACKING VIA COOPETITION
LABEL ASSIGNMENT AND SHADOW SETS
Feng Yan⇤, Weixin Luo⇤, Yujie Zhong, Yiyang Gan, Lin Ma†
Meituan Inc., China.
{yanfeng05, luoweixin, zhongyujie, ganyiyang}@meituan.com
forest.linma@gmail.com
ABSTRACT
Existing end-to-end Multi-Object Tracking (e2e-MOT) methods have not sur-
passed non-end-to-end tracking-by-detection methods. One possible reason lies
in the training label assignment strategy that consistently binds the tracked objects
with tracking queries and assigns few newborns to detection queries. Such an as-
signment, with one-to-one bipartite matching, yields an unbalanced training, i.e.,
scarce positive samples for detection queries, especially for an enclosed scenew
with the majority of the newborns at the beginning of videos. As such, e2e-MOT
will incline to generate a tracking terminal without renewal or re-initialization,
compared to other tracking-by-detection methods. To alleviate this problem, we
propose Co-MOT, a simple yet effective method to facilitate e2e-MOT by a novel
coopetition label assignment with a shadow concept. Speciﬁcally, we add tracked
objects to the matching targets for detection queries when performing the label
assignment for training the intermediate decoders. For query initialization, we ex-
pand each query by a set of shadow counterparts with limited disturbance to itself.
With extensive ablation studies, Co-MOT achieves superior performances without
extra costs, e.g., 69.4% HOTA on DanceTrack and 52.8% TETA on BDD100K.
Impressively, Co-MOT only requires 38% FLOPs of MOTRv2 with comparable
performances, resulting in the 1.4⇥faster inference speed. Source code is publicly
available at https://github.com/BingfengYan/CO-MOT.
1
INTRODUCTION
Multi-Object Tracking (MOT) is traditionally tackled by a series of tasks, e.g., object detection (Zhao
et al., 2024; Zou et al., 2023), appearance Re-ID (Zheng et al....

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000067. Conference bucket: ICLR.
Published as a conference paper at ICLR 2025
MORAL ALIGNMENT FOR LLM AGENTS
Elizaveta Tennant
University College London
University of Bologna
l.karmannaya.16@ucl.ac.uk
Stephen Hailes
University College London
s.hailes@ucl.ac.uk
Mirco Musolesi
University College London
University of Bologna
m.musolesi@ucl.ac.uk
ABSTRACT
Decision-making agents based on pre-trained Large Language Models (LLMs)
are increasingly being deployed across various domains of human activity. While
their applications are currently rather specialized, several research efforts are un-
derway to develop more generalist agents. As LLM-based systems become more
agentic, their influence on human activity will grow and their transparency will de-
crease. Consequently, developing effective methods for aligning them to human
values is vital.
The prevailing practice in alignment often relies on human preference data (e.g., in
RLHF or DPO), in which values are implicit, opaque and are essentially deduced
from relative preferences over different model outputs. In this work, instead of
relying on human feedback, we introduce the design of reward functions that ex-
plicitly and transparently encode core human values for Reinforcement Learning-
based fine-tuning of foundation agent models. Specifically, we use intrinsic re-
wards for the moral alignment of LLM agents.
We evaluate our approach using the traditional philosophical frameworks of Deon-
tological Ethics and Utilitarianism, quantifying moral rewards for agents in terms
of actions and consequences on the Iterated Prisoner’s Dilemma (IPD) environ-
ment. We also show how moral fine-tuning can be deployed to enable an agent to
unlearn a previously developed selfish strategy. Finally, we find that certain moral
strategies learned on the IPD game generalize to several other matrix game en-
vironments. In summary, we demonstrate that fine-tuning with intrinsic rewards
is a promising general solution for aligning LLM agents to human values, and
it might repr...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000079. Conference bucket: ICLR.
Published as a conference paper at ICLR 2025
LLMOPT: LEARNING TO DEFINE AND SOLVE GEN-
ERAL OPTIMIZATION PROBLEMS FROM SCRATCH
Caigao Jiang♥∗,
Xiang Shu♣∗,
Hong Qian♣†,
Xingyu Lu♥†,
Jun Zhou♥,
Aimin Zhou♣,
Yang Yu♦
♣East China Normal University, China
♥Ant Group, China
♦Nanjing University, China
shux@stu.ecnu.edu.cn,
{hqian,amzhou}@cs.ecnu.edu.cn,
{caigao.jcg,sing.lxy,jun.zhoujun}@antgroup.com,
yuy@nju.edu.cn
ABSTRACT
Optimization problems are prevalent across various scenarios. Formulating and
then solving optimization problems described by natural language often requires
highly specialized human expertise, which could block the widespread applica-
tion of optimization-based decision making. To automate problem formulation
and solving, leveraging large language models (LLMs) has emerged as a potential
way. However, this kind of approach suffers from the issue of optimization gener-
alization. Namely, the accuracy of most current LLM-based methods and the gen-
erality of optimization problem types that they can model are still limited. In this
paper, we propose a unified learning-based framework called LLMOPT to boost
optimization generalization. Starting from the natural language descriptions of op-
timization problems and a pre-trained LLM, LLMOPT constructs the introduced
five-element formulation as a universal model for learning to define diverse op-
timization problem types. Then, LLMOPT employs the multi-instruction tuning
to enhance both problem formalization and solver code generation accuracy and
generality. After that, to prevent hallucinations in LLMs, such as sacrificing solv-
ing accuracy to avoid execution errors, the model alignment and self-correction
mechanism are adopted in LLMOPT. We evaluate the optimization generalization
ability of LLMOPT and compared methods across six real-world datasets cov-
ering roughly 20 fields such as health, environment, energy and manufacturing,
etc. Extensive experiment results show that LLMOPT is able to model vario...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000086. Conference bucket: ICLR.