Yancy567/AIS-Bench huggingface dataset:
Conference: ICML 2025

PANDAS: Improving Many-shot Jailbreaking via
Positive Affirmation, Negative Demonstration, and Adaptive Sampling
Avery Ma 1 Yangchen Pan 2 Amir-massoud Farahmand 3
Abstract
Many-shot jailbreaking circumvents the safety
alignment of LLMs by exploiting their ability
to process long input sequences.
To achieve
this, the malicious target prompt is prefixed with
hundreds of fabricated conversational exchanges
between the user and the model. These exchanges
are randomly sampled from a pool of unsafe
question-answer pairs, making it appear as
though the model has already complied with
harmful instructions. In this paper, we present
PANDAS: a hybrid technique that improves
many-shot jailbreaking by modifying these
fabricated dialogues with Positive Affirmations,
Negative Demonstrations, and an optimized
Adaptive Sampling method tailored to the target
prompt’s topic. We also introduce ManyHarm,
a dataset of harmful question–answer pairs, and
demonstrate through extensive experiments that
PANDAS significantly outperforms baseline
methods in long-context scenarios.
Through
attention analysis, we provide insights into how
long-context vulnerabilities are exploited and
show how PANDAS further improves upon many-
shot jailbreaking. Our source code is available at
https://github.com/averyma/pandas.
Warning: This paper contains model behavior
that can be offensive or harmful in nature.
1. Introduction
The growing length of context windows in large lan-
guage models (LLMs) unlocks applications, such as agentic
LLMs (Park et al., 2023), that were previously impractical
or severely limited (Team et al., 2024; Ding et al., 2024; Jin
et al., 2024; Wu et al., 2024; Dong et al., 2024).
1University of Toronto, Vector Institute 2University of Oxford
3Polytechnique Montr´eal, Mila - Quebec AI Institute, University of
Toronto. Correspondence to: Avery Ma <ama@cs.toronto.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 20...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000051. Conference bucket: ICML.
Conference: ICML 2025

Curvature Enhanced Data Augmentation for Regression
Ilya Kaufman 1 Omri Azencot 1
Abstract
Deep learning models with a large number of pa-
rameters, often referred to as over-parameterized
models, have achieved exceptional performance
across various tasks.
Despite concerns about
overfitting, these models frequently generalize
well to unseen data, thanks to effective regular-
ization techniques, with data augmentation be-
ing among the most widely used. While data
augmentation has shown great success in clas-
sification tasks using label-preserving transfor-
mations, its application in regression problems
has received less attention. Recently, a novel
manifold learning approach for generating syn-
thetic data was proposed, utilizing a first-order
approximation of the data manifold. Building on
this foundation, we present a theoretical frame-
work and practical tools for approximating and
sampling general data manifolds. Furthermore,
we introduce the Curvature-Enhanced Manifold
Sampling (CEMS) method for regression tasks.
CEMS leverages a second-order representation
of the data manifold to enable efficient sampling
and reconstruction of new data points. Exten-
sive evaluations across multiple datasets and com-
parisons with state-of-the-art methods demon-
strate that CEMS delivers superior performance
in both in-distribution and out-of-distribution sce-
narios, while introducing only minimal compu-
tational overhead. Code is available at https:
//github.com/azencot-group/CEMS.
1. Introduction
Deep neural networks have demonstrated remarkable per-
formance across a wide range of applications in various
fields (Krizhevsky et al., 2012; Long et al., 2015; Mnih
et al., 2015; Noh et al., 2015; Vinyals et al., 2015; He
1Department of Computer Science, Ben-Gurion University of
the Negev, Beer-Sheva, Israel. Correspondence to: Ilya Kaufman
<ilyakau@post.bgu.ac.il>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyri...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000052. Conference bucket: ICML.
Conference: ICML 2025

CRADLE: Empowering Foundation Agents
Towards General Computer Control
Weihao Tan 1 * Wentao Zhang 1 * Xinrun Xu 2 * Haochong Xia 1 † Ziluo Ding 3 † Boyu Li 3 † Bohan Zhou 4 †
Junpeng Yue 4 † Jiechuan Jiang 4 † Yewen Li 1 † Ruyi An 1 † Molei Qin 1 † Chuqiao Zong 1 † Longtao Zheng 1 †
Yujie Wu 5 † Xiaoqiang Chai 5 † Yifei Bi 3 Tianbao Xie 6 Pengjie Gu 1 Xiyun Li 3 Ceyao Zhang 7 Long Tian 5
Chaojie Wang 5 Xinrun Wang 1 ‡ Börje F. Karlsson 3 ‡ † Bo An 1 5 Shuicheng Yan 8 5 Zongqing Lu 4 3
Everyday
Software
Professional
Software
Digital
Games
Memory
Information 
Gathering
Self-Reflection
Task Inference
Skill Curation
Action
Planning
Figure 1: The CRADLE framework empowers nascent foundation models to perform complex computer tasks via the same uniﬁed
interface humans use, i.e., screenshots as input and keyboard & mouse operations as output.
Abstract
Despite the success in speciﬁc scenarios, exist-
ing foundation agents still struggle to generalize
across various virtual scenarios, mainly due to the
dramatically different encapsulations of environ-
ments with manually designed observation and
action spaces. To handle this issue, we propose
the General Computer Control (GCC) setting to
restrict foundation agents to interact with software
through the most uniﬁed and standardized inter-
face, i.e., using screenshots as input and keyboard
and mouse actions as output. Then we introduce
CRADLE, a modular and ﬂexible LMM-powered
framework, as a preliminary attempt towards
GCC. Enhanced by six key modules: Information
Gathering, Self-Reﬂection, Task Inference, Skill
*Equal contribution 1Nanyang Technological University, Sin-
gapore 2Institute of Software, Chinese Academy of Sciences
3Beijing Academy of Artiﬁcial Intelligence 4Peking University
5Skywork AI 6The University of Hong Kong 7The Chinese Univer-
sity of Hong Kong, Shenzhen 8National University of Singapore.
Correspondence to: Weihao Tan <weihao001@ntu.edu.sg>, Bo
An <boan@ntu.edu.sg>, Shuicheng Yan <yansc@nus.edu.sg>,
Zo...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000054. Conference bucket: ICML.
Conference: ICML 2025

Exploring Large Action Sets with Hyperspherical Embeddings
using von Mises-Fisher Sampling
Walid Bendada 1 2 Guillaume Salha-Galvan 3 Romain Hennequin 1
Th´eo Bontempelli 1 Thomas Bouabc¸a 1 Tristan Cazenave 2
Abstract
This paper introduces von Mises-Fisher explo-
ration (vMF-exp), a scalable method for explor-
ing large action sets in reinforcement learning
problems where hyperspherical embedding vec-
tors represent these actions. vMF-exp involves
initially sampling a state embedding representa-
tion using a von Mises-Fisher distribution, then
exploring this representation’s nearest neighbors,
which scales to virtually unlimited numbers of
candidate actions. We show that, under theoretical
assumptions, vMF-exp asymptotically maintains
the same probability of exploring each action as
Boltzmann Exploration (B-exp), a popular alter-
native that, nonetheless, suffers from scalability
issues as it requires computing softmax values
for each action. Consequently, vMF-exp serves
as a scalable alternative to B-exp for exploring
large action sets with hyperspherical embeddings.
Experiments on simulated data, real-world public
data, and the successful large-scale deployment of
vMF-exp on the recommender system of a global
music streaming service empirically validate the
key properties of the proposed method.
1. Introduction
Exploration is a fundamental component of the reinforce-
ment learning (RL) paradigm (Amin et al., 2021; McFar-
lane, 2018; Sutton & Barto, 2018). It allows RL agents to
gather valuable information about their environment and
identify optimal actions that maximize rewards (Amin et al.,
2021; Chiappa et al., 2023; Dulac-Arnold et al., 2015; Jin
et al., 2020; Ladosz et al., 2022; McFarlane, 2018; Reynolds,
1Deezer Research, Paris, France.
2LAMSADE, Universit´e
Paris Dauphine, PSL, Paris, France. 3SPEIT, Shanghai Jiao Tong
University, Shanghai, China. Correspondence to: Walid Bendada
<bendadaw@gmail.com>.
Proceedings of the 42 nd International Conference on M...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000055. Conference bucket: ICML.
Conference: ICML 2025

Structure Is All You Need: Structural Representation Learning on
Hyper-Relational Knowledge Graphs
Jaejun Lee 1 Joyce Jiyoung Whang 1
Abstract
Hyper-relational knowledge graphs (HKGs) en-
rich knowledge graphs by extending a triplet to
a hyper-relational fact, where a set of qualifiers
adds auxiliary information to a triplet. While
many HKG representation learning methods have
been proposed, they often fail to effectively utilize
the HKG’s structure. This paper demonstrates that
thoroughly leveraging the structure of an HKG
is crucial for reasoning on HKGs, and a purely
structure-based representation learning method
can achieve state-of-the-art performance on var-
ious link prediction tasks. We propose MAYPL,
which learns to initialize representation vectors
based on the structure of an HKG and employs
an attentive neural message passing consisting of
fact-level message computation and entity-centric
and relation-centric aggregations, thereby comput-
ing the representations based solely on the struc-
ture. Due to its structure-driven learning, MAYPL
can conduct inductive inferences on new entities
and relations. MAYPL outperforms 40 knowledge
graph completion methods in 10 datasets, com-
pared with different baseline methods on different
datasets to be tested from diverse perspectives.
1. Introduction
Knowledge graphs (KGs) provide a structured representa-
tion of human knowledge, which eases organizing the re-
lationships between concepts or objects (Ji et al., 2022).
In KGs, information is represented in the form of a
triplet (Toutanova & Chen, 2015; Bordes et al., 2013), e.g.,
(Finding Nemo, set in, Sydney). Since the sim-
ple triplet form oversimplifies information, hyper-relational
knowledge graphs (HKGs) have been recently proposed to
add auxiliary details by adding qualifiers to a triplet (Rosso
1School of Computing, KAIST, Daejeon, South Korea. Corre-
spondence to: Joyce Jiyoung Whang <jjwhang@kaist.ac.kr>.
Proceedings of the 42 nd International Conference o...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000059. Conference bucket: ICML.
Conference: ICML 2025

Lightweight Dataset Pruning without Full Training
via Example Difficulty and Prediction Uncertainty
Yeseul Cho * 1 Baekrok Shin * 1 Changmin Kang 1 Chulhee Yun 1
Abstract
Recent advances in deep learning rely heavily
on massive datasets, leading to substantial stor-
age and training costs. Dataset pruning aims to
alleviate this demand by discarding redundant ex-
amples. However, many existing methods require
training a model with a full dataset over a large
number of epochs before being able to prune the
dataset, which ironically makes the pruning pro-
cess more expensive than just training the model
on the entire dataset. To overcome this limitation,
we introduce the Difficulty and Uncertainty-
Aware Lightweight (DUAL) score, which aims
to identify important samples from the early train-
ing stage by considering both example difficulty
and prediction uncertainty. To address a catas-
trophic accuracy drop at an extreme pruning ra-
tio, we further propose a pruning ratio-adaptive
sampling using Beta distribution. Experiments
on various datasets and learning scenarios such
as image classification with label noise and im-
age corruption, and model architecture general-
ization demonstrate the superiority of our method
over previous state-of-the-art (SOTA) approaches.
Specifically, on ImageNet-1k, our method reduces
the time cost for pruning to 66% compared to
previous methods while achieving a SOTA 60%
test accuracy at a 90% pruning ratio. On CIFAR
datasets, the time cost is reduced to just 15% while
maintaining SOTA performance. Implementation
is available at github/dual-pruning.
1. Introduction
Advancements in deep learning have been significantly
driven by large-scale datasets. However, recent studies have
*Equal contribution
1Kim Jaechul Graduate School of AI,
KAIST, Seoul, South Korea. Correspondence to: Chulhee Yun
<chulhee.yun@kaist.ac.kr>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000060. Conference bucket: ICML.
Conference: ICML 2025

(How) Do Language Models Track State?
Belinda Z. Li 1 Zifan Carl Guo 1 Jacob Andreas 1
Abstract
Transformer language models (LMs) exhibit
behaviors—from storytelling to code generation—
that seem to require tracking the unobserved state
of an evolving world. How do they do this? We
study state tracking in LMs trained or fine-tuned
to compose permutations (i.e., to compute the or-
der of a set of objects after a sequence of swaps).
Despite the simple algebraic structure of this prob-
lem, many other tasks (e.g., simulation of finite
automata and evaluation of boolean expressions)
can be reduced to permutation composition, mak-
ing it a natural model for state tracking in gen-
eral. We show that LMs consistently learn one of
two state tracking mechanisms for this task. The
first closely resembles the “associative scan” con-
struction used in recent theoretical work by Liu
et al. (2023) and Merrill et al. (2024). The second
uses an easy-to-compute feature (permutation par-
ity) to partially prune the space of outputs, and
then refines this with an associative scan. LMs
that learn the former algorithm tend to general-
ize better and converge faster, and we show how
to steer LMs toward one or the other with inter-
mediate training tasks that encourage or suppress
the heuristics. Our results demonstrate that trans-
former LMs, whether pre-trained or fine-tuned,
can learn to implement efficient and interpretable
state-tracking mechanisms, and the emergence
of these mechanisms can be predicted and con-
trolled. Code and data are available at https:
//github.com/belindal/state-tracking.
1. Introduction
Language models (LMs) are trained to model the surface
form of text. A growing body of work suggests that model
internals contain a latent, decodable state of the world—e.g.,
situations described by language and results of program
1MIT EECS and CSAIL. Correspondence to: Belinda Z. Li
<bzl@mit.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouve...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000061. Conference bucket: ICML.
Conference: ICML 2025

Physics-Informed Generative Modeling of Wireless Channels
Benedikt B¨ock 1 Andreas Oeldemann 2 Timo Mayer 2 Francesco Rossetto 2 Wolfgang Utschick 1
Abstract
Learning the site-specific distribution of the wire-
less channel within a particular environment of
interest is essential to exploit the full potential of
machine learning (ML) for wireless communica-
tions and radar applications. Generative model-
ing offers a promising framework to address this
problem. However, existing approaches pose un-
resolved challenges, including the need for high-
quality training data, limited generalizability, and
a lack of physical interpretability. To address
these issues, we combine the physics-related com-
pressibility of wireless channels with generative
modeling, in particular, sparse Bayesian genera-
tive modeling (SBGM), to learn the distribution of
the underlying physical channel parameters. By
leveraging the sparsity-inducing characteristics of
SBGM, our methods can learn from compressed
observations received by an access point (AP) dur-
ing default online operation. Moreover, they are
physically interpretable and generalize over sys-
tem configurations without requiring retraining.
1. Introduction
The accurate modeling of wireless channels is critical for
system design, network optimization, and performance eval-
uation in wireless communications and radar (Wang et al.,
2018; 2020; Yin & Cheng, 2016). There are two approaches
to wireless channel modeling: physically and analytically
oriented (Almers et al., 2007). The former integrates the
underlying physics in the description of wireless channels
and focuses on their accuracy and realism. In contrast, the
latter captures channel statistics, ignoring the underlying
physics, and provides an easy-to-use framework for evalua-
tion and system design schemes (Almers et al., 2007; Imoize
et al., 2021). Standardized channel models such as the
3GPP (3GPP, 2024a), COST (Liu et al., 2012) or WINNER
1Technical University of Munich...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000062. Conference bucket: ICML.
Conference: ICML 2025

On Measuring Long-Range Interactions in Graph Neural Networks
Jacob Bamberger * 1 Benjamin Gutteridge * 1 Scott le Roux * 1 Michael Bronstein 1 2 Xiaowen Dong 1
Abstract
Long-range graph tasks — those dependent on
interactions between ‘distant’ nodes — are an
open problem in graph neural network research.
Real-world benchmark tasks, especially the Long
Range Graph Benchmark, have become popular
for validating the long-range capability of pro-
posed architectures. However, this is an empirical
approach that lacks both robustness and theoret-
ical underpinning; a more principled characteri-
zation of the long-range problem is required. To
bridge this gap, we formalize long-range interac-
tions in graph tasks, introduce a range measure
for operators on graphs, and validate it with syn-
thetic experiments. We then leverage our measure
to examine commonly used tasks and architec-
tures, and discuss to what extent they are, in fact,
long-range. We believe our work advances efforts
to define and address the long-range problem on
graphs, and that our range measure will aid evalu-
ation of new datasets and architectures.
1. Introduction
Graphs have emerged as a rich data modality, capable of
modeling pairwise relationships between entities in diverse
applications. Graph neural networks (GNNs) have become
the dominant paradigm for learning from such data, with
message passing neural networks (MPNNs) (Gilmer et al.,
2020) standing out as the most widely used framework.
MPNNs operate by iteratively updating a node’s representa-
tion based on information aggregated from its local neigh-
borhood. This local message-passing mechanism has been
highly effective in a wide range of tasks, such as social net-
works (Monti et al., 2019), computational chemistry (Gilmer
et al., 2017) and recommendation systems (Fan et al., 2019).
*Equal contribution 1University of Oxford 2AITHYRA. Corre-
spondence to: Jacob Bamberger <jacob.bamberger@cs.ox.ac.uk>,
Benjamin Gutteridge <beng@robots.ox.ac.u...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000063. Conference bucket: ICML.
Conference: ICML 2025

Improving the Effective Receptive Field of Message-Passing Neural Networks
Shahaf E. Finder 1 2 Ron Shapira Weber 1 2 Moshe Eliasof 3 Oren Freifeld 1 2 4 Eran Treister 1 2
Abstract
Message-Passing Neural Networks (MPNNs)
have become a cornerstone for processing and an-
alyzing graph-structured data. However, their ef-
fectiveness is often hindered by phenomena such
as over-squashing, where long-range dependen-
cies or interactions are inadequately captured and
expressed in the MPNN output. This limitation
mirrors the challenges of the Effective Receptive
Field (ERF) in Convolutional Neural Networks
(CNNs), where the theoretical receptive field is
underutilized in practice. In this work, we show
and theoretically explain the limited ERF prob-
lem in MPNNs.
Furthermore, inspired by re-
cent advances in ERF augmentation for CNNs,
we propose an Interleaved Multiscale Message-
Passing Neural Networks (IM-MPNN) architec-
ture to address these problems in MPNNs. Our
method incorporates a hierarchical coarsening
of the graph, enabling message-passing across
multiscale representations and facilitating long-
range interactions without excessive depth or pa-
rameterization. Through extensive evaluations
on benchmarks such as the Long-Range Graph
Benchmark (LRGB), we demonstrate substantial
improvements over baseline MPNNs in captur-
ing long-range dependencies while maintaining
computational efficiency.
1. Introduction
Graph Neural Networks (GNNs) have emerged as power-
ful tools for solving a wide range of problems modeled
as graphs. Applications span diverse domains, including
combinatorial optimization (Cappart et al., 2023), particle
1Department of Computer Science, Ben-Gurion University, Is-
rael 2Data Science Research Center, Ben-Gurion University, Israel
3Department of Applied Mathematics and Theoretical Physics,
University of Cambridge, United Kingdom 4School of Brain Sci-
ences and Cognition, Ben-Gurion University, Israel. Correspon-
dence to: Shahaf E. Finder <finder...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000065. Conference bucket: ICML.
Conference: ICML 2025

Gradient-based Explanations for Deep Learning Survival Models
Sophie Hanna Langbein * 1 2 Niklas Koenen * 1 2 Marvin N. Wright 1 2 3
Abstract
Deep learning survival models often outperform
classical methods in time-to-event predictions,
particularly in personalized medicine, but their
“black box” nature hinders broader adoption. We
propose a framework for gradient-based explana-
tion methods tailored to survival neural networks,
extending their use beyond regression and clas-
sification. We analyze the implications of their
theoretical assumptions for time-dependent ex-
planations in the survival setting and propose ef-
fective visualizations incorporating the temporal
dimension. Experiments on synthetic data show
that gradient-based methods capture the magni-
tude and direction of local and global feature
effects, including time dependencies. We intro-
duce GradSHAP(t), a gradient-based counterpart
to SurvSHAP(t), which outperforms SurvSHAP(t)
and SurvLIME in a computational speed vs. accu-
racy trade-off. Finally, we apply these methods to
medical data with multi-modal inputs, revealing
relevant tabular features and visual patterns, as
well as their temporal dynamics.
1. Introduction
As medical databases expand to include patients’ detailed
medical history and genetic information, healthcare is shift-
ing from population-based models and traditional statistical
approaches targeting the “average” patient to more complex
personalized medicine, which tailors diagnoses to individ-
ual patient characteristics. Survival analysis is fundamental
to medical data analysis, modeling time-to-event outcomes
while accounting for censoring, enabling personalized risk
predictions, and assessing treatment effects to advance clini-
*Equal contribution 1Leibniz Institute for Prevention Research
and Epidemiology – BIPS, Bremen, Germany 2Faculty of Math-
ematics and Computer Science, University of Bremen, Germany
3Department of Public Health, University of Copenhagen, Den-
mark. Corre...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000070. Conference bucket: ICML.