Yancy567/AIS-Bench huggingface dataset:
Conference: ICML 2025

LEVIS: Large Exact Verifiable Input Spaces for Neural Networks
Mohamad Chehade * 1 2 Wenting Li * 2 Brian W. Bell 2 Russell Bent 2 Saif R. Kazi 2 Hao Zhu 1
Abstract
The robustness of neural networks is crucial in
safety-critical applications, where identifying a re-
liable input space is essential for effective model
selection, robustness evaluation, and the devel-
opment of reliable control strategies. Most ex-
isting robustness verification methods assess the
worst-case output under the assumption that the
input space is known. However, precisely identi-
fying a verifiable input space C, where no adver-
sarial examples exist, is challenging due to the
possible high dimensionality, discontinuity, and
non-convex nature of the input space. To address
this challenge, we propose a novel framework,
LEVIS, consisting of LEVIS-α and LEVIS-β.
LEVIS-α identifies a single, large verifiable
ball that intersects at least two boundaries of a
bounded region C. In contrast, LEVIS-β sys-
tematically captures the entirety of the verifiable
space by integrating multiple verifiable balls. Our
contributions are fourfold: (1) We introduce a ver-
ification framework, LEVIS, incorporating two
optimization techniques for computing nearest
and directional adversarial points based on mixed-
integer programming (MIP). (2) To enhance scal-
ability, we integrate complementarity-constrained
(CC) optimization with a reduced MIP formula-
tion, achieving up to a 6-fold reduction in runtime
while approximating the verifiable region in a
principled manner. (3) We provide a theoretical
analysis characterizing the properties of the veri-
fiable balls obtained through LEVIS-α. (4) We
validate our approach across diverse applications,
including electrical power flow regression and
*Equal contribution
1Chandra Department of Electrical
and Computer Engineering,
The University of Texas at
Austin, Austin, TX, USA
2Los Alamos National Labora-
tory,
Los Alamos,
NM, USA. Correspondence to:
Mo-
hamad Chehade <c...
Conference: ICML 2025

Enhancing Performance of Explainable AI Models
with Constrained Concept Refinement
Geyu Liang 1 Senne Michielssen 2 Salar Fattahi 1
Abstract
The trade-off between accuracy and interpretabil-
ity has long been a challenge in machine learning
(ML). This tension is particularly significant for
emerging interpretable-by-design methods, which
aim to redesign ML algorithms for trustworthy in-
terpretability but often sacrifice accuracy in the
process. In this paper, we address this gap by
investigating the impact of deviations in concept
representations—an essential component of in-
terpretable models—on prediction performance
and propose a novel framework to mitigate these
effects. The framework builds on the principle
of optimizing concept embeddings under con-
straints that preserve interpretability. Using a gen-
erative model as a test-bed, we rigorously prove
that our algorithm achieves zero loss while pro-
gressively enhancing the interpretability of the
resulting model. Additionally, we evaluate the
practical performance of our proposed framework
in generating explainable predictions for image
classification tasks across various benchmarks.
Compared to existing explainable methods, our
approach not only improves prediction accuracy
while preserving model interpretability across var-
ious large-scale benchmarks but also achieves this
with significantly lower computational cost.
1. Introduction
ML algorithms are often caught in a dilemma between in-
terpretability and performance. Models such as linear re-
gression (Hastie, 2009) and decision trees (Quinlan, 1986)
provide straightforward interpretability through parameter
weights and rule-based predictions. However, they fre-
1Department of Industrial and Operations Engineering, Univer-
sity of Michigan, Ann Arbor, MI, US. 2Department of Computer
Science, Princeton University, Princeton, NJ, US. Correspondence
to: Salar Fattahi <fattahi@umich.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning...
Conference: ICML 2025

Does Low Rank Adaptation Lead to Lower Robustness
against Training-Time Attacks?
Zi Liang 1 Haibo Hu 1 Qingqing Ye 1 Yaxin Xiao 1 Ronghua Li 1
Abstract
Low rank adaptation (LoRA) has emerged as
a prominent technique for ﬁne-tuning large lan-
guage models (LLMs) thanks to its superb efﬁ-
ciency gains over previous methods. While ex-
tensive studies have examined the performance
and structural properties of LoRA, its behavior
upon training-time attacks remain underexplored,
posing signiﬁcant security risks. In this paper, we
theoretically investigate the security implications
of LoRA’s low-rank structure during ﬁne-tuning,
in the context of its robustness against data poi-
soning and backdoor attacks. We propose an ana-
lytical framework that models LoRA’s training dy-
namics, employs the neural tangent kernel to sim-
plify the analysis of the training process, and ap-
plies information theory to establish connections
between LoRA’s low rank structure and its vul-
nerability against training-time attacks. Our anal-
ysis indicates that LoRA exhibits better robust-
ness to backdoor attacks than full ﬁne-tuning,
while becomes more vulnerable to untargeted
data poisoning due to its over-simpliﬁed infor-
mation geometry. Extensive experimental evalua-
tions have corroborated our theoretical ﬁndings.
1. Introduction
With the rapid growth in the parameter size of large language
models (LLMs), parameter-efﬁcient ﬁne-tuning (PEFT) (Xu
et al., 2023; Han et al., 2024) has gained increasing attention
in both research and industry communities. Among various
PEFT strategies, low-rank adaptation (LoRA) (Hu et al.,
2021) has emerged as the de facto standard for ﬁne-tuning
LLMs thanks to its computational efﬁciency and minimal
performance degradation.
1The Hong Kong Polytechnic University, Hong Kong, China.
Correspondence to: Haibo Hu <haibo.hu@polyu.edu.hk>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by the...
Conference: NeurIPS 2025

OmniFC: Rethinking Federated Clustering via
Lossless and Secure Distance Reconstruction
Jie Yan
Jing Liu
Zhong-Yuan Zhang∗
Central University of Finance and Economics
jieyan@email.cufe.edu.cn, liu_jing0623@163.com, zhyuanzh@gmail.com
Abstract
Federated clustering (FC) aims to discover global cluster structures across decentral-
ized clients without sharing raw data, making privacy preservation a fundamental
requirement. There are two critical challenges: (1) privacy leakage during collab-
oration, and (2) robustness degradation due to aggregation of proxy information
from non-independent and identically distributed (Non-IID) local data, leading to
inaccurate or inconsistent global clustering. Existing solutions typically rely on
model-specific local proxies, which are sensitive to data heterogeneity and inherit
inductive biases from their centralized counterparts, thus limiting robustness and
generality. We propose Omni Federated Clustering (OmniFC), a unified and model-
agnostic framework. Leveraging Lagrange coded computing, our method enables
clients to share only encoded data, allowing exact reconstruction of the global
distance matrix—a fundamental representation of sample relationships—without
leaking private information, even under client collusion. This construction is nat-
urally resilient to Non-IID data distributions. This approach decouples FC from
model-specific proxies, providing a unified extension mechanism applicable to
diverse centralized clustering methods. Theoretical analysis confirms both re-
construction fidelity and privacy guarantees, while comprehensive experiments
demonstrate OmniFC’s superior robustness, effectiveness, and generality across
various benchmarks compared to state-of-the-art methods. Code will be released.
1
Introduction
Traditional clustering methods presuppose centralized access to the entire dataset, enabling the
construction of global structures such as cluster centroids or kernel matrices. However, in federated
settings ...
Conference: NeurIPS 2025

CSPCL: Category Semantic Prior Contrastive
Learning for Deformable DETR-Based Prohibited
Item Detectors
Mingyuan Li1
Tong Jia1,2∗Hao Wang1
Bowen Ma1
Hui Lu1
Shiyi Guo1
Da Cai1
Dongyue Chen1
1 College of Information Science and Engineering, Northeastern University, China
2 State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University
542027743@qq.com, jiatong@ise.neu.edu.cn, ddsywh@yeah.net,
2010285@stu.neu.edu.cn, 2603813543@qq.com, guoshiyi@ise.neu.edu.cn,
2210329@stu.neu.edu.cn, chendongyue@ise.neu.edu.cn
Abstract
Prohibited item detection based on X-ray images is one of the most effective secu-
rity inspection methods. However, the foreground-background feature coupling
caused by the overlapping phenomenon specific to X-ray images makes general
detectors designed for natural images perform poorly. To address this issue, we
propose a Category Semantic Prior Contrastive Learning (CSPCL) mechanism,
which aligns the class prototypes perceived by the classifier with the content queries
to correct and supplement the missing semantic information responsible for classifi-
cation, thereby enhancing the model sensitivity to foreground features. To achieve
this alignment, we design a specific contrastive loss, CSP loss, which comprises
the Intra-Class Truncated Attraction (ITA) loss and the Inter-Class Adaptive Repul-
sion (IAR) loss, and outperforms classic contrastive losses. Specifically, the ITA
loss leverages class prototypes to attract intra-class content queries and preserves
essential intra-class diversity via a gradient truncation function. The IAR loss
employs class prototypes to adaptively repel inter-class content queries, with the
repulsion strength scaled by prototype-prototype similarity, thereby improving
inter-class discriminability, especially among similar categories. CSPCL is general
and can be easily integrated into Deformable DETR-based models. Extensive
experiments on the PIXray, OPIXray, PIDray, and CLCXray datasets d...
Conference: ICML 2025

On the Robustness of Reward Models for Language Model Alignment
Jiwoo Hong 1 Noah Lee 1 Eunki Kim 1 Guijin Son 2 Woojin Chung 1
Aman Gupta 3 Shao Tang 3 James Thorne 1
Abstract
The Bradley-Terry (BT) model is widely prac-
ticed in reward modeling for reinforcement learn-
ing with human feedback (RLHF). Despite its
effectiveness, reward models (RMs) trained with
BT model loss are prone to over-optimization,
losing generalizability to unseen input distribu-
tions.
In this paper, we study the cause of
over-optimization in RM training and its down-
stream effects on the RLHF procedure, accen-
tuating the importance of distributional robust-
ness of RMs in unseen data. First, we show that
the excessive dispersion of hidden state norms
is the main source of over-optimization. Then,
we propose batch-wise sum-to-zero regulariza-
tion (BSR) to enforce zero-centered reward sum
per batch, constraining the rewards with extreme
magnitudes. We assess the impact of BSR in
improving robustness in RMs through four sce-
narios of over-optimization, where BSR consis-
tently manifests better robustness. Subsequently,
we compare the plain BT model and BSR on
RLHF training and empirically show that robust
RMs better align the policy to the gold preference
model. Finally, we apply BSR to high-quality
data and models, which surpasses state-of-the-
art RMs in the 8B scale by adding more than
5% in complex preference prediction tasks. By
conducting RLOO training with 8B RMs, Al-
pacaEval 2.0 reduces generation length by 40%
while adding a 7% increase in win rate, further
highlighting that robustness in RMs induces ro-
bustness in RLHF training. We release the code,
data, and models: https://github.com/
LinkedIn-XFACT/RM-Robustness.
1KAIST AI 2OneLineAI 3LinkedIn Corporation. Correspon-
dence to: Jiwoo Hong <jiwoo hong@kaist.ac.kr>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by the author(s).
1. Introduction
Reward ...
Conference: ICML 2025

CHAMELEON: A Flexible Data-mixing Framework
for Language Model Pretraining and Finetuning
Wanyun Xie 1 Francesco Tonin 1 Volkan Cevher 1
Abstract
Training data mixtures greatly impact the gen-
eralization performance of large language mod-
els. Existing domain reweighting methods often
rely on costly weight computations and require
retraining when new data is introduced. To this
end, we introduce a flexible and efficient data
mixing framework, CHAMELEON, that employs
leverage scores to quantify domain importance
within a learned embedding space. We first con-
struct a domain affinity matrix over domain em-
beddings. The induced leverage scores determine
a mixture that upweights domains sharing com-
mon representations in embedding space. This
formulation allows direct transfer to new data by
computing the new domain embeddings. In ex-
periments, we demonstrate improvements over
three key scenarios: (i) our computed weights im-
prove performance on pretraining domains with
a fraction of the compute of existing methods;
(ii) CHAMELEON can adapt to data changes with-
out proxy retraining, boosting few-shot reason-
ing accuracies when transferred to new data; (iii)
our method enables efficient domain reweight-
ing in finetuning, consistently improving test per-
plexity on all finetuning domains over uniform
mixture. Our code is available at https://
github.com/LIONS-EPFL/Chameleon.
1. Introduction
Pretraining large language models (LLMs) relies heavily
on vast and diverse datasets, encompassing sources such as
academic papers, books, and code repositories (Brown et al.,
2020; Dubey et al., 2024). The composition of these datasets
1Laboratory for Information and Inference Systems, École
Polytechnique Fédérale de Lausanne (EPFL), Switzerland. Corre-
spondence to: Wanyun Xie <wanyun.xie@epfl.ch>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by the author(s).
significantly influences the generalizat...
Conference: ICML 2025

Temporal Difference Flows
Jesse Farebrother 1 2 † Matteo Pirotta 3 Andrea Tirinzoni 3 R´emi Munos 3
Alessandro Lazaric 3 Ahmed Touati 3
Abstract
Predictive models of the future are fundamental
for an agent’s ability to reason and plan. A com-
mon strategy learns a world model and unrolls
it step-by-step at inference, where small errors
can rapidly compound. Geometric Horizon Mod-
els (GHMs) offer a compelling alternative by di-
rectly making predictions of future states, avoid-
ing cumulative inference errors. While GHMs
can be conveniently learned by a generative ana-
log to temporal difference (TD) learning, existing
methods are negatively affected by bootstrapping
predictions at train time and struggle to generate
high-quality predictions at long horizons. This
paper introduces Temporal Difference Flows (TD-
Flow), which leverages the structure of a novel
Bellman equation on probability paths alongside
flow-matching techniques to learn accurate GHMs
at over 5× the horizon length of prior methods.
Theoretically, we establish a new convergence re-
sult and primarily attribute TD-Flow’s efficacy to
reduced gradient variance during training. We fur-
ther show that similar arguments can be extended
to diffusion-based methods. Empirically, we vali-
date TD-Flow across a diverse set of domains on
both generative metrics and downstream tasks, in-
cluding policy evaluation. Moreover, integrating
TD-Flow with recent behavior foundation models
for planning over policies demonstrates substan-
tial performance gains, underscoring its promise
for long-horizon decision-making.
1. Introduction
Predictive modeling lies at the heart of intelligent decision-
making, enabling agents to reason and plan in complex
environments. In Reinforcement Learning (RL), this pre-
†Work done at Meta 1McGill University 2Mila - Qu´ebec AI
Institute 3FAIR at Meta. Correspondence to: Jesse Farebrother
<jfarebro@cs.mcgill.ca>, Ahmed Touati <atouati@meta.com>.
Proceedings of the 42 nd International Con...
Conference: ICML 2025

Tractable Transformers for Flexible Conditional Generation
Anji Liu * 1 2 Xuejie Liu * 3 4 Dayuan Zhao 5 Mathias Niepert 2 Yitao Liang 3 Guy Van den Broeck 1
Abstract
Non-autoregressive (NAR) generative models are
valuable because they can handle diverse con-
ditional generation tasks in a more principled
way than their autoregressive (AR) counterparts,
which are constrained by sequential dependency
requirements. Recent advancements in NAR mod-
els, such as diffusion language models, have
demonstrated superior performance in uncondi-
tional generation compared to AR models (e.g.,
GPTs) of similar sizes. However, such improve-
ments do not always lead to improved conditional
generation performance. We show that a key rea-
son for this gap is the difficulty in generalizing
to conditional probability queries (i.e., the set
of unknown variables) unseen during training.
As a result, strong unconditional generation per-
formance does not guarantee high-quality condi-
tional generation. This paper proposes Tractable
Transformers (Tracformer), a Transformer-based
generative model that is more robust to differ-
ent conditional generation tasks. Unlike exist-
ing models that rely solely on global contextual
features derived from full inputs, Tracformers in-
corporate a sparse Transformer encoder to cap-
ture both local and global contextual informa-
tion. This information is routed through a de-
coder for conditional generation. Empirical re-
sults demonstrate that Tracformers achieve state-
of-the-art conditional generation performance on
text modeling compared to recent diffusion and
AR model baselines. Code is available at https:
//github.com/liuanji/Tracformer.
*Equal contribution
1Department of Computer Science, Uni-
versity of California, Los Angeles 2Institute for Artificial Intelli-
gence, University of Stuttgart 3Institute for Artificial Intelligence,
Peking University 4School of Intelligence Science and Technology,
Peking University 5Yuanpei College, Peking Universi...
Conference: ICLR 2025

Published as a conference paper at ICLR 2025
ADBM: ADVERSARIAL DIFFUSION BRIDGE MODEL
FOR RELIABLE ADVERSARIAL PURIFICATION
Xiao Li1∗, Wenxuan Sun1,2∗, Huanran Chen1,3, Qiongxiu Li4,
Yining Liu1,5, Yingzhe He6, Jie Shi6, Xiaolin Hu1†
1Department of Computer Science and Technology, Tsinghua University
2Peking University
3Beijing Institute of Technology
4Aalborg University
5Harbin Institute of Technology, Weihai
6Huawei Technologies
lixiao20@mails.tsinghua.edu.cn, sunwenxuan@stu.pku.edu.cn
huanranchen@bit.edu.cn, qili@es.aau.dk
22S030184@stu.hit.edu.cn, {heyingzhe, shi.jie1}@huawei.com
xlhu@mail.tsinghua.edu.cn
ABSTRACT
Recently Diffusion-based Purification (DiffPure) has been recognized as an ef-
fective defense method against adversarial examples. However, we find DiffPure
which directly employs the original pre-trained diffusion models for adversarial
purification, to be suboptimal. This is due to an inherent trade-off between noise
purification performance and data recovery quality. Additionally, the reliability of
existing evaluations for DiffPure is questionable, as they rely on weak adaptive
attacks. In this work, we propose a novel Adversarial Diffusion Bridge Model,
termed ADBM. ADBM directly constructs a reverse bridge from the diffused ad-
versarial data back to its original clean examples, enhancing the purification capa-
bilities of the original diffusion models. Through theoretical analysis and experi-
mental validation across various scenarios, ADBM has proven to be a superior and
robust defense mechanism, offering significant promise for practical applications.
Code is available at https://github.com/LixiaoTHU/ADBM.
1
INTRODUCTION
An intriguing problem in machine learning models, particularly Deep Neural Networks (DNNs), is
the existence of adversarial examples (Szegedy et al., 2014; Goodfellow et al., 2015). These exam-
ples introduce imperceptible adversarial perturbations leading to significant errors, which has posed
severe threats to practical app...
Conference: ICML 2025

Linear Transformers as VAR Models: Aligning Autoregressive Attention
Mechanisms with Autoregressive Forecasting
Jiecheng Lu 1 Shihao Yang 1
Abstract
Autoregressive attention-based time series fore-
casting (TSF) has drawn increasing interest, with
mechanisms like linear attention sometimes out-
performing vanilla attention. However, deeper
Transformer architectures frequently misalign
with autoregressive objectives, obscuring the un-
derlying vector autoregressive (VAR) structure
embedded within linear attention and hindering
their ability to capture the data generative pro-
cesses in TSF. In this work, we first show that
a single linear attention layer can be interpreted
as a dynamic VAR structure. We then explain
that existing multi-layer Transformers have struc-
tural mismatches with the autoregressive forecast-
ing objective, which impair interpretability and
generalization ability. To address this, we show
that by rearranging the MLP, attention, and input-
output flow, multi-layer linear attention can also
be aligned as a VAR model. Then, we propose
Structural Aligned Mixture of VAR (SAMoVAR),
a linear Transformer variant that integrates in-
terpretable dynamic VAR weights for multivari-
ate TSF. By aligning the Transformer architec-
ture with autoregressive objectives, SAMoVAR
delivers improved performance, interpretability,
and computational efficiency, comparing to SOTA
TSF models. The code implementation is avail-
able at this link.
1. Introduction
In recent years, autoregressive decoder-only Transform-
ers have made significant strides (Vaswani, 2017; Radford,
2018), powering Large Language Models (LLMs) (Brown
et al., 2020; Touvron et al., 2023) capable of handling com-
plex sequential data. Their core mechanism, autoregressive
1Georgia Institute of Technology. Correspondence to: Shihao
Yang <shihao.yang@isye.gatech.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by the author(...
Conference: ICML 2025

Surrogate Prompt Learning: Towards Efﬁcient and
Diverse Prompt Learning for Vision-Language Models
Liangchen Liu 1 Nannan Wang 1 Xi Yang 1 Xinbo Gao 1 Tongliang Liu 2
Abstract
Prompt learning is a cutting-edge parameter-
efﬁcient ﬁne-tuning technique for pre-trained
vision-language models (VLMs). Instead of learn-
ing a single text prompt, recent works have re-
vealed that learning diverse text prompts can ef-
fectively boost the performances on downstream
tasks, as the diverse prompted text features can
comprehensively depict the visual concepts from
different perspectives. However, diverse prompt
learning demands enormous computational re-
sources. This efﬁciency issue still remains un-
explored. To achieve efﬁcient and diverse prompt
learning, this paper proposes a novel Surro-
gate Prompt Learning (SurPL) framework. In-
stead of learning diverse text prompts, SurPL di-
rectly generates the desired prompted text fea-
tures via a lightweight Surrogate Feature Gen-
erator (SFG), thereby avoiding the complex gra-
dient computation procedure of conventional di-
verse prompt learning. Concretely, based on a
basic prompted text feature, SFG can directly
and efﬁciently generate diverse prompted fea-
tures according to different pre-deﬁned condi-
tional signals. Extensive experiments indicate
the effectiveness of the surrogate prompted text
features, and show compelling performances
and efﬁciency of SurPL on various benchmarks.
Code is available at https://github.com/
llcllc1997/SurPL.
1. Introduction
Recently, vision-language models (VLMs) (Radford et al.,
2021; Jia et al., 2021; Yao et al., 2021; Zhai et al., 2022)
have shown great capability in open-world visual under-
1Xidian University, Xi’an 710071, Shaanxi, China. 2The Uni-
versity of Sydney, Darlington, NSW 2008, Australia. Correspon-
dence to: Nannan Wang <nnwang@xidian.edu.cn>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by the author(...