ICML 2025

Enhancing Adversarial Robustness with Conformal Prediction: A Framework
for Guaranteed Model Reliability
Jie Bao 1 2 Chuangyin Dang 3 Rui Luo† 2 3 Hanwei Zhang 4 Zhixin Zhou 5
Abstract
As deep learning models are increasingly de-
ployed in high-risk applications, robust defenses
against adversarial attacks and reliable perfor-
mance guarantees become paramount. Moreover,
accuracy alone does not provide sufficient assur-
ance or reliable uncertainty estimates for these
models. This study advances adversarial train-
ing by leveraging principles from Conformal Pre-
diction. Specifically, we develop an adversarial
attack method, termed OPSA (OPtimal Size At-
tack), designed to reduce the efficiency of con-
formal prediction at any significance level by
maximizing model uncertainty without requiring
coverage guarantees. Correspondingly, we intro-
duce OPSA-AT (Adversarial Training), a defense
strategy that integrates OPSA within a novel con-
formal training paradigm. Experimental evalua-
tions demonstrate that our OPSA attack method
induces greater uncertainty compared to baseline
approaches for various defenses. Conversely, our
OPSA-AT defensive model significantly enhances
robustness not only against OPSA but also other
adversarial attacks, and maintains reliable pre-
diction. Our findings highlight the effectiveness
of this integrated approach for developing trust-
worthy and resilient deep learning models for
safety-critical domains. Our code is available
at https://github.com/bjbbbb/Enha
ncing-Adversarial-Robustness-wit
h-Conformal-Prediction.
1Huaiyin Institute of Technology, Huai’an, China 2Chengdu
Research Institute, City University of Hong Kong, Chengdu, China
3City University of Hong Kong, Hong Kong, China 4Saarland Uni-
versity, Saarbr¨ucken, Germany 5Alpha Benito Research, Los Ange-
les, USA. Correspondence to: Rui Luo <ruiluo@cityu.edu.hk>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 202...