Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability
Authors: Jie Bao
Chuangyin Dang
Rui Luo
Hanwei Zhang
Zhixin Zhou
Venue: ICML 2025 poster (2025)
Keywords: adversarial attack
game theory
adversarial training
conformal prediction
conformal training
Primary area: general_machine_learning
TL;DR: This paper proposes an adversarial attack to maximize uncertainty and uses adversarial training to enhance robustness.

Abstract: As deep learning models are increasingly deployed in high-risk applications, robust defenses against adversarial attacks and reliable performance guarantees become paramount. Moreover, accuracy alone does not provide sufficient assurance or reliable uncertainty estimates for these models. This study advances adversarial training by leveraging principles from Conformal Prediction. Specifically, we develop an adversarial attack method, termed OPSA (OPtimal Size Attack), designed to reduce the efficiency of conformal prediction at any significance level by maximizing model uncertainty without requiring coverage guarantees. Correspondingly, we introduce OPSA-AT (Adversarial Training), a defense strategy that integrates OPSA within a novel conformal training paradigm. Experimental evaluations demonstrate that our OPSA attack method induces greater uncertainty compared to baseline approaches for various defenses. Conversely, our OPSA-AT defensive model significantly enhances robustness not only against OPSA but also other adversarial attacks, and maintains reliable prediction. Our findings highlight the effectiveness of this integrated approach for developing trustworthy and resilient deep learning models for safety-critical domains. Our code is available at https://github.com/bjbbbb/Enhancing-Adversarial-Robustness-with-Conformal-Prediction.

Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability
Jie Bao 12 Chuangyin Dang² 3 Rui Luo 23 Hanwei Zhang*
Zhixin Zhou 5
Abstract
1. Introduction
Recent advancements in Deep Neural Networks (DNNs) have demonstrated remarkable efficacy across diverse do- mains (He et al., 2016b; Gupta & Verma, 2023b;a). Despite their success, a critical challenge remains in the precise quantification of predictive uncertainty during real-world deployment. Addressing this challenge, Conformal Pre- diction (CP) (Vovk et al., 2005) has emerged as a promis- ing paradigm, distinguished by its distribution- free proper- ties and robust uncertainty quantification capabilities. This data-driven methodology excels in both regression (Luo & Zhou, 2025a) and classification (Luo & Zhou, 2024) tasks by providing statistically rigorous confidence intervals and well-calibrated probability estimates. Consequently, CP effectively characterizes inherent data uncertainty while maintaining controlled classification error rates. Moreover, through the implementation of defensive mechanisms, CP substantially enhances model robustness against adversarial perturbations (Li et al., 2024), highlighting its significant potential for deployment in safety-critic applications such as autonomous driving (Doula et al., 2024) and medical diagnosis (Luo et al., 2024).
As deep learning models are increasingly de- ployed in high-risk applications, robust defenses against adversarial attacks and reliable perfor- mance guarantees become paramount. Moreover, accuracy alone does not provide sufficient assur- ance or reliable uncertainty estimates for these models. This study advances adversarial train- ing by leveraging principles from Conformal Pre- diction. Specifically, we develop an adversarial attack method, termed OPSA (OPtimal Size At- tack), designed reduce the efficiency of con- formal prediction at any significance level by maximizing model uncertainty without requiring coverage guarantees. Correspondingly, we intro- duce OPSA-AI (Adversarial Training), a defense strategy that integrates OPSA within a novel con- formal training paradigm. Experimental evalua- tions demonstrate that our OPSA attack method induces greater uncertainty compared to baseline approaches for various defenses. Conversely, our OPSA-AT defensive model significantly enhances robustness not only against OPSA but also other adversarial attacks, and maintains reliable pre- diction. Our findings highlight the effectiveness of this integrated approach for developing trust- worthy and resilient deep learning models for safety-critical domains. Our code is available at https:: thtb. com hom/boja jbbbbb Enha ncing Adversaria -obusttess--ueeeeeeeeeeeeeeee -iit H-Conformall-Pediicc..
In the context of CP, current research efforts in adver- sarial robustness model primarily focus adjusting non- consistent scoring functions to mitigate the impact of ad- versarial attacks test or calibration data (Gendler et al., 2021; Einbinder et al., 2022: Ghosh et al., 2023; Yan et al., 2024; Cauchois et al., 2024). However, these approaches present significant limitations. Firstly, ensuring comprehen- sive coverage often requires substantially increasing the size of the prediction set, which inevitably reduces the practi- cal value and accuracy of the prediction results. Secondly, these methods are typically optimized for specific types of adversarial attacks (Croce et al., 2021), rendering their robustness vulnerable when faced with unknown or novel attack vectors. Additionally, existing techniques exhibit notable deficiencies in computational efficiency and gen- eralization ability, significantly hindering their widespread adoption and practical application.
'Huaiyin Institute of Technology, Huai'an an, China "Chengdu Research Institute. City University of Hong Kong, Chengdu, China CCity University of Hong Kong, Hong Kong, China "Sarrland Uni- versity, Saarbrüccknn, Germany" Alpha Benito Research, Los Ange- les, USA. Correspondence to: Rui Luo <ruiluo @ cit...cu..............
Some CP methods enhance adversarial robustness even fur- ther by integrating adversarial training (AT) algorithms. In these algorithms, adversarial training is conceptualized as a
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

OPSA: Hard Quantile Framework for Adversarial Robustness
two-player zero-sum game, where a defender and an adver- sary strive to minimize and maximize classification errors, respectively (Nouiched et al., 2019). However, the inherent discontinuity of classification errors poses challenges for first-order optimization algorithms, making the practical implementation of this zero-sum framework difficult. Re- cently, (Robey et al., 2024) redefined adversarial training as a non-zero-sin game by pursuing distinct objectives, thereby advancing the training process.
2. Related Work
Conformal prediction (CP) (Vovk et al., 2005) is a method- ology designed to generate prediction regions for variables of interest, thereby enabling the estimation of model un- certainty by substituting point predictions with prediction regions. This methodology has been widely applied in both classification (Luo & Zhou, 2024; Luo & Colombo, 2024; Luo & Zhou, 2025b) and regression tasks (Luo & Zhou, 2025e:f). Furthermore, CP can be adapted to diverse real-world scenarios, including segmentation (Luo & Zhou, 2025c), time-series forecasting (Su al., 2024), and graph- based applications (Luo al., 2023; Tang al., 2025; Luo & Zhou, 2025d; Wang et al., 2025; Luo & Colombo, 2025; Zhang et al., 2025).
Despite these advancements, the application of Conformal Prediction (CP) within adversarial settings presents unique challenges (Gendler et al., 2021). Specifically, within the CP framework, the primary objective is to minimize the size of the prediction set while ensuring statistical coverage. Achieving this balance is crucial, as overly large prediction sets can reduce the practical utility and interpretability of the model's outputs. However, there is scarcity of research that directly addresses the optimization of prediction set size within CP, especially in adversarial contexts. Most existing studies focus on maintaining coverage under adversarial per- turbations, often at the expense of significantly enlarging the prediction sets (Ghosh et al., 2023). Notably, (Stutz al., 2021) introduced conformal training methods aimed at more effectively minimizing the prediction set size without com- promising coverage guarantees. This approach represents a promising direction for future work, as it seeks to enhance the practical applicability of CP by ensuring that the gen- crated prediction sets are both accurate and manageable in size.
Adversarial Attack against Conformal Prediction The emergence of adversarial phenomena (Goodfellow et al., 2014; Zhang et al., 2022) has raised significant security con- cerns in machine learning. Uncertainty estimation plays a vital role in ensuring the robustness of deep learning mod- els. Conformal Prediction (CP) (Vovk et al., 2005), offers distribution- coverage guarantees but encounters difficul- ties when subjected to data poisoning and adversarial attacks. Studies such as Liu et al. (2024) demonstrate that standard adversarial attack techniques, including PGD (Madry et al., 2017), can effectively compromise the robustness of con- formal prediction. Kumar et al. (2024) provide a survey and comparative analysis of robust conformal prediction methods, highlighting their strengths and limitations.
To address the aforementioned challenges, we propose adversarial training algorithm within the Conformal Predic- tion framework, aimed at minimizing the prediction set size while ensuring coverage. Specifically, our contributions are as follows:
Adversarially Robust Conformal Prediction To miti- gate the adversarial impact on CP, a series of studies have at- tempted to address this issue without involving training. Ad- versarially Robust Conformal Prediction (ARCP) (Gendler et al., 2021) combines conformal prediction with random- ized smoothing to ensure finite-sample coverage guaran- tees under ----noom-boudee adversarial noise. It leverages Gaussian noise to bound the Lipschitz constant of the non- conformity score, addressing unknown adversarial perturba- tions without training. Probabilistically Robust Conformal Prediction (PRCP) (Ghosh et al., 2023) adapts to pertur- bations using a ofunttll-oo-quuntti design. determining thresholds for both data samples and perturbations. It uses adversarial attacks to calculate empirical robust quantiles, independent of model training. Yan et al. (2024) propose Post- Dost-Training Transformation (PTT) and Robust Conformal Training (RCT) to improve the efficiency of robust confor- mal prediction. They modify RSCP into RSCP+ for certified guarantees and embed it into the training process. Zargar- bashi et al. (2024) derive robust prediction sets by bounding worst-case changes in conformity scores for adversarial eva- sion and poisoning attacks. They use CDF-based bounds to compute conservative prediction sets and thresholds for
We design a differentiable and smooth attack function based on negative gradients. This attack method max- imizes the size of the prediction set by introducing imperceptible perturbations.
We develop a CP defense model leveraging adversarial attacks with theoretical guarantees. By partitioning the training set into two subsets- one dedicated ensur- ing coverage and the other to minimize the prediction set size-we we address this a bi-objective optimization problem.
Through experiments CIFAR-10, CIFAR- and mini- ImageNet datasets, we demonstrate that our attack method generates the largest prediction set sizes com- pared to existing attack techniques, thereby increas- ing model uncertainty. Additionally, our adversarially trained model effectively minimizes model uncertainty and enhances robustness relative to other methods.
2

---

OPSA: Hard Quantile Framework for Adversarial Robustness
these scenarios. Jeary et al. (2024) introduce Verifiable Robust Conformal Prediction (VRCP), leveraging neural network verification to maintain coverage guarantees under adversarial attacks. VRCP supports arbitrary norm-bounded perturbations and extends to regression tasks. These meth- ods partially mitigate the adversarial impact; however, they either compromise the compactness of the prediction set or fail to maintain robustness against different types of attacks and perturbation sizes.
is the calibration set, then
91-a [(1 [I - -t largest value in
{y, (xi) i E I
The resulting prediction set, [(x; f. q1-). then guarantees marginal coverage:
P(u E Txx
(2)
This means that, under the assumption of exchangeability between calibration and test data, the prediction set includes the true label y with a probability of at least 1 a.
Adversarial Training for Conformal Prediction To fur-
ther enhance adversarial robustness, natural approach is to incorporate adversarial training into CP. Liu et al. (2024) propose Uncertainty- Uncertainty-Reducingg Adversarial Training (AT- UR) to improve CP efficiency and adversarial robustness by minimizing predictive entropy and using a weighted loss based on True Class Probability Ranking. They integrate AT-UR with AT (Madry et al., 2017), FAT (Zhang et al., 2020), and TRADES (Zhang et al., 2019), using adversarial examples generated by PGD (Kurakin et al., 2016). Luo et al. (2024) propose training specialized defensive models tailored to specific attack types and utilizing maximum and minimum classifiers to effectively combine these defenses. However, existing adversarial training methods for CP are limited specific attacks due the challenge of solving the max-min optimization in this context. To address this, we reformulate it as bi-level optimization, making our method attack-agnostic.
Remark 1 (Choice of Non-conformity Score). In the pre- ceding discussion and our proposed framework, we pri- marily adopt the specific non-conformity score s(x, = - (x) as utilized in the Threshold Response (THR) method (Sadinle et al., 2019). It is important to note that our methodology is not inherently limited to this particular score. A variety of other non-conformity score functions exist, such as those proposed in (Romano et al., 2020; Luo & Zhou, 2024; 2025b). The core principles of our attack and defense strategies can be readily extended to accommo- date alternative score functions s (x. y), provided that these functions are differentiable (or at least sub-diiferentiable) with respect to the model f's outputs (or more precisely, with respect to the values fx(x) upon which sy depends). Our choice of the THR score function is primarily moti- vated by its potential to yield efficient prediction sets under standard conditions. Furthermore, the resulting score for the true class, s(x. y) = fy()), simplifies the notation and derivation within our framework.
3. Method
Consider a classifier network f : - R that maps an input image E X logit vector f(x) K Here, X [0, 1]dd represents the image space, where pixel values are normalized and d denotes the image dimension. The integer K signifies the total number of classes, and E [K] = [1....,K] K} denotes the ground truth label for the
3.2. Adversarial Training as a Min-Max Problem.
Adversarial training is often conceptualized as a min-max optimization problem, aiming to find model parameters that minimize the loss on adversarially perturbed inputs. This saddle-point problem can be formulated as (Madry et al., 2017):
input x.
3.1. Conformal Prediction
max E(0, + t.,.)) ||e|,gr
For given classifier f and an input x, a prediction set can be formed by selecting all classes l whose corresponding logit fx(x) exceeds a certain threshold r. This prediction set is defined as:
min E (x.)DD
(3)
Here, (x, 3) represents a data point sampled from the dis- tribution D, consisting of an input x and its true label y. The term e denotes an adversarial perturbation, constrained within a specific norm-ball (e.g., |/el|p S r. where p could be or 2, and is the perturbation budget). The func- tion ε(O, x + e, y) is the loss incurred by the model with parameters 0 on the perturbed input x + e with respect to the true label y. The choice of the loss function I can vary depending on the specific task. In this paper, we will adopt a loss function for conformal prediction task.
T(x: f.r)= (k [K] fx(x)= 2 r}.
(L)
The THR method for conformal prediction (Sadinle et al., 2019) determines this threshold T as the (1 - a) quuuutle,,,, denoted q1-a. of conformity scores computed on a separate calibration set. To be more precise, if [(x,, for i E Ical
3

---

OPSA: Hard Quantile Framework for Adversarial Robustness
3.3. Adversarial Attack against Conformal Prediction
Algorithm Optimal Size Attack (OPSA)
Require: A single labeled data: (x, y);
An adversarial perturbation against conformal prediction aims to disrupt the prediction set [( 7) of classifier f while remaining imperceptible. The adversarial attack seeks to achieve the following objectives:
Perturbation budget: r; Temperature: T1; Maximum iteration number: J;
Targeted classifier: f:
Learning rate: 17-
Ensure: Adversarial perturbation: E²,
Maximize uncertainty: Increase the expected size of the conformal prediction set || (f(x e), I] to reduce the informativeness of predictions at any threshold is unknown since the attacker do not know the significance level of the defender.
I: function OPSA (Dimin: f, Ti,
2: +> Initialize Perturbation:
3:
et (.)
4: while j S or has not converged do
> Update Perturbation:
+ V M7, (x + e; ( e))
Maintain imperceptibility: Ensure the perturbation satisfies the constraint ||e|p S r for predefined bud- get to remain undetectable.
7:
1I ((0.00
8: レ j+1
9: end while
10: Return e*
We propose the following objective for fixed classifier net- work but it does not involve significance level a. We first define soft set size
11: end function
[xx T
)
f,
E kE|K]
(4)
applies scaled sigmoid function to these effective differ- ences (achieved by setting the internal threshold of Mr to fu(x + c)) to specifically target the maximization of the (soft) prediction set size. A key advantage of this formula- tion is that the attacker does not require knowledge of the defender's s chosen significance level a (and consequently, the operational threshold 7) craft the perturbation c*.
where (x) 1/(1++ is the sigmoid function and T > 0 is temperature hyperparameter. It is worth noting that for fixed (x; F, such that fu + T for every k E [K].
fa(x)) -x(x))=)= T
lim T1o
=1
T(x,TT)
(5)
Algorithm details the iterative process for finding the optimal perturbation e* for given input sample. Although presented conceptually as gradient ascent for simplicity, our actual implementation leverages the Adam optimizer for more effective and stable optimization, leading to higher- quality adversarial perturbations. critical component of each iteration is the projection of the updated perturbation back onto the feasible region. This step guarantees that the perturbation satisfies both the norm constraint, ||e||p < (confining to the ball (O)). and the image validity constraint, ensuring x t E [0, 1]d. The set of perturbations satisfying the image validity constraint can be expressed as [0, 1] - x = {e E Rd + E [0, 1]4).
and
lim (I) (xxxx)T)
(6)
T10
The sigmoid function and Mr represent the soft indica- tor function and set size function respectively. This is technique also employed in conformal training methodolo- gies (Stutz et al., 2021). For a given input sample (x, y), we propose that the attacker's 's objective is to find an adversarial perturbation e" (x, y) that maximizes this soft set size, using the perturbed true class score [x + e) as the internal ref- erence threshold within Mr. The optimization problem is thus formulated
3.4. Adversarial Robust Conformal Training
e" e"(x,y) arg max + (5xxx++555xxxxxx (7)
M
In the context of the min-max formulation presented in (3), adversarial training (AT) endeavors to identify a classifier f(x; 8), parameterized by 0. that minimizes the loss in- curred from potential adversarial perturbations applied to the input images. Our work posits that for achieving robust conformal prediction, it is particularly crucial to train the classifier against the specific perturbation e" generated by our OPSA method (detailed in Algorithm 1). The training of this classifier, f( ;)), is subsequently guided by the two loss components detailed below.
Sr x+
Here, the maximization is performed for each specific sam- ple (x, y) to find its corresponding optimal perturbation.
The construction of this objective function shares conceptual similarities with prior work, such as (Robey et al., 2024), particularly in its implicit use of logit differences of the form fx(x + <) fy(x + c). Our approach distinctively
4

---

OPSA: Hard Quantile Framework for Adversarial Robustness
Classification Loss. The classification loss, denoted as Lelass, is designed to encourage the inclusion of the true class label y within the soft prediction set (implicitly defined by r and T2), while simultaneously penalizing the inclusion of incorrect classes. For a given input x, its true label y, classifier f. threshold T, and temperature T2, this loss is defined as:
Algorithm 2 OPSA Adversarial Training (OPSA-AT)
Require: A set of labeled data: Dirain
Pre-trained classifier parameter: finti
Pre-determined Coverage Probability: 1 Oz
Maximum number of epoch: J':
Temperature: T2; Loss weight: A; Parameters for adversarial attack: r, T,. J.
fx(x) T2
[x) T2
Ensure: Robust conformal prediction model f".
Celase (x, f.
Σo ke
I: function - f, T2, K",
t> Initialize classifier parameter:
4 mit
To understand the behavior of this loss, consider the limit as T2 + O, where the sigmoid function (.) approximates an indicator function I(. 2 0). In this scenario, the first term, ( approaches 1 if (x) 2 T (i.e., the true label is correctly included in the hard prediction set), and 0 otherwise. The second term, Ener a approximates the count of incorrect classes l: * y for which (x) 2 (i.e., the number of incorrect labels erroneously included in the hard prediction set). Thus, Lelass effectively approximates the difference between an indicator for the true label's s inclusion and the count of incorrectly included labels. Maximizing this loss (or minimizing its negative) encourages both the inclusion of y and the exclusion of l + y from the prediction set defined by
4:
while S J' or A has not converged do
5:
for mini-Batch B C Pirain do
6:
Randomly split B into Busiu and Beal-
7:
D Train attacker's perturbation:
8:
for E Buain do
6 f- OPSA Vi), r, (.,0,,
[x(x)=7)
73
10:
end for
11:
b Find threshold:
12:
+- as the T(L + [Baal) - a)l largest
fy. (x;; 0) for (x,, 9i) E Beal-
13:
D- Update classifier parameter:
14:
I t set of indices of Buain
15:
A - - Ve Σ (xotal(X, + 4. 0, T2)
IEZ
16:
j 3 J +
17:
end for
Size Loss. The size loss, denoted as Lazze, aims to mini-
18:
end while
mize the average size of the confidence sets, thereby reduc- ing inefficiency. The soft prediction set size is defined in (4). We will use the same notation Mr and its definition.
19:
8" <- O
20: Return fAT(- 0r)
21:
end function
Total Loss Function. To balance the contributions of clas- sification accuracy and the efficiency of the confidence sets, the total loss function integrates the classification loss and the size loss using& weighting factor X:
The optimization of (model training) and the determina- tion of the threshold (calibration) are performed iteratively, as the optimal 7 depends on the current model f(: 0), and the update to 0 depends
Leotal y; 0, T2) = -elaaso (x, yi T,;
(8)
AMMr (x; f(:
Algorithm 2 outlines the steps of our adversarial robust con- formal training. In addition to the optimization procedure for 6, the algorithm first specifies the mini-batch determi- nation during the training process. For each batch, the data (now adversarial) is further split into training and calibra- tion subsets. The adversarial perturbation e* is generated based on the data designated for the training subset of the batch, and the threshold r is subsequently obtained from the calibration subset of the same batch.
where 0 are the parameters of f.
The algorithm trains the model on noisy images subjected the OPSA attack. We assume the defender possesses foise-free images with their corresponding labels. The perturbation c", as the out- put of Algorithm 1, is added to the images. Differing from some conformal training approaches such as (Stutz et al., 2021), the threshold r in our method is obtained by comput- ing the exact quantile, rather than being determined via a aigmoid-approximated mechanism. The objective is then to find 0* that minimizes the loss function defined in (8):
Adversarial Training Procedure.
3.5. Conformal Prediction with the Robust Classifier
Having obtained a robust classifier, [sT 0*), through the adversarial training procedure (OPSA-AT), we now inte- grate with the conformal prediction methodology. The application of conformal prediction largely mirrors the pro- cedure detailed in Section 3.1, with the crucial substitution
8* := arg min
Σ Lxotalx; + G. 0, T2).
Tunin
5

---

OPSA: Hard Quantile Framework for Adversarial Robustness
of the original classifier f(x) with our adversarially trained classifier [xT(x;0% (x) The following key aspects underpin this integration.
Data Splitting. A fundamental prerequisite for valid con- formal guarantees is that the dataset used for training the model parameters 0" (i.e., the adversarial training set) must be strictly disjoint from the samples utilized for the CP cali- bration phase. This separation is crucial for the integrity of the CP guarantees.
Aeuik Habil
(a) FGSM
Exchangeability for Calibration and Test Data. The validity CP hinges on the core assumption of exchange- ability between the calibration and test samples. It is im- portant emphasize that this exchangeability assumption does not impose specific requirements samples are adversarially perturbed or not. For instance, the CP guarantees hold if all calibration and test samples are clean (unperturbed), if all are perturbed (e.g., by the OPSA attack detailed in Algorithm 1), or a random subset of them perturbed, as long as the perturbation mechanism (or lack thereof) is applied consistently or randomly in a way that preserves exchangeability between the sets. The validity of CP relies solely on this exchangeability property of the (potentially transformed) data points.
whether these
And
(b) PGD
Experimental Focus. In the experimental evaluations pre- sented in this paper, we specifically investigate a challeng- ing scenario. While the adversarial training for 0* might be based on or include clean samples, both the calibration and test datasets are composed samples adversarially per- turbed by our OPSA method. This setup stringently tests the robustness of the conformalized predictions.
(c) TRADES
Figure I. Box-violin plots of CIFAR- 10 results under FGSM, PGD, and TRADES defense models
4. Experiments
Settings. In this section, we evaluate the performance of OPSA and OPSA-AT on the CIFAR-10 (Krizhevsky et al., 2009) and CIFAR-100 datasets using ResNet34 (He et al., 2016a) as the training framework. Additionally, we con- duct evaluations on the mini-ImageNNet (Deng al., 2009; Vinyals al., 2016) dataset using ResNet50. For attack, we compare OPSA to methods such as FGSM (Goodfel- low et al.. 2014), LO (Madry et al., 2017), #GD40 BETA 10 (Robey et al., 2024), Square (1000 queries) (An- driushchenko et al., 2020), 100 attack (Croce & Hein, 2020), and APGD100 100 (Croce & Hein, 2020). where the su- perscript indicates the attack iteration. For defense, we select adversarial training based on FGSM, PGD, as well as TRADES (Zhang et al., 2019), MART (Wang et al., 2019), and BETA-AT for comparison, with each model undergoing 10 iterations. For parameter settings, we adhere to the stan- dard perturbation budget of = 8/255 (0.03) and lo norm, applying step size of 2/2 2/255 attacks during training and
testing. The trade-off parameter for TRADES and MART is set 5, in alignment with their original implementa- tions. To initialize clean data for 5 epochs the CIFAR-10 10 and CIFAR- 100 datasets, while for the mini- ImageNet dataset, we perform 10 epochs of pre-trainne pre-training on clean data. Subsequently, on the CIFAR- 10 and CIFAR 100 datasets, we split the test set into Tenl and Zest in a ratio of 20% 80%. Given that the training and test sets of ImageNet are not interchange- able, we divide its training set into Ierin Ical, and Inst in proportions of 50%, 25%, and 25%, respectively. Follow- ing (Stutz et al., 2021), we set both 71 and 72 to 1 on the CIFAR-10 10, CIFAR-100,p approximate the THR method. We evaluate model robust- ness by launching attacks on both validation and test sets, setting the alpha parameter 10%.
models, We conduct pre-training
Mini-ImageNet datasets to
6

---

OPSA: Hard Quantile Framework for Adversarial Robustness
Training Algorithm
Attacks
Indicator
FGSM
PGD
TRADES
MART
BETA-AT
OPSA-ST
OPSA-AT
Coverage (%)
90.50 0.33
89.40 0.34
89.72 = 0.35
89.44 = 0.34
88.24 = 0.36
89.49 0.31
89.25 = 0.34
Clean
Size
2.00 = 0.01
4.15 0.02
1.51 1.01
1.29 0.01
7.06 + 0.03
1.30 1.30
1.24 0.01
SSCV
0.04 = 0.01
0.02 # 0.01
0.04 =.01
0.10 0.01
0.11 = 0.03
0.08 0.03
0.08 1 0.01
Coverage (%)
87.99 0.37
88.88 0.38
89.05 0.34
89.08 + 0.35
90.05 0.66
89.38 0.35
89.48 0.35
FGSM
Size
6.61 = 0.02
4.43 = 0.02
3.59 = 0.02
4.61 + 0.02
3.20 + 0.02
2.56 # 0.02
2.50 + 0.02
SSCV
0.03 = 0.01
0.03 = 0.01
0.02 0.01
0.03 0.01
0.08 + 0.07
0.02 = 0.01
0.03 + 0.01
Coverage (%)
88.24 + 0.36
88.84 + 0.36
89.08 = 0.35
89.36 + 0.34
88.90 + 0.34
89.19 = 0.34
89.29 0.35
P
Size
7.06 + 0.02
4.46 = 0.02
3.96 + 0.02
5.19 + 0.02
6.63 + 0.02
3,46 1: 0.03
3.24 + 0.03
SSCV
0.02 + 0.01
0.03 0.01
0.02 + 0.01
0.05 0.01
0.06 + 0.02
0.07 0.01
0.07 0.01
Coverage (%)
88.33 + 0.37
88.84 + 0.34
89.35 = 0.35
89.04 = 0.35
88.92 + 0.35
89.20 0.35
89.30 + 0.03
PGD
Size
7.07 = 0.02
4.47 - 0.02
3.97 1 0.02
5.19 0.02
6.84 0.03
3.46 = 0.03
3.24 0.03
SSCV
0.02 = 0.03
- 0.01
0.02 = 0.01
0.05 0.02
0.08 + 0.02
0.07 + 0.01
0.07
Coverage (%)
88.05 0.37
89.21 0.35
89.06 = 0.35
89.21 + 0.35
90.40 + 0.33
89.69 0.35
88.98 + 0.36
BETA 10
Size
6.38 = 0.02
3.89 - 0.02
2.77 0.02
3.89 0.03
3.19 + 0.01
2.10 + 0.02
2.04 0.02
SSCV
0.03 = 0.01
0.03 = 0.01
0.05 = 0.01
0.05 + 0.01
0.09 + 0.09
0.03 = 0.01
0.03 0.01
Coverage (%)
89.44 0.36
89.40
90.86 0.32
89,41 + 0.35
88.24 0.37
89.52 0.34
89.09 0.34
Square
Size
5.38 + 0.03
4.16 + 0.02
2.16 1 0.02
3.10 + 0.02
7.45 0.03
1.96 + 0.02
1,90 + 0.03
SSCV
0.01 = 0.01
0.02 = 0.01
0.06 0.01
0.01 + 0.03
0.32 + 0.08
0.05 = 0.01
0.03 + 0.01
Coverage (%)
89.25 + 0.34
89.40 + 0.35
89.58 = 0.35
89.59 + 0.33
90.64 + 0.33
89.81 = 0.34
89.78 + 0.35
Size
5.76 = 0.03
1 4.16
2.64 - 0.02
3,94 194::000 0.03
7.66 + 0.03
2.42 + 0.02
2.32 + 0.20
SSCV
0.02 + 0.01
0.02 - 0.01
0.05 0.01
0.05 0.01
0.25 0.06
0.05 - 0.00
0.06 0.00
Coverage (%)
89.22 + 0.36
89.40 + 0.34
89.58 = 0.35
89.56 - 0.34
88.24 - 0.36
89.82 0.33
89.76 + 0.34
Auto
Size
S.77 = 0.03
4.16 0.02
2.66 0.02
3.95 + 0.03
7.71 0.03
2.44 + 0.02
2.36 0.02
SSCV
0.02 + 0.01
0.02 + 0.01
0.05 0.00
0.05 0.01
0.60 + 0.30
0.05 + 0.00
0.00
Coverage (%)
89.61 + 0.36
89.11 0.34
89.62 0.35
90.46 + 0.32
89.92 + 0.35
89.70 0.33
89.50 50 + 0.34
OPSA10
Size
7.29 = 0.02
4.50 = 0.02
4.02 0.02
5.40 0.02
7.34 + 0.02
3.37 + 0.03
3.11 0.03
SSCV
0.09 = 0.00
0.04 = 0.01
0.03 = 0.01
0.04 + 0.01
0.02 + 0.01
0.02 + 0.00
0.03 + 0.01
Table 1. Mean and Standard Deviation of Coverage. Size, and SSCV for CIFAR-10
Metrics. To comprehensively assess the performance of shape-preserving prediction methods, we adopt three core metrics: Coverage, Size, and Size- Sita-Straiifed Coverage Viola- tion (SSCV).
varying prediction set sizes. It is defined as:
SSCV(T,
| E Jn
E ET(xxf.T) (1-a) a)
(II)
sup jE(+]
[J,]
The Coverage metric quantifies the proportion of test in- stances in Lest for which the true label is encompassed within the prediction set [xx; f, r), formulated
where {5,} partitions the possible prediction set sizes, and J, = E Test [I'(x;;; T)] e Sy. A smaller SSCV indicates more stable coverage across different set sizes.
Coverage
Σ
1 (Jr f(xx
[Test
Collectively, these metries strike balance between achiev- ing the desired coverage probability and maintaining in- formative prediction sets, while ensuring that conformal prediction's 's coverage guarantees hold irrespective of the underlying model's accuracy.
A higher coverage value signifies that the prediction sets consistently include the true labels.
The Size metric assesses the average quantity of labels within the prediction sets across all test instances, given by:
Results. We present our experimentatal results in Tables I and 2. To facilitate comparison among attacks, we highlight the best results in each column in bold, while underlining the best defense results in each row. Following prior con- formal prediction studies, we do not highlight results for Coverage, as they typically fluctuate around the theoretical - a level due to finite sample sizes and stochastic vari- ability. Therefore, under the constraint of maintaining the
Size
Σ [
(10)
Iiest]
I
where smaller sizes imply more concise and informative predictions.
The Strattiit Coverage Violation (SSCV) (Angelopou- los et al., 2021) examines the consistency of coverage across
7

---

OPSA: Hard Quantile Framework for Adversarial Robustness
Training Algorithm
Attacks
Indicator
FGSM
PGD
TRADES
MART
BETА-AT
OPSA-ATIO
Coverage (%)
90.26 + 0.32
88,99 + 0.34
90.50 + 0.33
89.24 + 0.36
91.39 = 0.31
89.38 = 0.36
Clean
Size
13.47 + 0.01
32.93 + 0.17
8.81 + 0.07
8.65 0.08
73.65 = 0.27
8.41 0.01
SSCV
0.08 + 0.01
0.09 +090+00011 0.01
0.07 + 0.01
0.06 + 0.01
0.40 + 0.28
0.05 0.00
Coverage (%)
90.67 + 0.33
89.36 + 0.34
90.55 + 0.33
90.48 0.33
88.50 0.37
89.76 0.33
FGSM
Size
60.36 + 0.21
38.01 + 0.20
28.27 + 0.17
29.17 + 0.20
25.71 + 0.12
24.52 + 0.22
SSCV
0.10 + 0.01
0.08 + 0.01
0.07 + 0.01
0.06 + 0.01
0.10 + 0.03
0.03 + 0.01
Coverage (%)
90.65 + 0.33
89.31 + 0.34
90.28 + 0.32
90.74 + 0.34
89.01 = 0.34
89.58 = 0.34
Size
64.94 = 0.22
38.22 + 0.20
32.05 + 0.20
35.17 = 0.23
68.70 = 0.22
33.85 = 0.30
SSCV
0.10 = 0.00
0.09 =.01
0.06 + 0.01
0.05 + 0.01
0.03 - 0.03
0.04 0.01
Coverage (%)
90,66 + 0.32
89.31 + 0.35
90.25 + 0.34
90.75 = 0.32
89.25 = 0.34
90.18 = 0.33
PGD40
Size
64.95 + 0.34
38.22 + 0.20
32.04 + 0.19
35.14 + 0.24
71.12 = 0.23
33.74 = 0.28
SSCV
0.10 0.00
0.09
0.07 + 0.01
0.05 0.01
0.07 0.05
0.04 0.01
Coverage (%)
90.49 + 0.33
89.49 + 0.33
90.36 + 0,32
90.64 + 0.32
88.36 + 0.35
89.41 + 0.34
BETA
Size
57.59 + 0.22
32.98 + 0.19
20.38 + 0.14
20.40 + 0.16
23.44 + 0.11
17.00 + 0.15
SSCV
0.10 + 0.00
0.09 + 0.01
0.10 + 0.00
0.08 + 0.01
0.10 + 0.02
0.06 + 0.02
Coverage (%)
88.98 + 0.33
88.99 + 0.35
90.86 + 0.33
89.79 + 0.34
91.39 + 0.32
89.31 + 0.34
Square
Size
28.15 + 0.18
33.03 + 0.18
11.61 + 0.09
11.60 + 0.10
74.69 = 0.27
13.81 = 0.27
SSCV
0.07 0.01
0.09 0.01
0.09 + 0.00
0.07 0.01
0.40 + 0.26
0.06 - 0.01
Coverage (%)
89.05 + 0.35
88.99 + 0.34
90.22 + 0.34
89.11 0.36
91.37 = 0.31
90.04 = 0.34
APGD 100
Size
32.00 + 0.21
33.06 + 0.18
13.54 + 0.11
13.48 0.12
75.05 = 0.28
16.01 = 0.20
SSCV
0.07 = 0.01
0.09
0.08 + 0.00
0.07 0.01
0.90 0.30
0.05 0.01
Coverage (%)
89.00 + 0.36
88.99 + 0.36
90.22 + 0.34
89.12 + 0.36
91.39 = 0.32
89.50 = 0.31
Auto
Size
32.00 + 0.21
33.06 + 0.18
13.70 + 0.22
13.58 + 0.12
75.10 + 0.27
13.42 0.16
SSCV
0.07 + 0.01
0.09 + 0.01
0.08 + 0.01
0.07 + 0.01
0.90 + 0.29
0.05 + 0.01
Coverage (%)
90.81 + 0.32
89.79 + 0.34
90.31 + 0.33
90.46 + 0.33
89.36 + 0.36
89.16 + 0.35
OPSA 10
Size
65.56 + 0.22
39.23 + 0.20
32.30 + 0.19
35.38 + 0.23
75.25 + 0.20
32.07 + 0.25
SSCV
0.10 + 0.00
0.07 + 0.01
0.09 0.01
0.07 0.01
0.10 + 0.00
0.03 0.01
Table 2. Mean and Standard Deviation of Coverage, Size, and SSCV for CIFAR-100
coverage guarantee, the primary metric of interest is the size of the prediction intervals.
1-2) that illustrate the variance in attack performance across different defense models under identical experimental con- ditions.
As shown in Table 1, we conducted an additional experi- ment on CIFAR-10 using soft thresholding (the conformal training method proposed by Stutz et al. (2021)) 1)), denoted as OPSA-ST. Training with a mini-batch size 64, our de- fense model demonstrates superior performance across all attack methods, yielding the minimal uncertainty (i.e., size) in the results. Furthermore, when excluding the two models inherently trained with OPSA attacks (OPSA-ST and OPSA- AT) and the BETA-AT defense model, our attack method consistently achieves the largest size among other defense frameworks. Notably, despite BETA-AT employing a train- ing attack strategy similar to ours and OPnn-ST(includin OPSA-AT) utilizing the OPSA attack methodology, the size produced by our attack method remains highly competitive with the strongest baseline attacks. To visually demonstrate these discrepancies, we present Box-violin plots (Figures
Additionally, our experimental results on the CIFAR-100 dataset (Table 2) further validate the superiority of our de- fense framework and the potency of our attack methodology compared to existing approaches. Notably, our attack consis- tently produces the largest size across all evaluated defense models except when applied to our own proposed defense mechanism. Furthermore, our defense exhibits significantly reduced predictive uncertainty compared to baseline meth- ods. Due to hardware limitations, we employed a batch size of 64 for standard training and a mini-batch size of 200 for conformal training experiments. While computa- tional constraints currently restrict our implementation to these parameters, we included remark (2) explaining that theoretical analysis suggests optimal performance would be achieved with mini- batch sizes between 500- - 1000, as
8

---

OPSA: Hard Quantile Framework for Adversarial Robustness
Asadi
(a) MART
(b) BETA
(c) OPSA-ST
OPSA-AT
Figure 2. Box-violin plots of CIFAR-10 results under MART, BETA-ST, OPSA-ST, and OPSA-AT defense models
this range better approximates the idealized exchangeability conditions required for robust conformal prediction.
5. Conclusion
In this study, we introduce a novel framework that integrates adversarial training with conformal prediction to enhance the robustness deep learning models against adversarial attacks. We treat adversarial training within this conformal framework dual-objective optimization challenge: the one hand, our designed attack method aims to maximize the uncertainty of the prediction set without prior knowl- edge of the coverage rate; on the other hand, our defense method strives to minimize the uncertainty of the prediction set while maintaining a certain coverage rate. However, it's worth noting that our current experimental setup is relatively limited. We plan expand experiments by including the PreAct ResNet network in future studies. Experimental validations on CIFAR-10, CIFAR-100, 100, and Imageee datasets reveal that, compared to existing methods, our pro- posed attack method generates greater uncertainty, while the defense model demonstrates significantly improved ro- bustness against various adversarial attacks. These findings strongly affirm the effectiveness of combining adversarial training with conformal prediction, providing new insights for developing reliable and resilient deep learning models in safety-critical applications.
Remark 2 (mini-Batch). In Section 3.4, emphasized the critical importance of partitioning datasets into training (Bwain) and calibration (Bea) subsets during model training. To effectively approximate the THR procedure, maintaining exchangeability between Buain and Beal within each mini- batch is essential. However, practical considerations of com- putational efficiency and memory constraints necessitate balance. We recommend setting the subset size between 5 10 times the number of classification categories (K) to optimize both approximation fidelity and training efficiency.
further elaborate on supplementary experimental tails across multiple appendices: Appendix B comprehen- sively profiles the computational overhead of various attack methodologies and defense models: Appendix C details our experimental results on the mini-ImageNet dataset; Ap- pendix D presents comprehensive accuracy metrics for all evaluated defense models under diverse attack scenarios; and Appendix E provides an ablation study analyzing the impact hyperparameter Tj on experimental outcomes.

---

OPSA: Hard Quantile Framework for Adversarial Robustness
Acknowledgements
Computer Vision and Pattern Recognition, pp. 216-226, 2024.
This work was partially supported by Hong Kong RGC and City University of Hong Kong grants (Project No. 9610639 and 6000864), Chengdu Municipal Office of Phi- losophy and Social Science grant 2024BS013, DFG grant No. 389792660, and VolkswagenStiftung Grant AZ 98514. Zhixin Zhou's 's research was supported by the Genesis Award for Scientific Breakthrough from Alpha Benito LLC.
Einbinder, B.-S., Bates, S., Angelopoulos, A. N., Gendler, A., and Romano, Y. Conformal prediction is robust to label noise. arXiv preprini arXiv:2209. 14295, 2. 2022.
Gendler, A., Weng, T.-W., Danicl, L.,, and Romano, Y. Ad- versarially robust conformal prediction. In International Conference on Learning Representations, 2021.
Impact Statement
Ghosh, S., Shi, Y., Belkhouja, T., Yan, Y., Doppa, J., and
Jones, Probabilistically robust conformal prediction. In Uncertainty in Artificial Intelligence, pp. 681-690,
The work presented in this paper aims to advance the field of machine learning, particularly through supplementary theoretical developments and explorations of adversarial training within the Conformal Prediction framework. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
PMLR, 2023.
Goodfellow, I. J., Shlens, J., and Szegedy, C. Explain- ing and harnessing adversarial examples. arXiv preprint arXiv:1412.6577 2014.
Gupta, K. and Verma, S. Shrink & cert: Bi-level opti- mization for certified robustness. New Frontiers in Adversarial Machine Learning, ICML, 2023a.
References
Andriushchenko, M., Croce, F., Flammarion, N., and Hein,
M. Square attack: a query-efficient black-box adversarial attack via random search. In European conference on computer vision, pp. 484-501. Springer, 2020.
Gupta, K. and Verma, S. Certvit: Certified robustness of pre- trained vision transformers. In The Second Workshop on New Frontiers Adversarial Machine Learning, 2023b.
Angelopoulos, A. N., Bates, S., Jordan, M., and Malik, Uncertainty sets for image classifiers using conformal prediction. In International Conference on Learning Representations, 2021.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 770-778. 2016a
Cauchois, M., Gupta, S., Ali, A., and Duchi, C. Robust validation: Confident predictions even when distributions shift. Journal of the American Statistical Association, pp- 1-66, 2024.
He, K., Zhang, X., Ren, S., and Sun, J. Identity mappings in deep residual networks. In Computer EisionEECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV 14, pp. 630- 645. Springer, 2016b.
Croce, F. and Hein, M. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference machine learning, pp. 2206-2216. PMLR, 2020.
Jeary, L., Kuipers, T., Hosseini, M., and Paoletti, N. Ver- ifiably robust conformal prediction. arXiv preprint arXiv:2405.11 18942, 2024.
Croce, F., Andriushchenko, M., Sehwag, V., Debenedetti, E., Flammarion, N., Chiang, M., Mittal, P., and Hein, M. Robustbench: standardized adversarial robustness benchmark. In Thirty-fifth Conference on Neural Informa- tion Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009.
Kumar, B., Kumar, T., and Patel, H. Evaluating confor- mal prediction techniques in adversarial environments: A comparative study. International Journal of Engineering Research & Technology, 2024.
Deng, J. Dong, W., Socher, Li, L.-J, Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Kurakin, A., Goodfellow, I., and Bengio, S. Adversarial ma- chine learning at scale. arXiv preprint arXiv:1611.0125: 2016.
Doula, A., Mühlhäuser, M., and Guinea, A. S. Ar-cp: Uncertainty- -wwre perception in adverse conditions with conformal prediction and augmented reality for assisted driving. In Proceedings of the IEFF/VVF Conference on
Li, Y., Chen, A., Qian, W., Zhao, C., Lidder, D., and Huai, M. Data poisoning attacks against conformal predic- tion. In Forty-first International Conference on Machine Learning, 2024.
10

---

OPSA: Hard Quantile Framework for Adversarial Robustness
Liu, Z., Cui, Y., Yan, Y., Xu, Y., Ji, X., Liu, X., and Chan, A. B. The pitfalls and promise of conformal inference der adversarial attacks. arXiv preprint 2xXiv:4405008880 2024.
Nouiched, M., Sanjabi, M., Huang, T., Lee, D., and Raza- viyayn, M. Solving class of non-convex min-max games using iterative first order methods. Advances in Neural Information Processing Systems, 32, 2019.
Luo, R. and Colombo, N. Entropy reweighted conformal classification. In The 13th Symposium on Conformal and Probabilistic Prediction with Applications, pp. 264-276. PMLR, 2024.
Robey, A., Latorre, F., Pappas, G. J.. Hassani, H., and Cevher, V. Adversarial training should be cast as non-zero-sum game. The Twelfth International Confer- ence on Learning Representations, 2024. URL https: / //openreview.o net forumm wtw.:mmiidddIIwwweeeeweee
Luo, R. and Colombo, N. Conformal load prediction with transductive graph autoencoders. Machine Learning, 1 114 (3):1-22, 2025.
Romano, Y., Sesia, M., and Candes, E. Classification with valid and adaptive coverage. Advances in Neural Infor- mation Processing Systems, 33:3581 1-351,, 2020.
Luo, R. and Zhou, Z. Trustworthy classification through rank-based conformal prediction sets. arXiv preprint arXiv:2407.04407, 2024.
Sadinle, M., Lei, J., and Wasserman, L. Least ambiguous set-valued classifiers with bounded error levels. Journal of the American Statistical Association, 114(525):223- 234, 2019.
Luo, R. and Zhou, Z. Conformal thresholded intervals for efficient regression. Proceedings of the AAAI Conference on Artificial Intelligence, 191111922111922219 2025a.
Stutz, D., Cemgil, A. T., Doucet, A., et al. Learning optimal conformal classifiers. arXiv preprint arXiv:2110009192. 2021.
Luo, R. and Zhou, Z. Conformity score averaging for classi- fication. In Proceedings of the Forty-second International Conference on Machine Learning (ICML), 2025b. ap-
Su, X., Zhou, Z., and Luo, Adaptive conformal inference by particle filtering under hidden markov models. arXiv preprint arXiv:2411.01158 2024.
pear.
Luo, and Zhou, Conditional conformal risk adaptation. arXiv preprint arXiv:2504.0611,, 2025c.
Tang, L., Luo, R., Zhou, Z., and Colombo, N. Enhanced route planning with calibrated uncertainty set. Machine Learning, 114(5):1-16, 2025.
Luo, R. and Zhou, Z.. Conformalized interval arithmetic with symmetric calibration. Proceedings of the AAAI Con- ference on Artificial Intelligence, 39(18))19207-199 2025d.
Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., al. Matching networks for one shot learning. Advances in neural information processing systems, 29, 2016
Luo, R. and Zhou, Conformal thresholded intervals for efficient regression. Proceedings of the AAAI Conference on Artificial Intelligence,
Vovk, V., Gammerman, A., and Shafer, G. Algorithmic learning in random world, volume 29, Springer, 2005.
2025e.
Luo, R. and Zhou, 7. Volume-sorted prediction set: Efficient conformal prediction for multi-target regression. arXiv preprint arXiv:2503.02200 2025f.
Wang, T., Zhou, Z-,, and Luo, R. Enhancing trustworthiness of graph neural networks with rank-based conformal train- ing. Proceedings of the AAAI Conference on Artificial Intelligence, 39(20);
2025.
Luo, R., Nettasinghe, B., and Krishnamurthy, V. Anoma- lous edge detection in edge exchangeable social network models. In Conformal and probabilistic prediction with applications, pp. 287-310. PMLR, 2023.
Wang, Y., Zou, D., Yi, J., Bailey, J., Ma. X., and Gu, Q. Im- proving adversarial robustness requires revisiting misclas- sified examples. In International conference on learning representations, 2019.
Luo, R., Bao, J., Zhou, Z., and Dang, C. Game-theoretic defenses for robust conformal prediction against ad- versarial attacks in medical imaging. arXiv preprint vrXiv:2411.04377 2024.
Yan, G., Romano, Y., and Weng, T.WW. Provably robust conformal prediction with improved efficiency. arXiv preprint arXiv:2404. /965/, 2024.
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. stat, 1050(9), 2017.
Zargarbashi, S. H., Akhondzadeh, M. S., and Bojchevski, A. Robust yet efficient conformal prediction sets. arXiv preprint arXiv:2407.09166 2024.

---

OPSA: Hard Quantile Framework for Adversarial Robustness
Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L.,, and Jordan, M. Theoretically principled trade-off between robustness and accuracy. In International conference machine learning, pp. 7472-7482. PMLR, 2019.
Zhang, H., Furon, T., Amsaleg, L.,, and Avrithis, Y. Deep neural network attacks and defense: The case of image classification. Multimedia Security, 1:41-75, 2022.
Zhang, J., Xu, X., Han, B., Niu, G., Cui, I.., Sugiyama, M., and Kankanhalli, M. Attacks which do not kill training make adversarial learning stronger. In International con- ference machine learning, pp. (1278-11287 PMLR, 2020.
Zhang, Z., Bao, J., Zhou, Z.. Colombo, N., Cheng, L., and Luo, R. Residual reweighted conformal prediction for graph neural networks. In Proceedings of the 41st Conference on Uncertainty in Artificial Intelligence (UAI), 2025. to appear.
12

---

OPSA: Hard Quantile Framework for Adversarial Robustness
A. Illustrative Example
To better understand the application the improved negative margin adversarial attacks within the conformal prediction framework, we use mee-iimnnsional prediction problem as an example. Please note that the parameter ranges here may differ from those in image-based scenarios, but this example sufficiently demonstrates the robustness of our algorithm. Consider the following example:
Assume four-class classification problem with classes K (1, The classifier f outputs logits as follows:
f1(2)
5 - 21
f2(2)
3 Ø1
f(z)-
fa(x)
2+ 0.5ar
f1(2)
1 0.2x1
The true class label is = 1. The goal is to design an adversarial perturbation such that, when added to the input x, causes the prediction set [(x; J) include more non-true classes, thereby increasing the model's uncertainty.
A.1. Original Input
Consider the original input x = 2. The logits are:
5-2
3+ = 5
f(2)
2 2 0.5 2 = 3
1 0.2 0.6
Based on these logits, the prediction set (2) might logits close to the highest logit.
(1,2), assuming a confidence threshold that includes classes with
A.2. Designing the Adversarial Perturbation
A.2.1. STEP DEFINE NEGATIVE MARGINS
For each class e [K], define the negative margin as:
f)( - ffo(x++
Specifically:
1(x+c) c)x - y
fx(x+c) - y= (3 (x1 + c)) - (5 (x1 + c)) = 2c +-+22 2xg - 2,
+() (5 (m1 + ()) = 1.5c + -3,
1x(x+6)-91 - 0.2( + +)) (5 (x) <)) -0.2r -0.2m1 4.
A.2.2. STEP 2: SEMPERATURE-SCALES NEGATIVE MARGIN
Introduce a temperature parameter T = 1 to scale the negative margins within the Sigmoid function:
[&(x + e) - o22 2m 2) (1.5cc 5c -3) + -022 0.2r1 4).
Mr + cf.v)
Σ KE[K]
T
13

---

OPSA: Hard Quantile Framework for Adversarial Robustness
A.2.3. STEP 3: OPTIMIZATION OBJECTIVE
Formulate the adversarial perturbation e* as:
arg (2 2x1 2) + x1 -3) + o 0.2x1 4))
|| $0.5
Given x =
-2e 2 +o - + 0.4 4),
Mr e; f,y) o2e + 2) o a 4.4).
A.2.4. STEP 4: EVALUATING Mr + e; f.v)
Evaluate My (x + G; f, y) for different values within the allowed range [-0.5, 0.5]:
e
2
1.5c
0.2c
4.4
a (2c + 2)
(1.5e 1)
1-0.ee 4.4)
Mr(x + +ef.
-0.5
-1.75
-4.3
0.7311
0.1521
0.0133
0.8965
-.25
1.5
-1.375
-4.35
0.8176
0.1839
0.0114
1.0129
0
2
-1
-4.4
0,8808
0,2689
0.0123
1.1619
0.25
2.5
-0.625
-4.45
0.9241
0.3446
0.0118
1.2805
0.3
2.6
-0.55
-4.46
0.9306
0.3685
0.0117
1.3108
0.4
2.8
-0.4
-4,48
0.9423
0.4013
0.0115
1.3541
0.5
3
-0.25
-4.5
0,9526
0.4378
0.0112
1.4016
Table 3. Evaluation of Mr + e; f. y) for different values.
A.2.5. STEP 5: SELECTING OPTIMAL
From the table, it is evident that M, (x + e; f. increases increases within the permissible range. Thus, the optimal perturbation is at the upper bound:
= 0.5.
A.2.6. STEP IMPACT ON CONFORMAL PREDICTION
Applying 0.5 the input x
x 2.0 2.5.
Calculate the perturbed logits:
5-2.5 25
3+2. 5.5
f(2.5)
2 2.5 = 3.25 2.5 0.5
Compute the negative margins:
225. =0,
M(2.5, 5.5, 1)2 = 5.5 25 0,
M(2.5,3.22 1), 3.25 2.5 0.75 0,
M (2.5, 0.5, 1)4 0.5 -2. <
14

---

OPSA: Hard Quantile Framework for Adversarial Robustness
Training Algorithm
Dataset
FGSM
PGD10
TRADES
MART
BETA-AT
OPSA-AT
CIFAR-10
65
655
321
445
469
1642
CIFAR-100
70
1398
342
742
1190
1689
mini-ImageNet
81
1766
452
881
1805
2024
Table 4. The time taken (in seconds) by each adversarial training model to complete one epoch of training on 100 batches, utilizing an
NVIDIA A100 80GB GPU.
Attacks
FGSM
PGD
TRADES
MART
BETA-AT
OPSA-AT
FGSM
18.5176
15.3600
17.2783
15.7824
18.5751
20.8580
Auto
1595.8604
2275.8080
2017.4780
2125.3660
669.0256
2543.7452
Square
272.9225
342.7212
336.6973
321.6511
107.9474
432.7322
PGD10
20.0746
18.3247
18.7224
19.7636
21.3279
21.5170
4G
27.1290
24.2051
24.0342
25,5516
29.1992
28,5025
APGD 100
37.0082
39.8945
35.6490
35.7549
33.0879
35.6442
BETA OPSA
32.7680
27.5900
27.1002
26.5268
29.0277
31.3223
25.9038
24.1116
25,4911
25.3905
27.2531
27.4187
Table 5. The attack execution time (in seconds) for various attack methods against different defense models, these results were derived from batches of tests conducted on 100 images randomly sampled from the CIFAR-100 test set. Note that Auto-Attack was executed with its default parameters, and the square black-box attack was configured to perform 1000 queries.
Thus, the negative margins are:
((3 (((.)) +o(-200) s 0.9526 + 0.6792 + 0.1192 = 1.7509.
Since both (2.55, 5.5, 1)2 0 and M2255, 1)3 0, the prediction set I(2.5) now includes classes 1, and 3, iee. r(2.5) 31.
Impact Analysis:
Before Perturbation: The prediction set [() = {1,2} included the true class and one non- true class 2. After Perturbation: The prediction set (2..5) (1,2,3) includes the true class 1 and two non-true classes 2 and 3.
This enlargement of the prediction demonstrates that the adversarial perturbation successfully increases the model's uncertainty by incorporating additional non-true classes.
B. Adversarial attack and adversarial training time
As detailed in this appendix, we present the training durations for various models and the computational overhead of different attack methods. Notably, our proposed model demonstrates scalability advantages its training time does not escalate substantially with increasing dataset complexity, maintaining efficient performance across diverse experimental configurations.
C. mini-ImageNet and figures
This appendix presents a comprehensive experimental evaluation on the mini- Imai-Im dataset. As evidenced in Table and Figure 3, the results clearly demonstrate both the superiority of our defense methodology and the remarkable efficacy of our attack strategy. Additionally, Figure visualizes our experimental results on the CIFAR-100 dataset.
15

---

OPSA: Hard Quantile Framework for Adversarial Robustness
Training Algorithm
Attacks
Indicator
FGSM
PGD
TRADES10
MART
BETA-AT
OPSA-AT
Coverage (%)
90.38 + 0.27
90.26 + 0.48
89.17 0.51
90.16 = 0.50
90.08 = 0.47
89.69 0.50
Clean
Size
43.52 +: 0.70
56.63 + 0.16
10.91 + 0.10
11.30 + 0.11
57.57 + 0.03
17.02 = 0.17
SSCV
0.38 + 0.01
0.90 + 0.20
0.09 + 0.01
0.07 + 0.01
0.01 + 0.00
0.02 + 0.01
Coverage (%)
90.49 + 0.47
91.15 0.47
90.49 + 0.50
89.79 0.48
89.90 + 0.46
89.14 0.50
FGSM
Size
49.49 +: 0.12
36.26 + 0.16
38.66 + 0.17
26.37 + 0.18
57.51 + 0.01
26.31 = 0.21
SSCV
0.53 0.18
0.08 + 0.10
0.10 = 0.00
0.15 = 0.06
0.01 = 0.00
0.04 = 0.04
Coverage (%)
89.97 0.50
91.15 0.44
90.89 0.47
89.48 0.49
89.64 0.49
88.80 = 0.49
PGD
Size
39.63
43.19 0.17
30.49 = 0.19
57.31 = 0.01
27.41 = 0.21
SSCV
0.90 0.20
0.10 0.01
0.06 + 0.02
0.23 + 0.12
0.01 = 0.00
0.06 + 0.04
Coverage (%)
89.92 0.49
91.15 0.48
90.89 0.48
89.48 0.48
89.71 0.49
88.83 0.51
PGD40
Size
39.63 0.17
43.19 0.18
30.48 = 0.20
57.31 + 0.01
27.39 = 0.21
SSCV
0.90 0.20
0.10 0.01
+ 0.03
= 0.12
=
0.06 1 0.04
Coverage (%)
90.36 0.48
91.07 0.44
90.70 0.47
90.29 0.49
89.90 0.47
0.49
BETA
Size
48.60 + 0.13
35.63 0.16
35.53 0.18
21.64 = 0.15
57.51 = 0.01
25.62 = 0.21
SSCV
+ 0.16
0.10 0.01
0.10 0.00
= 0.03
0.00 = 0.00
= 0.03
Coverage (%)
90.98 0.33
90.26 0.50
90.57 0.46
0.47
90.08 0.49
89.69 = 0.49
Square
Size
50.24 + 0.18
56.79 + 0.17
21.59 + 0.17
14.45 = 0.13
57.57 + 0.03
21.75 = 0.23
SSCV
0.57 + 0.01
0.90 + 0.20
+ 0.01
+
+
+ 0.01
Coverage (%)
90.05 0.35
90.26 0.47
90.05 0.49
90.05 0.49
90.08 0.00
89.69 = 0.48
APGD100
Size
51.37 +: 0.21
56.79 + 0.17
25.70 + 0.18
16.24 + 0.14
57.57 + 0.03
22.61 = 0.25
SSCV
0,50 + 0.01
0.90 = 0.19
0.08 = 0.02
0.08 = 0.01
0.00 = 0.00
0.06 = 0.01
Coverage (%)
90.16 + 0.48
91.28 + 0.44
90.31 + 0.33
89.56 = 0.48
89.92 - 0.49
89.96 = 0.51
OPSA10
Size
53.42 + 0.11
40.00 - 0.16
44.12 + 0.16
30.84 0.19
57.50 = 0.00
27.55 0.21
SSCV
0.90 + 0.30
0.10 + 0.01
+ 0.02
0.16 + 0.09
0.00 - 0.00
0.06 + 0.03
Table 6. Mean and Standard Deviation of Coverage. Size, and SSCV for mini- mini-ImageNet
D. Accuracy metrics on various datasets
This appendix presents the accuracy of various defense models under different attack methods across three datasets, with Tables 7-9 summarizing the results for CIFAR-10, CIFAR-I00, and nini-ImageNet respectively, Notably, while the OPSA attack method does not consistently yield the lowest accuracy, its focus targeting uncertainty leads significantly larger size metrics compared to alternative approaches.
E. Parameter analysis
Regarding the hyperparameters T2 and X, we refer to the detailed analysis in (Stutz et al., 2021), where these parameters were rigorously analyzed. For Ti its core function is to calibrate the sigmoid function approximate the Threshold Response (THR) method (Sadinle et al., 2019), enlargement of prediction Interval. As shown in Table 10, systematic variations of on CIFAR- 100 reveal critical threshold effect: randomly sampled values increase, the prediction set size expands progressively before plateauing at approximately 10.
16

---

OPSA: Hard Quantile Framework for Adversarial Robustness
Training Algorithm
Indicator
Attacks
FGSM
PGD
TRADES
MART
BETA-AT
OPSA-STIO
OPSA-ATIO
Clean
72.84%
48.48%
77.84%
81.69%
55.65%
81.33%
89.15%
FGSM
36.11%
44.32%
54.81%
43.35%
37.04%
64.75%
65.83%
PGD
29.60%
43.85%
50.76%
25.36%
29.06%
53.60%
57.27%
PGD40
29.60%
43.86%
50.75%
34.36%
28.94%
53.43%
57.13%
Accuracy
BETA 10
39.75%
49.64%
63.80%
50.09%
43.79%
70.57%
70.34%
Square
27.56%
43.09%
54.54%
40.29%
33.04%
58.26%
61.04%
APGD100
24.75%
42.16%
51.37%
33.16%
12.21%
52.05%
56.07%
Auto 24.04%
41.58%
48.70%
32.86%
26.65%
51.68%
55.30%
OPSA
30.21%
44.39%
51.58%
35.09%
19.48%
54.16%
57.91%
Table 7. accuracy for CIFAR-10
Training Algorithm
Indicator
Attacks
FGSM
1GD
TRADES10
MART
BETA-AT
OPSA-AT
Clean
43.65%
24.15%
51.55%
52.10%
6.07%
55.65%
FGSM
16.10%
20.69%
30.23%
30.18%
24.24%
37.04%
PGD
12.85%
20.49%
26.70%
25.35%
5.29%
29.06%
4G
12.81%
20.47%
26.67%
25.27%
4.93%
28.94%
Accuracy
BETA
17.94%
24.69%
38.22%
39.77%
25.94%
43.79%
Square
10.87%
20.05%
28.99%
28.68%
2.38%
33.04%
APGD 100
9.88%
19.46%
25.47%
24.32%
1.19%
27.73%
Auto
9.46%
18.73%
24.34%
23.12%
0.95%
26.65%
OPSA 10
13.14%
20.89%
27.60%
26.27%
5.78%
29.99%
Table accuracy for CIFAR-100
Training Algorithm
Indicator
Attacks
FGSM
PGD10
TRADES"0
MART
BETA-AT
OPSA-AT
Clean
18.39%
4.71%
43.67%
45.13%
1.56%
55.65%
FGSM
9.61%
20.47%
15.42%
25.39%
24.24%
37.04%
PGD
5.78%
16.72%
10.73%
20.70%
1.56%
29.06%
4G
5.70%
16.72%
10.65%
25.27%
1.56%
28.94%
Accuracy
BETA 10
10.86%
21.17%
19.14%
20.73%
1.56%
43.79%
Square
3.26%
2.19%
28.99%
22.14%
1.56%
33.04%
APGD 100
2.06%
2.08%
16.24%
24.32%
1.56%
27.73%
Auto
9.46%
18.73%
24.34%
23.12%
0.95%
26.65%
OPSA
6.07%
17.14%
11.15%
21.38%
1.56%
29.99%
Table 9, Accuracy for Mini-ImmgeNet
17

---

OPSA: Hard Quantile Framework for Adversarial Robustness
Ti
OPSA
OPSA-AT
Coverage (%)
89.62 0.35
0.001
Size
22.06 + 0.20
SSCV
0.03 + 0.01
Coverage (%)
89.76 + 0.34
0.1
Size
29.35 + 0.26
SSCV
0.03 0.01
Coverage (%)
89.95 + 0.35
Size
33.30 + 0.27
SSCV
0.03 0.01
Coverage (%)
89.88 + 0.34
10
Size
33.50 +: 0.25
SSCV
0.03 + 0.01
Coverage (%)
89.90 + 0.33
10
Size
33.50 = 0.26
SSCV
0.03 + 0.01
Coverage (%)
89.91 0.34
1000
Size
33.50 + 0.26
SSCV
0.03 + 0.01
Table 10. The effectiveness of OPSA attacks at different T1-
18

---

OPSA: Hard Quantile Framework for Adversarial Robustness
FGSM
(b) PGD
TRADES
(d) MART
(c)
BETA-ST
(d) OPSA-AT
Figure 3, Box-violin plots of mini ImageNet results under FGSM, PGD, TRADES, MART, BETA-ST ST. and OPSA-AT AT defense models
19

---

OPSA: Hard Quantile Framework for Adversarial Robustness
14
(a) FGSM
(b) PGD
TRADES
(d) MART
BETA-ST
(d) OPSA-AT
Figure 4. Box-violin plots of CIFAR100 results under FGSM, PGD, TRADES, MART, BETA ST. and OPSA-AT defense models
20


Reviews:
[{"review_id": "UuiP1JEonw", "reviewer": "Reviewer_2dn3", "summary": "The paper proposes a framework that integrates adversarial training with conformal prediction (CP) to enhance model robustness against adversarial attacks while maintaining reliable uncertainty estimates. It formulates adversarial training within the CP framework as a bi-level optimization problem, where an attacker seeks to maximize uncertainty while a defender aims to minimize it. The paper introduces a novel attack method that increases uncertainty without requiring coverage guarantees and develops a conformal training-based defense strategy that minimizes the size of prediction sets under adversarial perturbations. Experiments on CIFAR-10 and CIFAR-100 demonstrate that the proposed attack increases uncertainty more than existing methods, while the defense improves robustness against various adversarial attacks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We truly appreciate your recognition of the novelty and potential impact of our work. Your comments on clarifying the missing experimental details and time consumption are greatly appreciated. Below, we address your points and provide further explanations.\n\n**Experimental Setup**: In principle, any $\\ell_p$ norm could be used, but all experiments use the $\\ell_\\infty$ norm, with $\\epsilon=0.03$, following prior work ([A]). We will clarify this in the final version.\n\n**Additional Experiments**: \nWe included accuracy results for different defenses under various attacks (Tables 1 and 2), including three additional methods: APGD (100), AutoAttack (default), and Square Attack (black-box, 1000 queries). Our defense achieves the highest overall accuracy, while our attack produces the largest adversarial perturbations.\n\n# Table 1: Detailed Results for Attack Methods (Square, APGD$^{100}$, Auto, OPSA$^{10}$)\n\n## Square Attack [B]\n\n| Indicator | FGSM | PGD$^{10}$ | TRADES$^{10}$ | MART$^{10}$ | BETA-AT$^{10}$ | OPSA-AT$^{10}$ |\n|---------------|----------|------------|---------------|-------------|----------------|----------------|\n| Coverage (%)  | 88.98 ± 0.36 | 88.99 ± 0.35 | 90.86 ± 0.32 | 89.79 ± 0.34 | 90.65 ± 0.32 | 89.31 ± 0.34 |\n| Size          | 28.14 ± 0.20 | 33.03 ± 0.18 | 11.61 ± 0.09 | 11.60 ± 0.10 | 73.59 ± 0.27 | 13.81 ± 0.27 |\n| SSCV          | 0.05 ± 0.01 | 0.08 ± 0.01 | 0.04 ± 0.01 | 0.10 ± 0.03 | 0.46 ± 0.15 | 0.03 ± 0.01 |\n\n## APGD$^{100}$ Attack [C]\n\n| Indicator | FGSM | PGD$^{10}$ | TRADES$^{10}$ | MART$^{10}$ | BETA-AT$^{10}$ | OPSA-AT$^{10}$ |\n|---------------|----------|------------|---------------|-------------|----------------|----------------|\n| Coverage (%)  | 89.06 ± 0.36 | 88.99 ± 0.35 | 90.22 ± 0.32 | 89.11 ± 0.37 | 90.64 ± 0.33 | 90.04 ± 0.34 |\n| Size          | 32.00 ± 0.21 | 33.06 ± 0.18 | 13.54 ± 0.11 | 13.48 ± 0.12 | 73.93 ± 0.26 | 16.73 ± 0.20 |\n| SSCV          | 0.06 ± 0.01 | 0.08 ± 0.01 | 0.04 ± 0.01 | 0.10 ± 0.00 | 0.60 ± 0.17 | 0.09 ± 0.00 |\n\n## Auto Attack [C]\n\n| Indicator | FGSM | PGD$^{10}$ | TRADES$^{10}$ | MART$^{10}$ | BETA-AT$^{10}$ | OPSA-AT$^{10}$ |\n|---------------|----------|------------|---------------|-------------|----------------|----------------|\n| Coverage (%)  | 89.03 ± 0.36 | 88.99 ± 0.35 | 90.31 ± 0.33 | 89.11 ± 0.35 | 90.65 ± 0.31 | 90.50 ± 0.31 |\n| Size          | 32.04 ± 0.21 | 33.06 ± 0.18 | 12.69 ± 0.22 | 13.58 ± 0.12 | 74.00 ± 0.26 | 11.31 ± 0.22 |\n| SSCV          | 0.06 ± 0.01 | 0.08 ± 0.01 | 0.06 ± 0.01 | 0.10 ± 0.00 | 0.61 ± 0.18 | 0.09 ± 0.01 |\n\n## OPSA$^{10}$ Attack\n\n| Indicator | FGSM | PGD$^{10}$ | TRADES$^{10}$ | MART$^{10}$ | BETA-AT$^{10}$ | OPSA-AT$^{10}$ |\n|---------------|----------|------------|---------------|-------------|----------------|----------------|\n| Coverage (%)  | 90.75 ± 0.32 | 89.78 ± 0.34 | 90.31 ± 0.33 | 90.46 ± 0.32 | 89.29 ± 0.35 | 89.95 ± 0.35 |\n| Size          | 65.48 ± 0.22 | 39.14 ± 0.20 | 32.31 ± 0.19 | 35.28 ± 0.23 | 75.22 ± 0.20 | 33.30 ± 0.27 |\n| SSCV          | 0.10 ± 0.00 | 0.07 ± 0.01 | 0.08 ± 0.01 | 0.06 ± 0.01 | 0.10 ± 0.00 | 0.03 ± 0.01 |\n\n**Time Consumption**: See response to Reviewer 7J63.\n\n**Missing Evidence**: We acknowledge that our defense may not excel in all metrics under certain attacks. However, it prioritizes size minimization under equivalent coverage constraints, which is crucial for conformal prediction. On CIFAR-10, our model consistently achieves the smallest size, while on CIFAR-100, it remains competitive with state-of-the-art defenses.\nAutoAttack combines black-box and white-box strategies, and a key observation is that stronger defenses require longer attack times. This highlights two points:\n\n1. **Defense Strength**: The extended attack duration indicates our defense's resilience, as adversaries need more effort to breach it.\n\n2. **Training Time**: Longer training is necessary due to the defense's complexity and optimization goals, enhancing its robustness.\n\nWe have conducted additional experiments on the ImageNet-Mini dataset, where the model sizes of both our attack and defense methods consistently outperform baseline approaches. Due to page limits, we summarize that our defense achieves state-of-the-art performance across all adversarial attack scenarios. For detailed results and analysis, please refer to our response to Reviewer uZo5.\n\nReferences:\n\n[A] Robey, Alexander, $et$ $al$. ''Adversarial Training Should Be Cast as a Non-Zero-Sum Game.\" The Twelfth International Conference on Learning Representations. ICLR (2024). (from paper)\n\n[B] Maksym, Andriushchenko, $et$ $al$. ''Square Attack: a query-efficient black-box adversarial attack via random search.\" https://arxiv.org/abs/1912.00049.\n\n[C] Francesco, Croce, $et$ $al$. ''Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks.\" https://arxiv.org/abs/2003.01690.", "claims_and_evidence": "Claim 1: The proposed method \"minimizes the size of the prediction sets under adversarial perturbations while maintaining high coverage probabilities: evidence given in the experimental section.\"\n\nClaim 2: \"The defensive model significantly enhances robustness against various adversarial attacks and maintains reliable prediction intervals\": NO EVIDENCE GIVEN\n\nClaim 3: The \"findings highlight the effectiveness of integrating adversarial training with conformal prediction to develop trustworthy and\nresilient deep learning models for safety-critical domains\": Only partial evidence is given in the experimental section.", "methods_and_evaluation_criteria": "The methods and evaluation criteria are reasonable and required. \nHowever, some essential evaluations and methods are missing. \n\nMethods missing: APGD and AutoAttack; these are stronger attacks, and it would be interesting to see if the proposed adversarial training method still performs reasonably against these stronger attacks. \n\nEvaluations missing: It is unclear what the performance of the model is in terms of accuracy or what the performance of the attack is in terms of attack efficacy.", "theoretical_claims": "The theoritical claims in the paper seem to be reasonable and are proved.", "experimental_designs_or_analyses": "The experimental design is not clear; some key information required to truly understand the experiments is missing.\nThis information is:\n1. The l-p norm used, while the method section says that any l-p norm can be used, the attack norm used is not specified in the evaluations.\n2. The epsilon value used.", "relation_to_broader_scientific_literature": "The proposed idea in the paper is certainly very interesting and relevant to a broader scientific community.", "essential_references_not_discussed": "The related works that I am aware of are cited in the paper already.", "other_strengths_and_weaknesses": "Strengths:\n\nTo the best of my knowledge, the idea is novel, and the work has the potential to be very impactful and to pave the way towards safety-criticality.\n\n\nWeakness:\nCertain claims made are not proved in the paper, as written above in the \"claims\" section. \nWhile the work has the potential to pave the way towards safety-criticality, this work itself does not work towards safety-criticality. \nNo safety-critical application has been pursued in this paper. Therefore the lines 036-040 are an overclaim in my opinion. \n\nSome critical evaluations are missing in terms of performance against the current SotA adversarial attacks like APGD and AutoAttack. \n\nSome metrics, such as model performance in terms of accuracy under attack, are also missing.\n\nLastly, it is unclear what the increased time requirement for the adversarial training and the adversarial attack is in comparison to the other known adversarial attacks and training methods.", "other_comments_or_suggestions": "This work has the potential to be very impactful.\nHowever, as mentioned in the responses above, some key components are missing in terms of evaluations, metrics, and experimental details. \nIncluding these would be very helpful.", "questions_for_authors": "Knowing the experimental details of the proposed attack would be very helpful. \nAdditionally, knowing the time taken for the proposed attack and defense method (training) in comparison to the others attacks and training methods (respectively) would also be very helpful.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AzcDyT4yld", "reviewer": "Reviewer_uZo5", "summary": "This paper introduces a novel approach that integrates Conformal Prediction (CP) with Adversarial Training (AT) to enhance the adversarial robustness of deep learning models. The authors frame adversarial robustness as a bi-level optimization problem, where an attacker maximizes the uncertainty by enlarging the CP prediction set, while a defender minimizes this uncertainty while maintaining statistical coverage guarantees.\n\nKey contributions of the paper include:\n\n- Optimal Size Attack (OPSA) – A differentiable adversarial attack that increases the CP prediction set size to introduce uncertainty.\n- Adversarially Robust Conformal Prediction (OPSA-AT) – A defense mechanism that integrates CP with adversarial training to maintain small and reliable prediction sets.\n- Experimental validation on CIFAR-10 and CIFAR-100 – Showing OPSA outperforms baseline attacks like FGSM, PGD, and BETA in increasing uncertainty, while OPSA-AT provides superior robustness compared to other adversarial training methods.\n\nThe results demonstrate that the proposed approach provides a stronger defense against adversarial attacks while preserving prediction set efficiency, making it relevant for safety-critical applications.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for recognizing the novelty, strong theoretical foundation, comprehensive experiments, and practical implications of our work. We appreciate your valuable suggestions and feedback. Below, we address your concerns.\n\n**Time Consumption**: See response to Reviewer 7J63.\n\n**Experimental Diversity**: Our paper is theoretically supported and has demonstrated efficiency in experiments, confirming the effectiveness of our method. However, we agree that more experimental validation is beneficial. To strengthen our evaluation, we conducted additional experiments on the ImageNetMini dataset (80% training, 20% split for calibration and testing) using a ResNet50 trained for five epochs. Due to time constraints, we tested our top defenses (TRADES, MART, OPSA-AT) against strong attacks (Clean, FGSM, PGD, OPSA). As shown in Table 2, our attack and defense achieve state-of-the-art performance on Size (the most reliable metric) and deliver near-best results on SSCV, outperforming all baselines.\n\n# Table 2: Mean and Standard Deviation of Coverage, Size, and SSCV for imagemini\n\n| **Attacks** | **Indicator** | **Training Algorithm** | | |\n|-------------|---------------|------------------------|----------------------|----------------------|\n| | | **TRADES$^5$** | **MART$^5$** | **OPSA-AT$^5$** |\n| Clean | Coverage (%) | 89.64 ± 0.50 | 89.69 ± 0.50 | 89.69 ± 0.48 |\n| | Size | 18.55 ± 0.11 | 20.62 ± 0.12 | 17.01 ± 0.18 |\n| | SSCV | 0.08 ± 0.01 | 0.07 ± 0.01 | 0.10 ± 0.01 |\n| FGSM | Coverage (%) | 89.48 ± 0.50 | 90.34 ± 0.50 | 89.51 ± 0.49 |\n| | Size | 39.01 ± 0.12 | 48.88 ± 0.11 | 30.41 ± 0.23 |\n| | SSCV | 0.07 ± 0.11 | 0.10 ± 0.00 | 0.08 ± 0.02 |\n| PGD$^{10}$ | Coverage (%) | 89.38 ± 0.49 | 90.44 ± 0.46 | 89.66 ± 0.49 |\n| | Size | 38.11 ± 0.12 | 50.35 ± 0.10 | 34.88 ± 0.24 |\n| | SSCV | 0.05 ± 0.01 | 0.01 ± 0.00 | 0.14 ± 0.03 |\n| OPSA$^{10}$ | Coverage (%) | 89.71 ± 0.48 | 90.47 ± 0.46 | 89.95 ± 0.46 |\n| | Size | 40.92 ± 0.12 | 50.52 ± 0.09 | 35.63 ± 0.02 |\n| | SSCV | 0.23 ± 0.24 | 0.10 ± 0.00 | 0.11 ± 0.04 |\n\n\n**Additional Attacks**: We further tested our defense against APGD, Black-box (Square), and AutoAttack—an advanced ensemble of black-box and white-box attacks. As outlined in our response to Reviewer 2dn3 (Table 1), our method consistently demonstrates robustness across these attack scenarios. Notably, all attacks generated smaller prediction set sizes than OPSA, while our defense remained effective.\n\n**Ablation Studies**: Regarding the hyperparameters $\\lambda$ and $T_{2}$, we refer to the detailed analysis in [A], where these parameters were rigorously analyzed. For $T_{1}$, its core function is to calibrate the sigmoid function to approximate the Threshold Response (THR) method [B], enlargement of prediction Interval. As shown in Table 3, systematic variations of $T_{1}$ on CIFAR-100 reveal a critical threshold effect: as randomly sampled $T_{1}$ values increase, the prediction set size expands progressively before plateauing at approximately 10.\n\n# Table 3: The effectiveness of OPSA attacks at different T₁ values\n\n| **$T_{1}$** | **OPSA** | **OPSA-AT$^{10}$** |\n|--------|----------|----------------|\n| 0.001  | Coverage (%) | 89.62 ± 0.35 |\n|        | Size        | 22.06 ± 0.20 |\n|        | SSCV        | 0.03 ± 0.01  |\n| 0.1    | Coverage (%) | 89.76 ± 0.34 |\n|        | Size        | 29.35 ± 0.26 |\n|        | SSCV        | 0.03 ± 0.01  |\n| 1      | Coverage (%) | 89.95 ± 0.35 |\n|        | Size        | 33.30 ± 0.27 |\n|        | SSCV        | 0.03 ± 0.01  |\n| 10     | Coverage (%) | 89.88 ± 0.34 |\n|        | Size        | 33.50 ± 0.25 |\n|        | SSCV        | 0.03 ± 0.01  |\n| 100    | Coverage (%) | 89.90 ± 0.33 |\n|        | Size        | 33.50 ± 0.26 |\n|        | SSCV        | 0.03 ± 0.01  |\n| 1000   | Coverage (%) | 89.91 ± 0.34 |\n|        | Size        | 33.50 ± 0.26 |\n|        | SSCV        | 0.03 ± 0.01  |\n\n**Others**: We have incorporated discussions of the following two papers [C][D] into our analysis to provide a more comprehensive theoretical foundation.\n\nReference:\n\n[A] Stutz, David, $et$ $al$. '' Learning optimal conformal classifiers.\" arXiv preprint arXiv:2110.09192. (2021).\n\n[B] Sadinle, Mauricio, $et$ $al$. ''Least ambiguous set-valued classifiers with bounded error levels.\" Journal of the American Statistical Association (2024).\n\n[C] Kavya, Gupta, $et$ $al$. ''CertViT: Certified Robustness of Pre-Trained Vision Transformers.\" https://arxiv.org/abs/2302.10287. (2023).\n\n[D] Kavya, Gupta, $et$ $al$. ''Shrink \\& Cert: Bi-level Optimization for Certified Robustness.\" The Second Workshop on New Frontiers in Adversarial Machine Learning. (2023).", "claims_and_evidence": "The claims in the paper are largely well-supported by experiments and theoretical discussions.\n\n- The paper claims that OPSA induces greater uncertainty than existing adversarial attacks. This is well-supported by quantitative results, showing that OPSA consistently produces larger prediction sets compared to PGD and BETA.\n- The authors argue that OPSA-AT reduces uncertainty while maintaining coverage guarantees. Their experimental results show that OPSA-AT achieves the smallest prediction sets while keeping classification coverage above 1-α, providing clear evidence of effectiveness.\n- The bi-level optimization formulation is justified with mathematical derivations and an algorithmic implementation, making the claims about the framework’s validity credible.", "methods_and_evaluation_criteria": "- The paper evaluates robustness using coverage probability, prediction set size, and size-stratified coverage violation (SSCV). These are relevant metrics for assessing adversarial robustness in CP.\n- The choice of CIFAR-10 and CIFAR-100 is reasonable for a first demonstration, as they are standard benchmarks in adversarial robustness research.\n- Only ResNet-34 has been used.\n- The comparisons against FGSM, PGD, BETA, TRADES, and MART are comprehensive and provide a solid baseline for evaluating OPSA and OPSA-AT.\n\nA potential improvement would be to test the methods on other datasets and larger networks where adversarial robustness is crucial.", "theoretical_claims": "The theoretical claims in the paper appear to be correct and well-founded.\n\n- The bi-level optimization framework is mathematically well-posed and follows existing formulations from adversarial training literature.\n- The adversarial attack formulation (maximizing prediction set size) is a logical extension of conformal prediction principles.\n- The proofs in Appendix A and C support the convergence and correctness of the adversarial robustness guarantees.\n\nOne aspect that could be clarified is whether the theoretical guarantees hold under more general adversarial threat models (e.g., different norm constraints or black-box attacks).", "experimental_designs_or_analyses": "The experimental setup is well-structured and follows best practices in adversarial robustness evaluation.\n\n- The attack and defense methods are evaluated across multiple adversarial perturbation levels.\n- The results include statistical confidence intervals (mean ± standard deviation), which adds credibility.\n-The boxplot visualizations in Appendix D provide useful insights into attack and defense effectiveness.\n\nHowever, there are some areas for improvement:\n\n- Ablation Studies – It would be useful to show how OPSA-AT performs under different hyperparameters (e.g., $\\lambda$, T1, T2) to understand its sensitivity.\n- Computational Overhead – The paper does not discuss how much extra training/inference time OPSA-AT introduces.", "relation_to_broader_scientific_literature": "The paper builds upon and extends three key areas of machine learning research:\n\nAdversarial Robustness\n\n- Traditional adversarial training methods (e.g., Madry et al., 2017; TRADES (Zhang et al., 2019); MART (Wang et al., 2019)) focus on minimizing classification errors but do not address uncertainty.\n- This work extends adversarial training to the uncertainty estimation domain by integrating CP, which is a novel perspective.\n\nConformal Prediction\n\n- CP methods (e.g., Vovk et al., 2005; Ghosh et al., 2023) provide distribution-free coverage guarantees but struggle under adversarial conditions.\n- This paper proposes a way to defend CP models against adversarial perturbations, which is a valuable contribution.\n\nBi-Level Optimization in Machine Learning\n\n- The bi-level optimization framework aligns with recent advances in min-max adversarial training (Nouiehed et al., 2019; Robey et al., 2024).\n- Unlike previous work, the authors frame prediction set size minimization as a key objective, which is a fresh take on the problem.", "essential_references_not_discussed": "- CertViT: A method for achieving certified robustness in pre-trained Vision Transformers, presented at the ICML 2023 Workshop. It employs the Douglas-Rachford splitting algorithm to ensure both robustness and sparsity simultaneously.\n\n- Shrink & Cert: A bi-level optimization approach for enhancing certified robustness while maintaining sparsity constraints, also presented at the ICML 2023 Workshop.", "other_strengths_and_weaknesses": "Strengths:\n- Novelty – The integration of CP with adversarial training is original and impactful.\n- Strong theoretical foundation – The framework is mathematically sound with rigorous proofs.\n- Comprehensive experiments – The study provides strong empirical validation using diverse attack/defense comparisons.\n- Practical implications – The approach is useful for safety-critical applications where robustness matters.\n\nWeaknesses:\n- Limited dataset diversity – Experiments are only on CIFAR-10/100, which may not generalize to real-world settings.\n- Limited networks - Experiments are only on ResNet-34 network.\n- No evaluation of computational efficiency – OPSA-AT might be computationally expensive, but the paper does not discuss it.\n- No discussion on transferability – How well does the defense hold up against black-box attacks?", "other_comments_or_suggestions": "- Consider additional datasets (e.g., ImageNet) to show broader applicability.\n- Consider additional networks. Possibly larger networks. If large networks are not feasible computationally then mention that.\n- Analyze the time complexity of OPSA-AT to ensure practical feasibility.\n- Investigate black-box attacks to check whether OPSA-AT generalizes to unseen adversarial strategies.", "questions_for_authors": "- How does OPSA-AT perform on larger-scale datasets like ImageNet? This would provide a better sense of real-world applicability.\n- How does OPSA-AT perform on larger-networks like ResNet-50, ViT, etc.\n- What is the computational overhead of OPSA-AT compared to standard adversarial training? Understanding the trade-off between robustness and efficiency is crucial.\n- Does OPSA-AT maintain robustness against black-box or transfer attacks? The current experiments focus on white-box settings, so it’s unclear if the defense generalizes.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ckmi8R43w3", "reviewer": "Reviewer_7J63", "summary": "The paper proposed a conformal prediction based adversarial attack and training method.  To enable computationally tractable implementations, the authors propose a smoothed surrogate loss. The attack and defense methods are tested on CIFAR10 and CIFAR100.\n\n\n## update after rebuttal\nI believe the authors have sufficiently addressed the comments and concerns raised and so I have increased my score to a 4: Accept.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the recognition of the novelty and clarity of our work. Thanks for your valuable feedback on addressing the effectiveness problem. We would like to address your concern.\n\n**Time Consumption**:\nTable 4 reports the time (in seconds) per epoch for each adversarial training model on 100 batches. Table 5 presents the execution time (in seconds) of attacks, including three additional methods: APGD (100), AutoAttack (default), and Square Attack (black-box, 1000 queries). These results were derived from 8 batches of tests conducted on 100 images randomly sampled from the CIFAR-100 test set.\n\n# Table 4: Adversarial Training Model Time\n\n| Dataset | FGSM | PGD$^{10}$ | TRADES$^{10}$ | MART$^{10}$ | BETA-AT$^{10}$ | OPSA-AT$^{10}$ |\n|-------------|----------|------------|---------------|-------------|----------------|----------------|\n| CIFAR-10    | 65       | 655        | 321           | 445         | 469            | 1642           |\n| CIFAR-100   | 70       | 1398       | 342           | 742         | 1190           | 1689           |\n| IMAGENETmini| 81       | 1766       | 452           | 881         | 1805           | 2024           |\n\n\n# Table 5: Attack Execution Time\n\n| Attacks | FGSM | PGD$^{10}$ | TRADES$^{10}$ | MART$^{10}$ | BETA-AT$^{10}$ | OPSA-AT$^{10}$ |\n|-----------|----------|------------|---------------|-------------|----------------|----------------|\n| FGSM        | 18.5176  | 15.3600    | 17.2783       | 15.7824     | 18.5751        | 20.8580        |\n| Auto        | 1595.8604| 2275.8080  | 2017.4780     | 2125.3660   | 669.0256       | 2543.7452      |\n| Square      | 272.9225 | 342.7212   | 336.6973      | 321.6511    | 107.9474       | 432.7322       |\n| PGD$^{10}$      | 20.0746  | 18.3247    | 18.7224       | 19.7636     | 21.3279        | 21.5170        |\n| PGD$^{40}$      | 27.1290  | 24.2051    | 24.0342       | 25.5516     | 29.1992        | 28.5025        |\n| APGD$^{100}$   | 37.0082  | 39.8945    | 35.6490       | 35.7549     | 33.0879        | 35.6442        |\n| BETA$^{10}$     | 32.7680  | 27.5900    | 27.1002       | 26.5268     | 29.0277        | 31.3223        |\n| OPSA$^{10}$     | 25.9038  | 24.1116    | 25.4911       | 25.3905     | 27.2531        | 27.4187        |\n\n\nOur defense requires more computation than others, but experiments show its complexity does not scale significantly with dataset or model size. In attacks, our method outperforms FGSM and PGD$^{10}$ in speed, runs slightly faster than PGD$^{40}$ and APGD, and is 100 times faster than AutoAttack. Notably, it consistently generates the largest adversarial perturbations, inducing the highest model uncertainty across all defenses.\n\n**Formula problem**:  Thank you for noticing that there were no commas in Equations 16 and 22; we have now added them.", "claims_and_evidence": "The experiments largely support the claims made by the authors.  It would be helpful if the authors compared the computational cost of the various attacks and defenses.", "methods_and_evaluation_criteria": "The method and evaluation criteria make sense for the application.", "theoretical_claims": "I did not check the proofs.", "experimental_designs_or_analyses": "I checked the CIFAR10 and CIFAR100 experiments and found no issues.", "relation_to_broader_scientific_literature": "The  adversarially robust conformal prediction method proposed here naturally integrates adversarial training.  In contrast with the recent approach of Liu et al. (2024), a key novelty of the present work is the attack-agnostic manner in which it frames the problem.", "essential_references_not_discussed": "I am not aware of any key missing references.", "other_strengths_and_weaknesses": "I find the core idea to be original and the paper to be well written.", "other_comments_or_suggestions": "Equations 16 and 22 are missing a comma at the end.", "questions_for_authors": "1) On page 8 the authors write “Notably, OPSA surpasses PGD, even when PGD is given four times more iterations.” It would be helpful here, and for the other of the comparisons, if the authors could comment on the computational cost of their attack and defense methods, as compared to the other methods in tables 1 and 2.  Without this information, it is difficult to truly compare the effectiveness of the new attack/defense with the competing methods.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TWYD01CUh5", "reviewer": "Reviewer_jGyb", "summary": "This paper studies adversarial robustness of Conformal Prediction. Specifically, this paper develops an attack method that does not require coverage guarantees and integrates it with a conformal training-based defense strategy by minimizing the size of the prediction sets under adversarial perturbations while maintaining high coverage probabilities. Experimental evaluations on the CIFAR10 and CIFAR-100 datasets show that the attack method induces greater uncertainty compared to baseline approaches, while the defensive model significantly enhances robustness against various adversarial attacks for most cases.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the recognition of our paper’s clarity and thank the reviewer for their careful attention to detail. While we share many points of agreement, the main misunderstanding lies in the interpretation of outcomes, which we would like to clarify.\n\n**Tabular Data Interpretation**:\n        \nWe will expand the text explanations to provide more detail on Table 1 and Table 2 from the paper. Both tables follow a similar structure, with each row representing a specific metric for different defense models under a particular attack on the test set. To compare defense methods, focus on each row, where coverage is approximately the same (ideally around 90%), with slight variations due to sampling differences. For the 'Size' metric, smaller values are better, and the same applies to SSCV—lower values are preferred. To compare attack methods, look at each column. Unlike the defense metrics, a larger size and SSCV indicate a more effective attack method.\n\n**Experimental Outcome Interpretation**:\n\nFirst, we must clarify that we utilized the **complete** CIFAR-10 and CIFAR-100 datasets, with training sets for model training and test sets split (20% for calibration, 80% for testing).\n\nSecond, in conformal prediction, a key research question is how to minimize uncertainty (prediction set size) while maintaining equivalent coverage probability. Indeed, Coverage and SSCV results show some instability in measuring attacks, with clean images exhibiting higher SSCV than attacks. As with the accuracies ([A, B, C]), this outcome is expected. The inherent trade-off between adversarial robustness and SSCV naturally leads to this phenomenon. Similar to TRADES and MART, SSCV is not maximized on the clean dataset because the training process specifically accounts for clean data.  \n\nThird, in terms of defense, OPSA-AT achieves the best set size at 3/6 on CIFAR-100 but achieves 5/6 on CIFAR-10. Even when OPSA-AT is not the best on some attacks, it achieves the second-best performance. No other defense reaches such good performance on CIFAR-10 and CIFAR-100. Considering overall performance, we still believe our defense is effective. To further support our arguments, we carry out experiments on ImageNetMini, where OPSA-AT outperforms other defenses on Size (see Table 2 of response to Reviewer uZo5)\n\n**Formula Issues**:\n\nTypo in Equation 13:  We apologize for incorrectly changing \"$\\delta$\" to \"p\" in Equation 13 and have corrected this error.\n\nDefinition issue of Equation 16: Equation 16 is central to conformal training, constructed as a differentiable loss function to approximate a hard threshold. It is similar to Equation 13 but differs in that, for defensive purposes, we need to consider coverage, so Equation 16 subtracts the calculated tau value over B_cal.\n\nClarification for Equation 19: $\\mathcal{B}_{\\pi(1)}$ is simply the mini-batch that appears in the first position after a permutation $\\pi$ is applied to the indices of the mini-batches. The exchangeability property ensures that the statistical properties of the mini-batches remain unchanged under any reordering (as in [D]).\n\n\n        \n[A] Zhang, Hongyang, $et$ $al$. ''Theoretically principled trade-off between robustness and accuracy.\" International conference on machine learning. PMLR (2019).\n\n[B] Dobriban, Edgar, $et$ $al$. ''Provable tradeoffs in adversarially robust classification.\" IEEE Transactions on Information Theory (2023).\n\n[C] Robey, Alexander, $et$ $al$. ''Adversarial Training Should Be Cast as a Non-Zero-Sum Game.\" The Twelfth International Conference on Learning Representations. ICLR (2024).\n\n[D] Li, Yangyi, et al. \"Data Poisoning Attacks against Conformal Prediction.\" International Conference on Machine Learning. PMLR (2024).", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "The proof is not checked.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "The proposed adversarial training method is tailored for conformal prediction.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "Strength:\n1.\tThe paper is well-written and easy to follow.\nWeaknesses:\n1.\tThe effectiveness of the proposed adversarial training method is not clearly evident in the experimental results. Regarding attacks, the proposed attack method indeed achieves the largest set size as it’s designed to maximize the set size. However, in terms of defense, in cifar100, the proposed OPSA-AT only achieves the best set size against 50% attacks (3 out 6). Note that the authors only experiment on cifar10 and cifar100 datasets (50% datasets) and OPSA-AT is trained to maintain a small set size against attacks, showing that the effectiveness of the proposed method is not adequate.  Additionally, in terms of another metric, SSCV, the dominance of the proposed method is also not clear on cifar10 dataset.", "other_comments_or_suggestions": "1.\tIn eqn 13, there is no $\\delta$ but $\\delta $ is mentioned below.\n2.\tThe definition of eqn 16 is not clear. It denotes a scaler, or a vector, or a set? $k$ is mentioned above but not appears in the equation.\n3.\tIn Eqn 19, what is $B_{\\pi(1)}$ ?\n4.\tTable 1 and table 2 are really hard to understand and parse. It is recommended to enhance the presentation to make key information clear.", "questions_for_authors": "1.\tIn Table 1,2, why is the most effective attack method to increase SSCV the clean images themselves? It seems that the designed attack methods are unnecessary and meaningless as clean image is the best one.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper addresses the problem of adversarial training in a novel framework including conformal prediction (CP). Adversarial training and uncertainty estimation are formulated as bi-level optimization problem. Additionally, the paper proposes a novel attack method that increases uncertainty. Experiments are conducted on CIFAR-10 and CIFAR-100.
Reviewers agree that the paper proposes an interesting framework with a medium to strong technical contribution. The only weakness is the method evaluation which purely considers very low resolution datasets and a single architecture, which is different from the preact-resnet, which would be the default on the RobustBench benchmark.The AC encourages the authors to include experiments on PreAct Resnet in the final version.
Decision: Accept (poster)