Game-Theoretic Defenses for Adversarially Robust Conformal Prediction
Authors: Rui Luo
Jie Bao
Suqun Cao
Chuangyin Dang
Zhixin Zhou
Venue: Accepted by TMLR ()

Abstract: Adversarial attacks pose major challenges to the reliability of deep learning models in safety-critical domains such as medical imaging and autonomous driving. In such high-stakes applications, providing reliable uncertainty quantification alongside adversarial robustness becomes crucial for safe deployment. Although conformal prediction can provide certain guarantees for model performance under such conditions, unknown attacks may violate the exchangeability assumption, resulting in the loss of coverage guarantees or excessively large predictive uncertainty. To address this, we propose a synergistic framework that integrates conformal prediction with game-theoretic defense strategies by modeling the adversarial interaction as a discrete, zero-sum game between attacker and defender. Our framework yields a Nash Equilibrium defense strategy, which we prove maintains valid coverage while minimizing the worst-case prediction set size against an optimal adversary operating within the defined attack space. Experimental results on CIFAR-10, CIFAR-100, and ImageNet further demonstrate that, under Nash equilibrium, defense models within our framework achieve valid coverage and minimal prediction set size. By bridging adversarial robustness and uncertainty quantification from a game-theoretic perspective, this work provides a verifiable defense paradigm for deploying safety-critical deep learning systems, particularly when adversarial distributions are unknown or dynamically evolving but contained within a known attack space.

Published in Transactions on Machine Learning Research (01/2026)
Defenses for Adversarially Robust Conformal Prediction
Rui Luo
ruluuo@cityreddu..k
City University of Hong Kong
Jie Bao
148610389788qq.com
Huoigin Institute of Technology
Suqun Cao
caosuqun@hyit.cdee edu..cm
Huaögin Institute of Technology
Chuangyin Dang
meedang@cityu. edu. hk
City University of Hong Kong
Zhixin Zhou
zahew@alphaberiioo.om com com
Alpha Benilo Research
Reviewed OpenReview:
Abstract
Adversarial attacks pose major challenges to the reliability of deep learning models in safety-critical domains such as medical imaging and autonomous driving. In such high- stakes applications, providing reliable uncertainty quantification alongside adversarial ro- bustness becomes crucial for safe deployment. Although conformal prediction can provide certain guarantees for model performance under such conditions, unknown attacks may violate the exchangeability assumption, resulting in the loss of coverage guarantees or excessively large predictive uncertainty. To address this, propose synergistic frame- work that integrates conformal prediction with game-theoretic defense strategies by mod- eling the adversarial interaction as a discrete, zero-sum game between attacker and de- fonder. Our framework yields a Nash Equilibrium defense strategy, which we prove main- lains valid coverage while minimizing the worst-case prediction set size against an opti- mal adversary operating within the defined attack space. Experimental results on CIFAR- 10, CIFAR-100, and ImageNet further demonstrate that, under Nash equilibrium, defense models within our framework achieve valid coverage and minimal prediction set size. By bridging adversarial robustness and uncertainty quantification from game-theoretic per- spective, this work provides a verifiable defense paradigm for deploying safety-critical deep learning systems, particularly when adversarial distributions are unknown or dynamically evolving but contained within a known attack space. The Python code is available at hwwoon///bbbbbbbb Gamne Theoretic-CP.
1 Introduction
The reliability of deep learning tochnologies (Chen et al., 2025a is facing systematic challenges posed by adversarial attacks. Such attacks, through carefully crafted subtle input perturbations, can cause medical imaging analysis models to produce fatal misdiagnoses (Ma et al. 2021 or induce autonomous driving sys- tems (Badjie et al. 2024 to make dangerous decisions (Chen et al. 2025b), with potential risks that have transcended traditional algorithmic fault tolerance boundarics. Although adversarial training techniques have euhanced model robustness by ivolctively generating allack samples and have mitigated malicious input threats to some extent, their defense mechanisms still exhibit significant uncertainty and cannot pro- vide verifiable robustness guarantecs. Against this backdrop, Conformal Prediction (CP) has emerged as a promising paradigm, with its core advantages lying in distribution-fret properties and robust uncertainty quantification capabilities. This data-driven approach generates statistically rigorous confidence intervals

---

Published in Transactions on Machine Learning Research (01/2026)
and precisely calibrated probability estimates, demonstrating excellent performance in both regression and classification tasks, thereby providing novel solution for constructing trustworthy deep learning systems.
However, the application of Conformal Prediction (CP) methods faces fundamental limitations, as their effec- tiveness strictly depends on the exchangeability assumption between training and test sets. This assumption is often difficult to satisfy real-world application scenarios, particularly under adversarial attack condi- tions where attackers may employ unknown attack strategies maliciously perturb input data. Existing rescarch efforts primarily attempt to mitigate the impact of adversarial samples test data distribution by reconstructing non-conformity scoring functions, but these approaches still face significant methodolog- ical bottlencoks. First, the strategy of expanding prediction set sizes to ensure statistical validity directly leads to a significant reduction in the decision utility of prediction results. Second, existing improvement schemes are mostly optimized for specific attack paradigms and struggle to guarantee coverage generaliz- ability when confronting unknown attack vectors. Furthermore, the engineering implementation of complex non-conformity scoring functions presents technical obstacles, and cxisting methods exhibit systematic de- ficiencies in both computational complexity and cross-scenario generalization capabilities, which severely constrains the practical application value of CP methods in adversarial environments.
Therefore, developing universal conformal prediction framework is essential. Such a framework needs to address three critical challenges: first, establishing a theoretical guarantee system that ensures arbitrary standard non-conformity scoring functions maintain preset coverage probabilities in open-world scenarios; second, optimizing uncertainty quantification strategies to minimize prediction set sizes while preserving pre- diction reliability; third, enabling the framework simultaneously achieve effective coverage maintenance and prediction set size minimization when confronting arbitrary unknown attack vectors in adversarial vironments.
Schematic Payoff Matrix (on Evaluation Set)
PAR
Defender
0.50
1.00
0.20
Defense Model
Z
Defender
0.20
0.50
1.00
X
Defender
1.00
0.20
0.50
B
Attacee
Astacker Y Attack Method
Attacker
Adversarially attacks
Construct payoff matrix
Schematic Nash Equilibrium Strategy
Defender's Nash Strategy
Attacker's Nash Strategy
Probability
0.33
0.33
0.33
0.33
0.33
Dofender A
Defender B
Defender C
Anacke X
Attacker Y
Attacker
Solve for the Nash equilibrium
Figure 1: Address adversarial attacks in conformal prediction by constructing game-theoretic model.
2

---

Published in Transactions on Machine Learning Research (01/2026)
2 Preliminary and Problem Setup
Consider a classifier f: X - IRx which maps an imput image x E 2' to a probability vector f(x) E RR Here, X' = [0, 1] represents the image space, where pixel values are normalized and d denotes the image dimension. The integer K signifies the total number of classes, and y E [K] = {I...., K} denotes the ground truth label for the input x.
2.1 Conformal Prediction
We first train the classifier f on the training set Drrain and then compute nonconformity scores on the calibration set Deal- On the test Deest for each input x, we construct prediction set C(x) [K] such that the true label y contained in C(x) with probability at least 1 a, where (0, 1) user-specified significance level. Formally, we guarantee
P(Y ((x))21-0
(1)
In conformal prediction, different nonconformity score functions 8y(x, v) can lead to varying outcomes. resulting in prediction sets of different sizes. For instance, choice for the nonconformity score is s(x, k) 1 - [x),, where k is the class label. We compute the nonconformity scores s(x, k) in Deal- Subsequently, the conformal quantile 9 is determined as the (1 - a-)th quantile of these scores:
o1-a [(1 + (Paal)(1 ath largest value in {s(2s, M6) MiE
2
This quantile provides the precise value for our prediction threshold, such that T = Q1-a- For a new test input x E Dreat: the prediction set C(-) is constructed using this threshold:
C(x; {k E [K] (y S 91-a)-
(3)
This guarantee fundamentally relies on the assumption that calibration data Deat and test data Drest are exchangeable.
2.2 Violation of Exchangeable Assumption
In adversarial scenarios, attackers manipulate the test set images Diest, transforming the original input samples into adversarial examples xade through perturbation function (attack function) Io, ie., x (z). This transformation clearly disrupts the exchangeability property that underpins conformal prediction, thereby rendering standard conformal prediction unreliable in the presence of adversaries. To address this, there is a pressing need for more robust calibration and model selection strategies that explicitly account for adversarial perturbations.
2.3 Problem Setup
We consider a problem where an attacker selects an optimal attack S. {g from a set possible allacks, and a defender aims choose a best. classification model J E (5}3 and a threshold r minimize the expected size of the prediction set for adversarially perturbed test samples. The optimization must ensure that the true label is included in the prediction set with probability of at least - The allakrrr, within this robust. framework, seeks to maximize the prediction set size. The defender formulates the problem as:
min max Σ [C(xa ;5 ;[;,T)} f TER 9t. (EDtest
(4)
subject to: IP (ss E C(xadu, $3. T)) 2 aa
3
Game Theoretic Framework for Adversarially Robust CP
As shown in Figure D the optimization problem can be naturally framed as a two-player zero-sum game. As described in the problem setup, the defender aims to minimize the prediction set size while maintaining the
3

---

Published in Transactions on Machine Learning Research (01/2026)
coverage guarantee, whereas the attacker seeks to maximize it. This section first addresses how we ensure the critical coverage constraint, P (ye E C(x r)) 2 1 a, under adversarial conditions, before formulating the full game.
3.1 Robust Coverage Guarantee under Known Adversarial Attacks
To satisfy the coverage constraint in Eq. under adversarial perturbations, we employ a robust calibration strategy. The robust conformal quantile oi Pi-a for model ʃj is then set as the maximun across all attacks:
max fi
Remark (Intuition behind Max-Quani Max--Quntille. The non-conformity score quantifies the discrepancy wetw the prediction and the true label. Since adversarial attacks aim to induce misclassification, they typically inflate these scores compared to clean data. By selecting the maximum quantile across all considered attacks, we effectively establish threshold calibrated to the "worst-case perturbation. Intuitively, if a threshold is high enough to encapsulate the true label under the strongest attack (which produces the highest non-conformity scores), it will inherently cover the true label under weaker attacks, thereby preserving the coverage guarantee across the entire set of known attacks.
Prediction sets C(xadv; fjs ai are constructed using víaa as the threshold, consistent with Section 2.1 Additionally, we present Theorem ! with its corresponding proof provided Appendix A.I
Theorem (Robust Coverage Guarantee). For any classifier f, E (ffl/=1 and any adversarial attack In E ) that was included in the calibration process used compute ai . the prediction set C(xadv; : Ri provides coverage probability of at least - for adversarially perturbed test samples xqdr
P (ly C(xfdo, fo (i-))-a) 2 a
(6)
3.2 Game Formulation and Nash Equilibrium
Building upon the robust coverage guarantee established in the preceding section, the constrained optimiza- tion problem presented in Fq. 4 can be formulated as a two-player zero-sum game. In this game, the defender aims to minimize the prediction set size, while the attacker seeks to maximize it.
3.2.1 Participants and Strategies
This game involves two participants:
The defender's strategy is to select classifier fy from finite set of models F fp). For each chosen fj, robust conformal quantile di will serve as the threshold for con- structing the prediction set.
The Defender:
The Attacker: The attacker's strategy to select an adversarial attack Jr. from a finite set known attack methods g f91....,
3.2.2 Payoff Function
The payoff of the game is defined as the expected size of the prediction set. For a defender's chosen fa and an allacker's chosen Ia, the payofl (denoted as PayofT[]s, gn)) is the average prediction set size on adversarially perturbed test samples:
Payoff(ff...) ) = Ex-Px [[c(x""; of -
(7)
where Px is the distribution of the unperturbed x, x denotes the adversarial sample generated by applying attack In the original sample x. The defender seeks to minimize this payoff, while the attacker seeks maximize it.
4

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm Game-Theoretic Framework for Adversarially Robust Conformal Prediction
1: Input: labeled data D, unlabeled test data Diesn set of defensive models fz. .....,P set of potential attack functions In, = I...., Coverage probability - a. 2: Randomly split D into ((r, yi) De) {(r,, Vi) and ((r), Ui) Wi
> Step 1: Train all defensive models.
for l......... do Train defensive model fg on the training set Drrsin- end for
b Step 2: Compute robust. conformal quantiles for each model.
6: for each defensive model fj,j = ......... p do
7: for cach attack method In. = 1, m do
8: Generate adversarial calibration set: x = In (X,, yi, fylIED
Use [x(xx) compute non-conformity scores (8) (xf" v1) Wi)
Compute the quantile threshold as: at= as the [(1 + [Deal) a)]-th largest score in (xf",.)) 1a)
10:
11:
end for
Determine the maximum quantile threshold aia = max,= a:a for defensive model fy.
12:
13: end for
D Step 3: Estimate the prediction sizes on the evaluation set.
14: for each defensive model fj.j :..j=1..... ,,...,p do
15: for each attack methods In. n 1,..., do
16:
Generate adversarial evaluation set: x In (x;, Vi. Dj Construet the prediction using the model's robust quantile: C(x,"; J Dia Estimate the payoff by averaging the prediction set sizes over the adversarial evaluation set: Payoff( 9n)
17:
18:
19:
Deval Dier C(x²"; f fz. R-all-
20:
end for
21:
end for
b Step 4: Find Nash Equilibrium & Construct Robust Conformal Predictor.
22: Compute the Nash Equilibrium by solving a Linear Program: Payoff( fj. 9n).
23: Output the Nash Defense Strategy d" (di,...,dy and Nash Attack Strategy a" = (ai,.... am)
24: Use the Nash Defense Strategy d" to define the final robust predictor.
b Step 5: Testing Nash Defense Model.
25: for
test point Xnew Deess (which may be subjected to unknown adversarial attack g E 9 to become x do
26: Randomly sample defensive model ʃ3 according to the Nash Equilibrium strategy d'.
27:
Construct and output the prediction set using the sampled model fy and its corresponding pre- computed robust quantile q'-a; C(xeda)) {3/ : (x(xd v/) s d-a).
28: end for
3.2.3
Nash Equilibrium
Given that this is finite two-player zero-sum game (with finite strategy sets F and G), the existence of Nash Equilibrium in mixed strategics is guaranteed by the fundamental Minimax Theorem v. Neumann 1928).
Let d be mixed strategy for the defender (a probability distribution over F) and a be a mixed strategy for the attacker (a probability distribution over g). The expected payoff under these mixed strategies is [[Payof](d, a)).
Definition (Nash Equilibrium in Mixed Strategies): A pair of strategies (d", a*) constitutes a Nash Equilibrium if, for all other possible mixed strategics d and a:
E[Payoff(d'' a)] E(Payoff(d', a²"] s [|Payoff(d, a")]
(8)
5

---

Published in Transactions on Machine Learning Research (01/2026)
In the context of our zero-sum game, the defender d seeks to minimize the expected payoff (prediction set sixe), and the attacker seeks to maximize it. The Nash Equilibrium (d", a") is the stable point where no player can gain an advantage by unilaterally deviating.
The Minimax Theorem states that the minmax value (the best the defender can do) is equal to the maxmin value (the best the attacker can do), and this common value, V, is the value of the game. The optimal mixed strategies (d', a*) achieve this value V. Formally, the value of the game V is defined by the equality:
V = E[Payoff(dd "a"",""")) a²)] = min max B Payoff (.))
(9)
maa min E Payoolll
a
To compute this equilibrium, we first construct payoll matrix where rows correspond to defender's pure strategies fz E F and columns correspond to attacker's pure strategies In E G, with each entry being Pf gn). We then solve for the optimal mixed strategics a") ) by solving a Linear Program derived from the Minimax equality, which yields a solution that is robust and realistic, especially when pure strategy Nash Equilibria do not exist or when players wish to introduce unpredictability. Algorithm outlines the entire process of our framework.
Remark 2. Our framework finds Nash Equilibrium, defined by optimal mixed strategies for the defender (d*) and attacker (a*). This creates a stable state where no player can gain an advantage by unilaterally deviating, as any such move would be met by the opponent's optimal counter-play. The defender's strategy d², an unpredictable probabilistic ensemble, thus robust against the attacker's best possible strategic response, with the comprchensiveness of the attack set G defining the strict boundary and scope of this guarantee.
4 Related Work
Conformal prediction (CP) (Vokk et al. 2005) is methodology designed to generate prodiction regions for variables of interest, facilitating the estimation of model uncertainty by providing prediction sets rather than point estimates. CP has been successfully applied to both classification [uo & Colombo,
2024 Luo & Zhou
2025b) and regression tasks (Luo & Zhou, 2025c Bao et al. 2025a Guo ct al.
2026). Its flexibility allows
adaptation to various real-world scenarios, including segmentation (Luo & Zhou
2025c
games (uuo et al.
2024), time-series forecasting (Su et al. 2024). and graph-based applications (T.uo et al. 2023
Tang et al.
2025
Luo & Zhou, 2025d Wang et al. 2025a by Luo & Colombo
2025
Zhang al. 2025).
The advent of adversarial examples (Goodfellow et al. 2014 Zhang et al. 2022) has posed substantial secu- rity challenges within the field of machine learning. In this context, uncertainty quantification emerges as a critical factor for enhancing the resilience of deep learning models. Conformal Prediction (CP) (Papadopou- los et al.] 2002 Vork et al.] 2005), renowned for its ability to provide distribution distributtonnindeeender coverage guarantees, faces notable obstacles when confronted with data poisoning and adversarial perturbations Em- pirical investigations, such as those presented by Liu et al. (2024), reveal that conventional adversarial attack strategies including the Projected Gradient Descent (PGD) method (Madry et al. 2017 undermine the reliability of conformal prediction frameworks.
can significantly
Adversarially Robust Conformal Prediction To alleviate the adverse effects on conformal prediction (CP) under adversarial settings, a number of rescarch efforts have been dedicated to tackling this challenge without relying on model retraining.
Adversarially Robust Conformal Prediction (ARCP) (Gendler et al. 2021 integrates conformal prediction with randomized smoothing to provide finite-sample coverage guarantees in the presence of L2- tg-norm-bounded adversarial noise. By incorporating Gaussian noise, it constrains the Lipschitz constant of the non-conformily score, thereby handling unknown adversarial perturbations without the need for training adjustments. Prob- abilistically Robust Conformal Prodiction (PRCP) (Ghosh al. 2023) employs a quantilo-of-quantile ap- proach adapt to perturbations, establishing thresholds for both data samples and perturbations. It utilizes adversarial attacks to compute empirical robust. quantiles, independently of the model training process.
6

---

Published in Transactions on Machine Learning Research (01/2026)
Yan et al. 2024) put forward two methods, Post-Training Transformation (PTT) and Robust Conformal Training (RCT), to enhance the efficiency of robust conformal prediction. They revise RSCP into RSCP+ to provide certified guarantees and incorporate it into the training phase. ZZagarbbahh et al. (2024) develop robust prediction sets by bounding the worst-case variations in conformity scores under adversarial evasion and poisoning attacks. They employ CDF-based bounds to calculate conservative prediction sels and corre- sponding thresholds for these scenarios, [Jeary al. (2024) introduce Verifiable Robust Conformal Prediction (VRCP), which utilizes neural network verification techniques uphold coverage guarantees amid adversar- ial attacks. VRCP accommodates arbitrary norm-bounded perturbations and can be extended to regression
tasks.
While these approaches offer partial alleviation of the adversarial impact, they either sacrifice the com- pactness the prediction sets or struggle to sustain robustness against diverse attack types and varying perturbation magnitudes.
Adversarial Training for Conformal Prediction To bolster adversarial robustness in conformal pre- diction (CP), a straightforward and intuitive strategy involves integrating adversarial training techniques into the CP framework.
Bao et al. 2202bb put forward a method that enhances adversarial robustness under the CP setting by training on samples with uncertain attack models and incorporating conformal training with a hard thresh- old. Moreover, Liu et al. (2024) introduced Ehcertainty-Reduciin Adversarial Training (AT-UR) aimed at elevating both the efficiency and adversarial robustness of CP. This is achieved by minimizing predictive entropy and employing a weighted loss function grounded in True Class Probability Ranking. They further combined AT-UR with several established adversarial training methods, namely Adversarial Training (AT) (Madry al. 2017), Friendly Adversarial Training (FAT) (Zhang et al. 2020), and Trade-off-inspired Adver- sarial Defense via Surrogate-loss Mininization (TRADES) (Zhang et al. 2019). For generating adversarial examples, they used the Projected Gradient Descent (PGD) approach.
5 Experiment Results
5.1 Experimental Setup
To comprehensively evaluate the effectiveness and robustness of our proposed game-theoretic framework across datasets of varying complexity and scale, we have designed series of detailed experiments. This section will provide thorough introduction to the datasets used in the experiments, the defense models, the attack methods, as well as the specific parameter configurations.
Payoff Matrix on Evaluation Set (CIFAR10- AAPS) Evaltsation Set Size
Payoff Matrix on Test Set (CIFARIO -PS)
Tist Set Size
Adkepelio02206ieen RX18. 5.15
5.11
5.42
5.62
5.35
5.71
5.08
5.07
5.06
A6kggdi2zzEEE.ee.... RNLB
5.13
5.10
5.39
5.60
5.34
5.67
5.08
5.08
5.07
BAROACCANNUTSS 4.2
4.87
6.17
6.35
6.10
6.57
5.02
4.8
4.87
Deline Model
4.89
4.86
6.06
6.28
5.99
6.70
5.08
4.87
4.86
ChesotA4Ai 14
4.74
4.78
5.11
5.38
5.04
5.43
4.68
4.68
4.74
4.77
5.10
5.37
5.05
5.43
4.71
4.58
5.76
4.24
4.28
4.29
6.44
6.28
7.10
6.43
6.37
AR20
Rata
Spper
Arral
Attack Method
Attakk Methol
Figure/ 2: Evaluate the payoff matrix both the CIFAR-10 dataset's evaluation set and test set using the APS method.

---

Published in Transactions on Machine Learning Research (01/2026)
Table 1: Performance of various Conformal Prediction (CP) methods under the Nash defense strategy on the CIFAR-10 dataset. The table presents the mean and standard deviation of coverage, size, and SSCV (for specific indicator calculations refer to Appendix A.2 against a range of attack strategies. All results are averaged over 20 independent random splits.
Attacks
Indicator
APS
RANK
RAPS
SAPS
TOPK
Coverage
0.997 (0.001)
0.998 (0.001)
0.997 (0.001)
0.997 (0.001)
0.997 (0.001)
Clean
Size
4.826 (0.092)
5.263 (0.070)
4.833 (0.095)
4.666 (0.105)
5.655 (0.093)
SSCV
0.100 (0.000)
0.098 (0.001)
0.100 (0.000)
0.100 (0.000)
0.097 (0.001)
Coverage
0.982 (0.003)
0.982 (0.002)
0.982 (0.003)
0.981 (0.003)
0.980 (0.003)
FGSM
Size
5.237 (0.086)
5.356 (0.076)
5.241 (0.090)
5.069 (0.096)
5.653 (0.091)
SSCV
0.099 (0.002)
0.082 (0.002)
0.099 (0.000)
0.098 (0.001)
0.080 (0.003)
Coverage
0.983 (0.003)
0.984 (0.002)
0.983 (0.002)
0.982 (0.002)
0.981 (0.003)
PGD
Size
5.189 (0.086)
5.342 (0.075)
5.193 (0.088)
5.040 (0.099)
5.655 (0.089)
SSCV
0.099 (0.002)
0.084 (0.002)
0.099 (0.001)
0.098 (0.002)
0.081 (0.003)
Coverage
0.995 (0.001)
0.989 (0.002)
0.995 (0.001)
0.991 (0.002)
0.986 (0.002)
APGD
Size
4.879 (0.092)
5.287 (0.066)
4.887 (0.093)
4.762 (0.103)
5.654 (0.091)
SSCV
0.100 (0.000)
0.089 (0.002)
0.100 (0.001)
0.100 (0.000)
0.086 (0.002)
Coverage
0.997 (0.001)
0.998 (0.001)
0.997 (0.001)
0.997 (0.001)
0.997 (0.001)
CW
Size
4.917 (0.090)
5.291 (0.069)
4.924 (0.091)
5.124 (0.095)
5.658 (0.089)
SSCV
0.100 (0.000)
0.098 (0.001)
0.100 (0.001)
0.100 (0.000)
0.097 (0.001)
Coverage
0.898 (0.008)
0.898 (0.007)
0.899 (0.008)
0.898 (0.007)
0.900 (0.007)
PIFGSM
Size
5.553 (0.085)
5.416 (0.083)
5.563 (0.088)
5. 5.282 (0.094)
5.656 (0.088)
SSCV
0.099 (0.002)
0.005 (0.005)
0.099 (0.002)
0.066 (0.013)
0.006 (0.004)
Coverage
0.995 (0.001)
0.989 (0.002)
0.995 (0.001)
0.992 (0.001)
0.986 (0.002)
GN
Size
5.562 (0.091)
5.532 (0.072)
5.570 (0.086)
5.305 (0.078)
5.653 (0.092)
SSCV
0.100 (0.000)
0.089 (0.002)
0.100 (0.000)
0.100 (0.000)
0.086 (0.002)
Coverage
0.997 (0.001)
0.998 (0.001)
0.997 (0.001)
0.998 (0.001)
0.997 (0.001)
Square
Size
4.833 (0.088)
5.269 (0.068)
4.840 (0.094)
4.756 (0.099)
5.655 (0.088)
SSCV
0.100 (0.000)
0.098 (0.001)
0.100 (0.000)
0.100 (0.000)
0.097 (0.001)
Coverage
0.996 (0.001)
0.998 (0.001)
0.997 (0.001)
0.997 (0.001)
0.997 (0.001)
SPSA
Size
4.852 (0.091)
5.266 (0.070)
4.858 (0.094)
4.699 (0.107)
5.658 (0.088)
SSCV
0.100 (0.000)
0.098 (0.001)
0.100 (0.001)
0.100 (0.000)
0.097 (0.001)
Coverage
0.949 (0.013)
0.989 (0.001)
0.951 (0.012)
0.968 (0.021)
0.982 (0.029)
Nash
Size
5.558 (0.088)
5.532 (0.072)
5.566 (0.087)
5.304 (0.078)
5.657 (0.092)
SSCV
0.099 (0.001)
0.089 (0.001)
0.100 (0.001)
0.093 (0.006)
0.083 (0.028)
5.1.1 Datasets
Our experiments are conducted on three standard computer vision benchmarks: CIFAR-10, CIFAR-100, and ImageNet. For CIFAR-10 and CIFAR-100, we utilize their official test sets. For ImageNet, we use the official validation set. For each dataset, we randomly split the corresponding set into three disjoint subsets: 30% for the calibration set (Deal), 30% for the evaluation set (Dovat, and the remaining 40% for the final test set (Dreat)
5.1.2 Adversarial Attacks
We generate adversarial examples using the torchattacks library Kim 2020), with the perturbation for all attacks constrained within an Tor-norm ball of radius = 8/255. As baseline, we use original, unperturbed
8

---

Published in Transactions on Machine Learning Research (01/2026)
Payoff on Evaluation (CIFARI00 SAPS)
Payoff Matrix Test (CIFARI00 SAPS)
Evaluation Set Size
Test Set Size
wwppliieeeteeeee RNIR 71.24
71.45
71.48
71.43
71.61
F1.18
71.20 16
Atoggii BNN
71.26 7133 71.46
71.52
71.45
71.62
71.21 71 71.19
Cleabi24Deua 34,10 -2.73
Define Modil
WAN 34
02.76
62.83
62.20
62.29
62.
Deberelessaiiiggg... XCT4-12 XCTTI1
63.65
63.38
63
89
64.28
63.87
64.00
63.56
63.52 63.53
DesombentiiC.gge XCTAL1
53.71
64.03
63.34
64.34
63.92
64.11
63.62 63, 63.58
Bongoe2sotssneess WRNT 16 76.18
76.
58
76.
42
76.60 76.40
76.61
76.
76.18
76.08
PangrezZubesttees WRNX0 16
16-76.24
76,24
76.61
76.45
45
76666
76.44
76.63
13 76. 76.1
dop
Arack Method
Anaci Mechod
Figure 3: Evaluate the payolf matrix on both the CIFAR-100 dalaset 's evaluation set and test set using the SAPS method.
Calculated Nash Equilibrium Strategy (CIFAR10 APS)
Defender's Nash Strategy
Attacker's Nash Strategy
18-
10
0,5
Probability
Probability
0.4-
0.50
0.50
64
0.07
Medpeltizniiiiicil...... lest RN18
BaERR4MiLeddUUS
Seraedeni2022//e Ligh XCIT-512
XattoEE WRO--28-10
APGD
CW
PGSM
GN
PGD
PIFGSM
SPSA
Square
dlan
Chen2024Daia WRN 3d
Figure 4: Obtain the Nash attack and Nash defense strategies the CIFAR-10 dataset using the APS method.
images, The specific attack parameters are set as follows: the FGSM (Fast Gradient Sign Method) attack (Goodfelow et al. 2014) uses the defanlt epsilon of 8/2255; PGD (Projected Gradient Descent) (Madry) et al. (017) is configured with 10 iteration steps, a step size (a) of 1 1/255, and a random starting point within the e-ball; APGD (Auto-PGD) (Croce & Hein 2020 is set to run for 10 steps utilizing the cross-entropy loss function. Furthermore, the CW (Carlini & Wagner) attack (Carlini & Wagner 2017 is configured with 50 optimization steps, a learning rate of 0.01, and a confidence parameter (c) of 1; PIFGSM (Projected Iterative FGSM) (Gao el al., 2020) is execuled for 10 ilerations; and the default library implementation is used for Gaussian Noise (GN). For the CIFAR-10 10 and CIFAR-100 datasets, we also incorporate two additional black- box methods: the Square attack (Andriushchenko al. 2020), which is limited to maximum of 1000 queries per sample, and SPSA (Simultancous Perturbation Stochastic Approximation) for which employ the default. library implementation.
(Uesato et al., 2018
9

---

Published in Transactions on Machine Learning Research (01/2026)
Calculated Nash Equilibrium Strategy (CIFAR100 SAPS)
Defender's Nash Strategy
Attacker's Nash Strategy
0.56
12
64
Probubility
0.61
Probability
0,39
0.04
Addepaiin Miitiit RN10
ChecitaDaes 3482 14.
CatutiDeounphddd wEV-36-10 3-110
Debandent jeit XOTLL
Pang D0g Aorttttt WRN199
ANDD
FCSM
PGD
FIFGSM
SPSA
dlop
Figure 5: Obtain the Nash attack and Nash defense strategies on the CIFAR-100 dataset using the SAPS method.
Dissiiratien of Predicion Set Siee under CN Attack Methee: APS, Batase: AR11)
Distribution of Prediction Set Sixe under MEGGMMAuad (LP Macked: APS Datame: CIEARM)
Disrilution of Predition Sei Size under Nask-Attcck (CP Mecbett: Datsmr: CHEAIL
Men diaaae 8o
Minue Must (4
Figure Box plots size for different defense models using the APS method on the CIFAR-10 dataset under various attacks.
5.1.3 Defensive Models
To ensure fairness and reproducibility, all defensive models, denoted as fj, are sourced from the RobustBench benchmark (Croce et al.) 2021). RobustBench provides a standardized library of publicly available, state- of-the-art models. Since there inconsistencies in training models across different datasets, mainly
selected these models (Addepalli et al. 2022 Bai et al. 2024 Chen & Tee 2024
Debenedetti et al.; 2023
Xu et al.] 2023 Cui et al. 2024 Addepalli et al. 2022
Pang et al.) 2022 Peng et al. 2023). These models have verilied adversarial robustness and serve as a recognized reference for progress in the field.
In our experiments, the significance level a consistently set to 0.1 across all trials. We employ five widely recognized CP methods (Huang al. (0244a) evaluate the performance of our framework, namely APS [Romano et al., 2020), RAPS (Angelopoulos et. al. 2021), TOPK (Angelopoulos et al. 2021), SAPS (Huang et al.) 2024b), and RANK (Luo & Zhou 2025a) approaches. We use coverage, size, and SSCV (Angelopoulos 2021 as evaluation metrics.
ct al.
A core finding of our game-theoretis framework is the exceptional robustness of the Nash defense stralegy, irrespective of the Conformal Prediction (CP) method. As shown in Table 1, our Nash defense guarantees that the expected prediction set size remains at or below the threshold set by the attacker's own Nash stratogy. This upper bound holds consistently, with minor deviations attributable the inherent generalization gap between the evaluation and test sets.
10

---

Published in Transactions on Machine Learning Research (01/2026)
Distribuaias of Predicios Set Siee under CW Anack
Disxiluttion of Predicion Set Stee under Atack
Diarilution of Preditien Set Siee under Nash Antack
Defoce Stang
Figure 7: Box plots of size for different defense models using the SAPS method on the CIFAR-100 dataset under various attacks.
Additionally, as shown in Figures G and 7 when confronted with Nash attacks, our Nash defense strategy consistently outperforms many standalone defense models. While some defense models demonstrate com- parable performance, their vulnerabilities are also exposed (as illustrated in the left and middle pancls of Figures 0 and 7) The key advantage of Nash defense lies in its guaranteed performance ceiling: ensures that the size of the prediction set does not exceed the upper bound defined by the Nash attack value, guarantee that other individual models cannot provide against their respective worst-casc attacks. This is the essence of its robustness: it prevents adversaries from gaining an advantage by unilaterally altering their strategies, thereby ensuring reliable worst-case performance.
As depicted in Figures 2 3 it illustrates the errors the payoff matrices between the evaluation and test sets generated by Nash strategies, while Figures 4 5 showcases the attack and defense strategies under Nash equilibrium. We conducted detailed experiments on attack methods that fall outside the framework construction (for details, see Appendix A.3). Additionally, other results are presented in Appendix A.5
6 Conclusion
In this study, we have introduced an innovative game-theoretic framework designed to tackle the significant vulnerability of conformal prediction when subjected to adversarial attacks, which fundamentally breach the foundational assumption of exchangeability. By conceptualizing the adversarial interaction between the defender and attacker as zero-sum game, we have transitioned the defense strategy from relying on static model selection to pursuing a dynamic, strategic equilibrium. Through rigorous theoretical analysis and ex- tensive empirical validation on benchmark datasets such as CIFAR-10, CIFAR-100, and ImageNet, we have demonstrated that the Nash Equilibrium solution yields a mixed defense strategy that offers provable robust- ness within the defined game-theoretic framework. Although our Leave- One-Ontt---te experiments suggest strong empirical generalization capabilities against unseen attacks, recognize notable limitation: the stringent theoretical coverage guarantees are contingent upon the boundaries of the attack space. Consequently, ad- versarial perturbations that are exceptionally potent or markedly distinct from those in the training set may still undermine the validity of the coverage guarantees. Despite this limitation, our framework provides a verifiable defense mechanism that ensures worst-case performance against optimal adversarics within the modeled attack scope. By bridging the domains of adversarial robustness and uncertainty quantification, this work establishes a principled approach towards achieving reliable artificial intelligence. Future work could explore extending this framework to continuous strategy space, integrating certified robustness techniques, to provide guarantecs against adversarial perturbations g & G.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (Grant 62506315) and City University of Hong Kong (Grants 9610639, 7020161).
11

---

Published in Transactions on Machine Learning Research (01/2026)
References
Sravanti Addepalli, Samyak Jain, et al. Efficient and effective augmentation strategy for adversarial training. Advances in Newral Information Processing Systems, 35:1488 1501, 2022.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: query- efficient black-box adversarial attack via random search. In European conference on computer vision, pp. 484-50 501. Springer, 2020.
Anastasios Nikolas Angelopoulos, Stephen Bates, Michael Jordan, and Jitendra Malik. Uncertainly sets for image classifiers using conformal prediction. In International Conference on Learning Representations, 2021.
Bakary Badjie, José Cecílio, and Antonio Casimiro. Adversarial attacks and countermeasures image classification-bass based deep learning models in autonomous driving systems: A systematic review. ACM Computing Surveys, 2024.
Yatong Bai, Mo Zhou, Vishal M Patel, and Somaych Sojoudi. Mixednuts: Training-free accuracy-robustnes balance via nonlinearly mixed classifiers. Transactions on Machine Learning Research, 2024.
Jie Bao, Nicolo Colombo, Valery Manokhin, Suqun Cao, and Rui Luo. A review and comparative analysis of univariate conformal regression methods. In Fourteenth Symposium on Conformal and Probabilistic Prediction with Applications (COPA 2025), pp. 222-044. PMLR, 2025a.
Jie Bao, Chuangyin Dang, Rui Luo, Hanwei Zhang, and Zhixin Zhou. Enhancing adversarial robustness with conformal prediction: A framework for guaranteed model reliability. In Proceedings of the Forty-second International Conference on Machine Learning. PMLR, 2025b.
Nicholas Carlini and David Wagner. Towards evaluating the robust of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39 57. leee, 2017.
Erh- Erh-Chung Chen and Che-Rung Lee. Data filtering for efficient adversarial training. Pattern Recognition, 151:110394, 2024.
Wei Chen, Shian Du, Shigui Li, Delu Zeng, and John Paisley. Entropy-informed weighting channel normal- izing flow for deep generative models. Pattern Recognition, pp. 112442, 2025a.
Wei Chen, Shigui Li, Jiacheng Li, Junmei Yang, John Paisley, and Delu Zeng. Dequantified diffusion- schrödinger bridge for density ratio estimation. In Forty-sccond International Conference on Machine Learning, 2025b.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206 2216-226..-2 PMLR, 2020.
Francesco Croce, Maksym Andriushchenko, Vikash Schwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: standardized adversarial robustness benchmark. In Thirty--ifth Conference Neural Information Processing Systems Datasets and Bench- marks Track (Round 2), 2021.
Jiequan Cui, Zhuotao Tian, Zhisheng Zhong, Xiaojuan Qi, Bei Yu, and Hanwang Zhang. Decoupled kullback- leibler divergence loss. Advances in Neural Information Processing Systems, 37:74461 74486, 2024.
Edoardo Debenedetti, Vikash Sehwag, and Prateek Mittal, A light recipe to train robust vision transformers. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp. 225-255 IEEE, 2023.
Lianli Gao, Qilong Zhang, Jingkuan Song, Xianglong Liu, and Heng Tao Shen. Patch-wise attack for fooling deep neural network. In European Conference Computer Vision, pp. 307-322 Springer, 2020.
Asaf Gendler, Tsui- Wei Weng, Luca Daniel, and Yaniv Romano. Adversarially robust conformal prediction. In International Conference on Learning Representations, 2021.
12

---

Published in Transactions on Machine Learning Research (01/2026)
Subhankar Ghosh, Yuanjie Shi, Taha Belkhouja, Yan Yan, Jana Doppa, and Brian Jones. Probabilistically robust conformal prediction. In Uncerlainty in Artificial Intelligence, pp. 681-690. PMLR, 2023.
Tan J Goodfellow, Jonathon Shlens, and Christian Svegedy. Explaining and harnessing adversarial examples. arXiv preprint Xr:: 12..6772, 2014.
Naixin Guo, Rui Luo, and Zhixin Zhou. Fast conformal prediction using conditional interquantile intervals. arXiv preprint arXic:2601.0228 2026.
Jianguo Huang, Jianqing Song. Xuanning Zhou, Bingyi Jing, and Hongxin Wei. Torchep: A python library for conformal prediction, 2024a.
Jianguo Huang, HuaJun Xi, Linjum Zhang, Huaxiu Yao, Yue Qiu, and Hongxin Wei. Conformal prediction for deep classifier via label ranking. In Forty-first International Conference on Machine Learning. 2024b.
Linus Jeary; Tom Kuipers, Mehran Hosseini, and Nicola Paoletti. Verifiably robust conformal prediction. arXiv preprint X6x::240 18942, 2024.
Hoki Kim. Torchattacks: pytorch repository for adversarial attacks. ar preprint arXis:2010.01950, 2020.
Ziquan Liu, Yufei Cui, Yan Yan, Yi Xu, Xiangyang Ji, Xue Liu, and Antoni B Chan. The pitfalls and promise of conformal inference under adversarial atlacks. arXiv preprinl MXXv:2405008886 2024.
Rui Luo and Nicolo Colombo. Entropy reweighted conformal classification. In The 18th Symposium on Conformal and Probabilistic Prediction with Applications, pp. 264-276, PMLR, 2024.
Rui Luo and Nicolo Colombo. Conformal load prediction with transductive graph autoencoders. Machine Learning, 114(3):1-22, 2025.
Rui Luo and Zhixin Zhou. Reliable classification through rank-based conformal prediction sets. Pattern Recognition, pp. 112330, 2025a.
Rui Luo and Zhixin Zhou. Conformity score averaging for classification. In Proceedings of the Forty-sccond International Conference on Machine Learning (ICML), 2025b. to appear.
Rui Luo and Zhixin Zhou. Conditional conformal risk adaptation. arXiv preprint arXic:2504.07611, 2025c.
Rui Luo and Zhixin Zhou. Conformalized interval arithmetic with symmetric calibration. Proceedings of the
AAAI Conference on Artificial Intelligence, 19(11))19207-11225, 2025d.
Rui Luo and Zhixin Zhou. Conformal thresholded intervals for efficient regression. Proceedings of the AAAI Conference on Artificial Intelligence, 39(18): 19216 -99223, 2025e.
Rui Luo and Zhixin Zhou. Density-sorted prediction set: Efficient conformal prediction for multi-target regrossion. Pattern Recognition, pp. 112513, 2025f.
Rui Luo, Buddhika Nettasinghe, and Vikram Krishnamurthy. Anomalous edge detection in edge exchangeable social network models. In Conformal and probobilistic prediction with applications, pp. 287-310. PMLR, 2023.
Rui Luo, Jie Bao, Zhixin Zhou, and Chuangyin Dang. Game-theoretic defenses for robust conformal predic- tion against adversarial attacks medical imaging. arXiv preprint orXiv:2411 11.111104477 2024.
Xingjun Ma, Yuhao Niu, Lin Gu, Yisen Wang, Yitian Zhao, James Bailey, and Feng Lu. Understanding adversarial attacks on deep lcarning based medical image analysis systems. Pattern Recognition, 110: 107332, 2021.
Alcksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. stat, 1050(9), 2017.
13

---

Published in Transactions on Machine Learning Research (01/2026)
Tianyu Pang, Min Lin, Xiao Yang, Jun Zhu, and Shuicheng Yan. Robustness and accuracy could be recon- cilable by (proper) definition. In International conference. on machine learning, pp. 17258 17277. PMT.R, 2022.
Harris Papadopoulos, Kostas Proedrou, Volodya Vovk, and Alex Gammerman. Inductive confidence machines for regression. In Machine learning: ECML 2002: 19th European conference on machine learning Helsinki, Finland, August 19-23, 2002 proceedings 18, pp. 445-356. Springer, 2002.
Yun Peng, Weilin Xu, Cory Cornelius, Matthew Hull, Kevin Li, Rahul Duggal, Mansi Phute, Jason Martin, and Duen Horng Chau. Robust principles: Architectural design principles for adversarially robust cmns. arXiv preprint 1.X 2023.
Yaniv Romano, Matteo Sesia, and Emmanuel Candes. Classification with valid and adaptive coverage. Aduances in Newral Information Processing Systems, 53:3581-3591, 2020.
Xiaoyi Su, Zhixin Zhou, and Rui Luo. Adaptive conformal inference by particle filtering under hidden markov models. Xiv preprint or 2rXiu:2411.01555 2024.
Lingxuan Tang, Rui Luo, Zhixin Zhou, and Nicolo Colombo. Enhanced route planning with calibrated uncertainty set. Machine Learning, 114(5):1 16, 2025.
Jonathan Uesato, Brendan O' O'donoghue, Pushmeet Kohli, and Aaron Oord. Adversarial risk and the dangers of evaluating against weak attacks. In International conference on machine learning, pp. 5025-5034. PMLR, 2018.
v. Neumann. Zur theoric der gesellschaftsspicle. Mathematische annalen, (00(1):295-320) 1928.
Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world, vol- ume 29, Springer, 2005.
Ting Wang, Kaiyuan Zhang, Rui Luo, Zhixin Zhou, and Wen Jung Li. Trustworthy drug-drug interaction prediction model via weighted aggregating conformal prediction. In The 23rd Asia Pacific Bioinformatics Conference, 2025a.
Ting Wang, Zhixin Zhou, and Rui Luo. Enhancing trustworthiness of graph neural networks with rank-based conformal training. Proceedings of the AAAT Conference on Artificial Intelligence, 39(20):21261 (1122)):1261-21260 21268. 2025b.
Yuancheng Xu, Yanchao Sun, Micah Goldblum, Tom Goldstein, and Furong Huang. Exploring and exploiting decision boundary dynamics for adversarial robustness. arXiv preprint arXin:2302.03015, 2023.
Ge Yan, Yaniv Romano, and Tsui-Wei Weng, Provably robust conformal prediction with improved efficiency, arXiv preprint arXi::2404.1 19651, 2024.
Soroush H Zargarbashi, Mohammad Sadegh Akhondzadeh, and Aleksandar Bojchevski. Robust yet efficient conformal prediction sets. arXiv preprint 2rXiv:2407.09165, 2024.
Hanwei Zhang, Teddy Furon, Laurent Amsaleg, and Yannis Avrithis. Deep neural network attacks and defense: The case of image classification. Multimedia Security, 1:41-75, 2022.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theo- retically principled trade-off between robustness and accuracy. In International conference on machine learning, pp. 7482. PMLR, 2019.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankanhalli. Attacks which do not kill training make adversarial learning stronger. In International conference on machine learning, pp. 11278 11287. PMLR, 2020.
Zheng Zhang, Jie Bao, Zhixin Zhou, Lixin Cheng, Rui Luo, et al. Residual reweighted conformal prediction for graph noural networks. In The 4 1st Conference on Uncertainty in Artificial Intelligence, 2025.
14

---

Published in Transactions on Machine Learning Research (01/2026)
A Appendix
A.1 Proof for Theorem 1
Proof. Let ʃj be a given classifier and let xtdl be a test sample generated by an arbitrary attack In. E {0a b
By the definition of the conformal quantile, for the specific attack In, the quantile oh= computed on the calibration set Deal guarantees:
P (x(gg(x)) Vi. fi), s gd-a) 2 1 a.
(10)
The robust quantile q/ for model fj is defined as:
d-a- max -i
(11)
ke (1mm.)
From this definition, it directly follows that for our chosen attack Ini
diaa 2 af-a..
(12)
The event sxpddy, w/)) z ufth. is subset of the event s(x,dv, y1s S ai Therefore, the probability of the latter is at least as large the probability of the former:
P((xxx P (s(xxddv .v) S ai-a) TP (s(x)dy, ,,,,,,y,))))) ve) sf-aa)
(13)
(x)) The condition for the true label yi to be in the prediction set C(x;dv S3. Ri-a) is preciscly s(x,dv, v.) S zi Combining these steps, we have:
P(( E C(xadv : vi-a) = (s(x,evv ,yi) S ri-a)
2 P (x(xadv, .bi) S di-
a.
This completes the proof.
A.2 Conformal Prediction Evaluation
The Coverage measures the proportion of test instances in Drest where the true label is contained within the prediction set I'(x;,, and is defined
Coverage
Σ (vs T(x.))
(14)
[D
Drest
VE
A higher coverage indicates that the prediction sets reliably contain the true labels.
The Size metric calculates the average number of labels the prediction sets across all test instances,
Size
Σ [[(x:))
(15)
[Deest
iEDeoa
where smaller sizes denote more precise and informative predictions.
The Size-Stratified Coverage Violation (SSCV) evaluates the consistency of coverage across different predic- tion set sizes. It is defined as
[i E J, 21 E((+))] - (1)
SSCV(T, ($)))))) sup jE(s]
(16)
[J
where {5,} partitions the possible prediction set sizes, and J, {i E Drest [[(x+]] E Sz}. smaller SSCV indicates more stable coverage across different, set sizes.
15

---

Published in Transactions on Machine Learning Research (01/2026)
A.3 Generalization to Unknown Attacks and Theoretical Discussions
In this section, extend our analysis to evaluate the robustness of the proposed framework against unknown adversarial threats and provide further theoretical insights into the game-theoretic formulation. Specifically, we address the generalization capability of the Nash defense strategy, discuss the implications of the finite atlack space assumption, and provide an intuitive interpretation of the robust coverage guaranlee.
performance against attack strategies that were not encountered during the training or calibration phases. While our main framework assumes the attacker selects from predefined set G, real-world scenarios may involve "unknown" atlacks y & G. To rigorously evaluate the generalization capability of our Nash defense strategy, conducted Leave-One-Out (LOO) and Lcave-Two-Out (LTO) experiments.
A critical consideration for any adversarial defense
Experimental Setup: In the LOO setting, we systematically exclude one attack strategy from the attacker's portfolio G during the calibration of conformal quantiles and the computation of the Nash Equilibrium. The resulting defense strategy is then evaluated against the ercluded attack. Similarly, in the LTO setting, two attacks are simultaneously excluded from the defender's knowledge base and used solely for testing.
Analysis of Results: The results for CIFAR-10 and CIFAR-100 are presented in Table 2 and Table 3 (referring to the tables provided in Appendix A.5 and A.6), respectively.
Robustness Consistency: For the majority of attack methods, including APGD, CW, PGD, and Square Attack, the Nash defense strategy maintains valid coverage (close to or excceding the target 1 1 0.90) even when these specific atlacks are excluded from the calibration set. For instance, in the LOO experiments on CIFAR-10, excluding APGD results in a coverage of 0.995 against APGD itself. This suggests that the remaining attacks in G provide a sufficient "basis" approximate the worst-case boundary of the perturbation space for these attack types.
We observe performance drop when testing against PIFGSM (Projected Tterative FGSM) when it is excluded from the game. As shown in the tables, the coverage against an unknown PIFGSM drops significantly (e.g., to ~ 0.70 in LOO settings). This highlights a fundamental property of dala-driven defenses: if an unknown atlack explores a region of the perturbation space that is orthogonal to or significantly more aggressive than the known set G, the exchangcability assumption is violated too severely for the surrogate attacks to compensate. However, when PIFGSM is included in the set (as in our main experiments), the framework successfully adapts, restoring valid coverage.
Limitations and Boundary Cases:
These findings demonstrate that while the framework generalizes well to unknown attacks that share struc- tural characteristics with the known set, the defender must ensure the predefined set G is diverse enough to approximate the "worst-case" capabilities of potential adversary.
A.4 CIFAR10 results
In this section, we provide supplementary payoff matrices (on the evaluation set) and payoff matrices (on the test set) for other methods applied to the CIFAR-10 dataset, along with corresponding results such as box plots.
We will elaborate on the specific meanings of these figures in detail. Figures and are box plots showing the results of using the RANK and TOPK CP methods on the CIFAR-10 dataset. The figure plots the size of the prediction sets, and it can be seen that the Nash defense yields the smallest prediction sets. Figures 12 and 13 respectively display the Nash attack strategics and Nash defense strategics obtained on the validation set. for four different CP methods (RAPS, SAPS, RANK, TOPK). Figures 14 and 15 show the sizes of prediction sets for RAPS and SAPS as CP methods under single attacks and Nash attacks, respectively. We mainly focus on the non-singlo-attack scenario for Nash strategics. It can be observed that there is inconsistency among the defense models that are optimal against different single attacks. Therefore, the Nash defense employs different strtegiess, namely, combined defense model. Figures 16 to T9 plot the
10
16

---

Published in Transactions on Machine Learning Research (01/2026)
Table 2: Robustness against Unknown Attacks (Generalization) on CIEARIO. Values denote Size (Std) / Coverage (Std).
Excluded Attack(s)
RANK
RAPS
SAPS
TOPK
Leave-One-Out (LOO)
Ludioatee
Size
Covesage
Staee
Coverage
Stxc
Coverage
Size
Covesage
Stze
Coverage
APGI
4.88 (1.119) 0.596 (0.001)
3.77 [0.08) 10,979 (0.1112)
4.89 (0.11) 00.93 (0.300)
3,99 (0.19) /0.982 (0.004)
3.56 [0.00] / /0,975 (0.0112
CW
4.32 (0.09) 0.597 (0.001)
3.57 (0.05) 0.996 (0.001)
4.83 (0.11) / 0.9377 (0.001)
4.95 (0.09) / /0,008 (0.001)
3.56 (3.05) / /0.995 (0.001)
PGSM
5.25 (0.06) 0.981 (0.003)
3.94 (0:03) /0965 (0.002)
5.25 (0.10) / 0.952 (0.000)
4.37 (0.15) / /0:971 (0.003)
3.36 [0.05) / /0.055 (0.002)
GN
5.89 (0.17) 0.596 (0.001)
3.78 [0.07) 009999 (0.015)
5,50 (0.07) /0.936 (0.001)
4.27 (0.29) /0.960 (0.004)
3.57 (0.08] /0.962 (0.0013)
PGD
5.19 (0.06)
(0.003)
3.91 [0.00] 0.970 (0.001)
5.20
00.984 (0.000)
4.33 (0.18) /0.988 (0.003)
3.37 (9.07) 000966 (0.002)
PIPGSM
3.21 (0.08) 0.70G (0.017)
1.36 (0.02) 0.666 (0.008)
3.20 (1.006) 0.703 (0.014)
2.44 (0.04) 0.7334 (0.007)
1.76 (0.08) / 0.707 (0.010)
SPSA
4.96 10) 0.587 (0.001)
3.82 (0.07) 0.996 (0.001)
4.87 (0.10) / 0.397 (0.001)
4.05 (0.19) 0.957 (0.001)
3.54 [0.00) / /0.995 (0.001)
Squaree
4.8L (0.00) 0.907 (0.001)
(0.07) 0.906 (0.001)
4.85 (0.10) / (0.001)
4.30 (0.16) / (0.001)
3.54 (0.05) / 000005 (0.001)
Leave-Twa- Out (LTO)
Indieator
Size
Coverage
Size
Ooverage
Size
Coverage
Size
Covrrage
Size
Coverage
APGD CW
4.32 (0.10) / 0.506 (0.002)
3.52 (0.08) 00.988 (0. (0.000)
4.91 (0.10) / 0.006 00.0011
4.47 (0.53) / 0.950 (0.008)
3.50 [0.00) / 00:855 (0.010)
PGSM
5.08 (0.20) 0.988 (0.007)
3,55 (0.10)
(0.008)
5.06 (0:20) / 00.985 (0.007)
4.25 (0.31) /09999 (0.005)
(0.09) /0.965
APGD + ON
5.18 (1.75) 0.599 (0.0022)
3.78 (0.07) 0.974 (1.005)
5.17 (0.75) / /0.954 (0.0005
4.23 (0.35) / /0.962 (o0004)
3.20 (3.00) / /..969 (0.0017
APGD PGD
5:06 (0.18) 0.588 (0.006]
3.84 (0.00) 0.975 (0.005)
5.04 (0.18) 000350 (0.006)
4.25 (0.50) /0:960 (0.005)
3.30 (0/00) /0.071 (0.005)
APGD PIRGSM
(0.43) 0.802 (0.101)
1.26 (0.06) 0.770 (0.120)
2.75
0.808 (0.105)
2:00 (0.25) 00800 (0.088)
1.73 (0.04) 0.828 (0.076)
APGD + SPSA
4.89 (0.11) 0.506 (0.001)
3.50 (0:08) 0.987 (0.000)
4.87 (0.00) 0.006 001)
4.07 (0.25) /.950 (0.008)
3.35 (0.00) / 000855 (0.010)
APGD Square
4.88 (0.11) 0.596 (0.0022)
3.79 (0.08) 0,988 (0.001)
4.86 (0.09) 0..96 (0.001)
4.15 (0.25) 00950 (0.008)
3.55 (0.09) / 00,955 (0.010)
CW FGSM
(0.18)
(0.008)
3.90 (0.05) 0,979 (0.017)
6.08 (0.19) 0,980
4.66
(0.012)
3:38 (0.09) 0.997 (0.020)
CW GN
5.20 (1.72) 0.596 (0.0011
3.83 (0.07) 0.983 (0.014)
5:20 (0.72) 0.330 (0.0001)
4.59 (0.31) 0.968 (0.020)
3.30 (3.00) (0379 (0.011)
CW PGT)
5.08 (0.17) 0.500 (0.007)
3.80 (0.05) 0.985 (0.013)
5.06 (0.17) / 0.000 (0.007)
4.64 (0.36) / 00965 (00012)
3.50 (0.00) / 0.981 (0.015)
CW PIFGSM
2.91 (0.53) 0.628 (0.125)
1.28 (0.07) 0.827 (0.162)
2:95 (0.35) 0.829 (0.125)
2:56 (0.55) 0.865 (0.125)
1,75 (0.00) 0.870 (0.115)
SPSA
4.91 (0)
(0.001]
3.65 (0.06)
(0.001)
4.60 (0.00)
(0.001)
(0.52)
(0.001)
(0.00) /0.005 (0.001)
CW + Square
4.90 (0.12) 0.597 (0.001)
3.84 (0.07) 0.996 (0.001)
4.88 (0.10) 0.997 (0.00 (0.001)
4.57 (0.45) / 0.908 (0.001)
3.56 (0.09) 7 00.055 (0.001)
PGSM- GN
0.88 (0.00) 0.500 (0.007)
3.2G (0.09) 0.300 (1.1014)
6.38 (0.551 / 01.500 (0.007)
4.42 (0.33) / 0:978 (0.005)
3.58 (1.00 (..00 (OLUID)
FGSM PGD
5:25
(0.003)
3.92 (0/08)
(0.004)
6.22 (0.10)
(0.000)
4.42 (0:26) / 0.996 (0.008)
3.58 (3.00] 0.961 (0.006)
PGSM PIFCSM
2.55 (0.17) 0.791 (0.108)
1.35 (0.01) 0-785 (0.121)
2.92 (0.17) / 0.790 (0.102)
2.42 (0/0)) / 0.822 (0.091)
1.70 (0.08) / 0,882 (0.077)
PGSM- SPSA
5:07 (0.22) 0.969 (0.008)
5.68 (0.08] 0.970 (0.017)
5.04 (0:21) / 0.080 (0.008)
4.26 (0.20) / 0 966 (0.011)
3.55 (0.00) / 0.074 (0.021)
FGSM- Square
5:06 (0.25) 0.589 (0.008)
3:87 (0.08) 0,979 (0.017)
5.04 (0:22) / 0,990 (0.008)
4.34 (0.24) 0.967 (0.012)
3.58 (0.09) 0.975 (0.021)
GN
6.34 (0.00) 0.500 (0.006)
3,85 (0.06] 0,970 (0.001))
5.34 (0.50) 0,320 (0.006)
4.40 (0.25) / 0.977 (0.006)
3.58 (3,00) / 0.360 (0.003)
GN+ + PIPGSM
3.21 (0.25) 0.825 (0.1433)
1.37 (0.02) 0.792 (0.127)
3.20 (0.25) 0.823 (0.144)
2.43 (0.00) / 0.829 (0.097)
1.75 (0.02) 0,828 (0.076)
CN + SPSA
5.15 (0.78) 0.596 (0.001)
3,50 (0.07) 0.988 (0.013)
5.14 (0.77) / 0.300 (0.000)
4.24 (0.37) / 0.080 (0.009)
3.55 (0.00) 0.0T8 (0.017)
GN+ Square
5.14
(0.001]
3.79 (0.07) 0.985 (0.014)
5.14 (0.78) / 0,996 (0.001)
4.32
(0.000)
3.55 (0.00) / 0.070 (0.017)
PGD PIEGSM
3.02 (0.20) 0.805 (0.104)
1.34 (0.02) 0.791 (0.127)
2.99 (0.20) / 0.805 (0.103)
2.40 (0.05) / 0.825 (0.092)
1.75 (0.03) / 0.839 (0.085)
PGD SPSA
5,01 (0.20) 0.990 (0.067)
3.67 (0.07) 0.985 (0.015)
5.02 (0.19) 0.990 (0.007)
4.21 (0.25) 0.967 (0.011)
3.55 (0.09) 0,080 (0.015)
PGD Square
5,35
0.590 (0.007)
3.86 (0.08)
(1.1113)
5.01 (0.20)
(0.0007
4.32
0.967 (0.011)
3.30 (1U0)) 0,300 (0.015)
PIPGSM SPSA
2.82 (0.39) 0.832 (0.129)
1.28 (0.06) 0.828 (0.159)
2.81 (0.38) 0.832 (0.130)
2:21 (0.23) 0.850 (0.118)
1.76 (0.08) 0.868 (0.1 12)
PIFGSM Squarre
2.52 (0. 80) 0.652 (0.129)
1:20 (0.06) 0.821 (0.157)
2,80 (0.38) 0.83.1 (0.130)
2.26 (0.19) 0.851 (0.118)
1.75 (0.00) 0.847 (0.093)
SPSA Square
4.56 (0.
10)
0.907 (0.001)
3.81 (0.07) 0.996 (0.001)
4.85 (0.10) / 0.907 (0.001)
4.12 (0.19)
0.968 (0.001)
3.55 (0/06) / 0.095 (0.001)
sizes of prediction sets for four different CP methods (RANK, RAPS, SAPS, TOPK) on the evaluation set and test set under various attacks and their corresponding defense models. This is used to verify that the process is commutative. It can be seen that the sizes of prediction sets generated on the two datasets are basically consistent, meaning that our evaluation set is fully commutative with the test set. This mutually corroborates our theory of Nash defense.
Distribution of Prediction Set Size under Nash Attack (CP Method: RANK, Dataset: CIFARRO)
7.50
Minizum Muss Size Defezies) (354)
7.00
6.75
6.50
6.25
5.75
5,00
Ambpsreectiiioo..o J004
wehesheettttppp acunda
Unidon
Defenue Strategy
Figure& 8: Box plots of size for different defense models using the RANK method on the CIFAR-10 dataset.
17

---

Published in Transactions on Machine Learning Research (01/2026)
Table 3: Robustness against Unknown Attacks (Generalization) on CIFAR100. Values denote Size (Std)
/ Coverage (Std).
Excluded Astack(s)
APS
RANK
RAPS
SAPS
TOPK
Leave--One-Dut (LOD)
Indieatee
Stae
Comcuge
Stae
Coverage
Size
Coverage
Size
Coyerage
Size
Covesage
APGD
51.94 (1.37) 11.508 (0.001)
64.01 (0.916 0,3559 (C.U21)
02,00 (1.36) 0,338 (0.001)
63.36 (0.55) /0.999 (0.0011)
61.26 (0.32) /0.599 (0.1K1)
CW
52.29 (1.36) 0..599 (0.001)
64.01 (0.96) 00,959 [0.001]
52.34 (1.36) /0,999 (0.001)
63.56 (0:57) /0.999 (0.001)
64.26 (0.52) /0:509 (0.001)
PGSM
(1.35)
(0.002)
64.00 (0.96)
(0.002)
54.00 (1.38) 0.088 [0.002)
63.59 (0.58)
(0.002)
61.26 (0.92)
/0:981 (0.002)
GN
57.50 (1:52) 00000 (0.002)
64.07 (0:06) 0.090 [0.001]
57.55 (1.32) 0.002 (0.002)
63.00 (0:57) 0.900 (0.001)
61.26 (0.02) /0.900 (0.001)
PGT
54.08 (1.96) 00.663 (0.002)
64.08 (0.96) 0.963 (0.002)
54.00 (1:36) 0.983 (0.002)
63.57 (0:57) 0.985 (0.002)
01:26 (0.92) /0:885 (0.000)
PIECSM
21.76 (0.77) 00.692 (0.012)
27.46 (1.15) 0.090 (0.012)
24.79 (0.89) 0.092 (0.014)
27.22 (1.13) /0.600 (0.012)
27.70 (1.12) 00.691 (0.012)
SPSA
03.50
00.00 (0.001)
63:59 (0.91)
(0.1001)
51.51 (1,47) /0.98 (0.0013
63.00
0.908 (0.001
65.25
/0.008 (0.0001)
Squire
51.40 (1.35)
(..09 (0.001)
64.00 (0.91) 0.950 (0.001)
51.43 (1.36) 0.998 (0.001)
63,33 (0.50) / 0.999 (0.001)
64.25 (0.93)
/0,999 (0.001)
Soeree-Two-Oose (LTOO
Indicatoc
Stoc
Comergge
Stase
Coveraaee
Sizee
Covetage
Size
Conerage
Stoe
Corrrage
APGD CW
12:15 19] 00.008 (0.001)
64.01 (0.95) 0.900 (0.001)
52.18 (1:37) 0.935 (0.001)
63.49 (0.57) /0990 (0.001)
61.28 (0.96) /0.589 (0.001)
APGD FG833
03:18 (1.84) 00.000 (0.000)
64.02 (0.95)
(0.009)
68.22 (1.22) 00.990 (0.009)
63.51 (0.57) /0.900 (0.009)
64.28 (0.96) 00.000 (0.000)
APGD GIN
54.76 (3.12) 0.505 (0:0044
64.04 (0.15) 0.3994 (0.015)
54.79 (8.11) 00.90 (0.004]
69.55 (0.57) 00.94 (0.000)
61.25 (0.95)
00.990 (0.000)
APGD PGD
53.00 (1.74) 0.5911 (0.008)
64.02 (0.95) 0.991 (0.008]
50.06 (1.72) 00.99: (0.008)
63.50 (0:57) 0.991 (0.008)
61.25 (0.96) 00.991 (0.008)
APGD PIFGSM
22:90 (2.05) 00.300 (0.139)
27.42 (1.11) 0.531 (0.1
22.90 (2,07) 00,829 (391
27.05 (1.16) 00833 (0.145)
27:67 (1.07) 0.5531 (0.142))
APGD SPSA
01:77
(0.0001)
64.00 (0.95)
(0.0001
51.75 (1.28) 00,958 (0.001)
63.188 (0.50) /0.98 (01001)
16.22% (0.56) 00.008 (0..001)
APGD Squaze
51.74 (1.35)
(0.001)
64,00 (0.95) 0,950 [0:001]
51.72 (1,32) 0..99 (0.001)
63.39 (0.55) 0.909 (0.001)
64.25 (0.95) 005900 (0.000)
CW + IGSM
53.35 (1.73) 0.500 (0.000)
64.02 (0:05) 0.050 (0.009]
53.33 (1.71) 0.000 (0.009)
63.80 (0.96) 0.900 (0.000)
61.28 (0.96) 0.500 (0.000)
CW + GN
54.93 (2.07) 0.595 (0.004)
64.04 (0.05) 0.054 (0. 0051
54.96 (2:96) 0.005 [0.004)
63.65 (0.56) 0.501 (0.000)
61.28 [0.06] 0.504 (0.004)
CW PGD
13:19 (1.04) 0.5001 (0.008)
64.02 (0.05) 0.501 (0.008)
68.23 62) 0.991 [0.005]
63.5p (0.56) 0.991 (0.008)
61:25 (0.96) 00:001 (0.008)
CW PIRGSM
23.02 (1.94) 0.880 (0.1440)
(1.12) 0.858 (0.145)
0.881 (0.140)
(1.15) 00.33 (0.146)
27.07 (1/07) 0.553 (0.145)
CW SPSA
03.94 (1.41) 0.50M (0:0011)
64,00 (0:H0)
1,2006 (C.001)
51.91
(1,17)
6.935 [0.001]]
63.48 (0.55) 0.0M8 (0.0011
64.29 (1636) 0.908 (1.0801)
CW + Square
51.91 (1.42) 0.598 (0.001)
64.00 (0.00)
0,959 (0.0011
51.88
(1:35) 0.998 (0.001)
60.49 (0.56) 0.999 (0.001)
64.29 (0.95) 0.599 (0.000)
PGSM- GN
15:96 (2.10) 0.566 (0.006)
64.05 (0.95) 01385 (0.0005)
60.90 (208) 0.386 (0.006)
63,67 (0.56) 0.985 (0.005)
61.28 (0:33) 0:366 (0.000)
PGSM PGD
54.222
(0.000)
64.00 (0.15) 18011 (0.0000)
54.25 (1.66)
(0.002)
63.66 (0.56) 0.982 (0L0055)
644 (0.96) 0:092 (1L1000)
PGSM PIFGSM
22.77 (1.20) 0.785 (0.108)
25.76 (1.94] 0-782 (0,100))
22,81 (1.
0.786 (0.107)
25.38 (0.96) 0.782 (0.109)
26,03 (1/00) 0,753 (0,110))
PGSM- SPSA
52.97 (1.07) 0.560 (0.000)
61.01 (0,05) / 0,800 0000
52.05 (1.94] 0,089 (0.000) 009)
63.19 (0.56) 0.900 (0.000)
64.25 (0.98) 0.900 ( (000))
PGSM- Square
52.94 [1.001 0.500 (0.000)
64.01 10.05) 0, 00000 00 0001
52.02 (1 96) 0.000 (0.000)
63.51 (0:56) 0.900 (0.000)
61.26 (0.06] 0:900 (0.000)
GN PGD
15.60 (2.22) 0.567 (0.006)
64.05 (0.05) 0.367 (0.001)
55.84 (2:21) 0.367 (0.005)
63.06 (0.56) 0.967 (0.00-)
61.25 (0.94) 0:967 (0.000)
GN PIPGSM
21.86
0.816 (0.126)
27.44 (1.12) 0,809 (0.120)
24.97
(0.126)
27.15 (1.11) 00,00 (0.120)
27.73 (1/01) 0,500 (0.120)
GN SPSA
54.35 (3.52) 0.005 (0.003)
64.00 (0.05) 0.994 (0.004)
54.52 (3:30) 0.995 (0.005)
63.54 (0.90) 0.901 (0.00:)
61:20 (0.95) / 0.994 (0.000)
GN+ + Squae
54.32 (3.34) 00.505 (0.001)
64.00 (0.95) 0.504 (0.005)
54.50 (5:38) 0.995 (0.004)
63.56 (0.56) 0.594 (0.004)
61.20 (0.56] / 0.504 (0.00))
PGD- PIPCISM
23.63 (1.54) 0.797 (0.108)
27.43 (1.11) 0.797 (0.109)
23.74 (1.25
0.798 (0.107)
27.19 (1.12) /0.977 (0.109)
27.73 (1.08) 0.798 (0.108)
PGD- SPSA
52.81
0..00 (0.103)
64.01 (0.16)
(0.1005)
52.79 (1.80) 0,390 (O.0IS)
63.48 (0.56) 0.991 (OL00N)
64:29 (0,56) 0.0001 (0L,009
Square
(1.85)
(0.008)
64.01 (0.95)
(0.008)
52.76 (1.55)
(0.008]
63,50 (0.55)
(0.008)
64.29 (0.95) 00:911 (0.000)
PIEGSM + SPSA
22-48 (2,58) 0.828 (0.139)
27 40 (1.12) 0.832 (0.143)
22.59 (2.35) 0.829 (0. (0.1
26, 13.96 (1.14) 0.831 (0.143)
27.72 (1.08) 0.832 (0.1
PIFGSM Square
22:50 (2.57) 0.830 (0.1400)
27 40 [1.121 0.833 (0.1445)
22.59 (2:34) 0.8331 (0.1440)
26.97 [1 (3) 0.833 (0.145)
27.72 (1.00 0.853 (0.145)
SPSA+ Square
51.47 (1.36) 0.598 (0.00L)
63.59 (0.95) 0.958 (0.001)
(0.001)
63,32 (0.58) 0.908 (0.001)
64.26 (0.95)
0:598 (0.000)
Distribution of Prediction Set Size under Nash Attack (CP Method: TOPK, Dataset: CIFAR10)
7.25
Minitman Muss Sice of (8.67)
6.75
Expeced
6.50
6.00
5.75
5,50
Chanltitee Jone 36,00
Gebtuenngee Inits
Defmae Strategy
Figure 9: Box plots of size for different defense models using the TOPK method on the CIFAR-10 dataset.
A.5 CIFAR100 results
In this section, we provide supplementary payoff matrices (on the evaluation set) and payoff matrices (on the test set) for other methods applied to the CIFAR-100 dataset, along with corresponding results such as box plots.
Figures 20 to 23 respectively present the Nash attack strategics and Nash defense strategics derived on the validation set for four different CP methods (APS, RANK, RAPS, TOPK). Figures 24 to 27 are box plots showing the results the CIFAR-100 test set using APS, RANK, RAPS, and TOPK CP methods. The
18

---

Published in Transactions on Machine Learning Research (01/2026)
Calculated Nash Equilibrium Strategy (CIFAR10 RAPS)
Defender's Nash Strategy
Attacker's Nash Strategy
10
0.38
41-
68-
Probubility
Probability
0.5
0.51
0.07
Addepalla222iiiiie det RN18
Debender XCIT-S12
Xuen23Esphriigt WRN-28-10
APGD
CW
PGSM
GN
PGD
PIFGSM
SPSA
Square
dea
Chmzttaaaaaa.. WRN 34,
Figure 10: Obtain the Nash attack and Nash defense strategies the CIFAR-10 dataset using the RAPS method.
Calculated Nash Equilibrium Strategy (CIFAR10 SAPS)
Defender's Nash Strategy
Attacker's Nash Strategy
0.
Probability
Probability
0.5
0.55
0.45
0.43
Addepeltanzaridddd... RN18
Catoeiarroupel WRN-34-10
Debetedenzazzz ight XCTT-112
APGD
CW
FGSM
GN
PCD
PIFGISM
SPSA
Square
dren
Chen2224Daaa..... WRN 34
Figure 11: Obtain the Nash attack and Nash defense strategics the CIFAR-10 datasct using the RANK method.
figures plot the sizes of prediction sets, and it can be observed that, compared to single defense models, the Nash defense generates the smallest prediction sets. When the Nash defense strategy single one, we plot the results for type of attack. When the Nash attack employs mixed strategy, separately display
19

---

Published in Transactions on Machine Learning Research (01/2026)
Calculated Nash Equilibrium Strategy (CIFAR10 RANK)
Defender's Nash Strategy
Attacker's Nash Strategy
1.00
1.00
10
68-
Probubility
Probability
0.5
Addepalla222iiiiil det RN18
ChmzstaDaaa... WRN. 10
Cai2t:3Decoppeed WRN-34-10
Debensdet Ight XCTT-112
APGD
CW
RGSM
GN
PGD
PIFGSM
SP5A
Square
dea
Figure 12: Obtain the Nash attack and Nash defense strategies the CIFAR-10 dataset using the RANK method.
Calculated Nash Equilibrium Strategy (CIFAR10- - TOPK)
Defender's Nash Strategy
Attacker's Nash Strategy
1.00
0,
Probabiltty
Probability
0.5
Addepeltanzaridddd... RN18
DarpodaDroupeel WRN-34-10
Debeteden ight. XCTT-512
APGD
CW
FGSM
GN
PCD
PIFGISM
SPSA
Square
dren
Chen2 WRN 34
Figure 13: Obtain the Nash attack and Nash defense stratogics on the CIFAR-10 dataset using the TOPK method.
box plots of the prediction set sizes for different methods under one type of attack within the mixed strategy. Figures 28 to 31 illustrate the sizes of prediction sets for four different CP methods (APS, RANK, RAPS, TOPK) the CIFAR-100 evaluation and test set under various attacks and their corresponding defense
20

---

Published in Transactions on Machine Learning Research (01/2026)
Diariilstion of Predicion Set Siee uider ON Attack
Distribatinn of Prodiction Size under PEF GSM Antack
Disrilutiinn of Predittion Siee under Nash Antack
70
770
A28
Stang
Figure 14: Box plots of size for different defense models using the RAPS method on the CIFAR-10 dataset under various attacks.
nieriuuiion of Predicion Sei Size mnder EN Atteck MeBool CIFAR19)
Ditribatioo of Prediction Set Size under GIEG Atrack
Dierthution of Prediction See Size under Nach Attick Meched:
CIEAR
Plaan Max
10
20
20
Figure 15: Box plots of size for different defense models using the SAPS method on the CIFAR-10 dataset under various attacks.
Payoff Matrix on Eveluation Set (CIFARIO- RANK)
Payoll Matrix on Test Set (CIFARIO RANK)
Evallation Set Size
Test Size
Iakkpull0022200iannn... RN08-
6.04
6.03
6.04
6.04
6.04
6.06
6.02
6.03
6.02
Addepdii:22Ellldar.nn RNLB
6.04
6.02
6.04
6.04
6.04
6.07
6.02
6.02
6.02
6:39
6.38
6.53
6.56
6.52
6.60
6.40
6:30
6.38
Delems Model
6.39
5.38
6.52
6.55
6.11
6.57
5.40
6.39
5.38
Deferse
DEs WEN 3..0
5.33
5.33
5.43
5.47
5.42
5.53
5.31
5.30
5.30
Checii WAN 11.18
5.32
5.33
5.42
5.46
5.40
5.51
5.30
5.30
5.30
5.36
5.37
5.42
5.50
5.42
5.49
5.35
5.34
5.37
5.38
5.43 5.49
5.42
5.49
5.36
5.35
5.35
5.16
5.15
5.24
5.17
ARGD
Squar
daab
Arra
Attekk Medol
Mitholl
Figure 16: Evaluate the payoff matrix on both the CIFAR-10 dataset's 's evaluation and test set using the RANK method.
models. This is used to verify that the process is commutative. It can be seen that the sizes of prediction sets generated on the two datascts are basically consistent, indicating that our evaluation set is fully commutative with the test set.
Table | presents the performance of various CP methods under the Nash defense strategy on the CIFAR- 100 datasot. The table shows the average values and standard deviations of coverage, size, and SSCV for series of attack strategics. can be observed that under Nash attacks, the generated prediction sets are consistently the largest.
21

---

Published in Transactions on Machine Learning Research (01/2020)
Payoff
Evalnation (CIFARIO.. RAPS)
Payoff on Test (CIFARIO RAPS)
Evalustion Set Size
Tex Size
Adkeeeli0025000mmm RN18.
5.12
5.09
5.39
5.61
5.36
5.69
5.08
5.07
5.07
Aooepiii022Eiicberr RNIR
5.13
5.10
5.30
5.61
5.34
5.06
5.07
5.07
5.07
Define Model
4.67
5.32
4.98
5.38
4.66
4.70
4.63
4.62
Deberadentizzi MArA512
4.23
4.29
4.64
5.70
4.57
4,99
4.20
4.18
4.17
4.27
4.31
4.55
5.73
4.58
5.00
4.22
4.22
4.20
Suzeensploong WRN-2-110
6.14
7.15
7.39
6.46
7.20
6.96
6.45
6.42
6.26
6.08
7.07
7.33
6.30
7.13
6.87
6.42
6.36
6.23
AR38
AP30
RGSM
Sgat
Amack Mehhod
Amac Method
Figure 17: Evaluate the payoff matrix both the CIFAR-10 dataset 's evaluation set and test set using the RAPS method.
Payoff Matrix Evaluation (CIFAR10 SAPS)
Payoff Matrix 'ees Set (CIFARIO SAPS)
Test
5.24
5.49
5.16
5.34
5.19
Define Malel
6.11
6.05
6.69
6.87
6.64
6.62
6.14
6.07
Delane Model
6.08
6.05
6.63
6,80
6.57
6.08
6.14
6.15
6,06
Cara2224DNAA WRN 14.10-
4.92
5.15
5.25
5.26
5.21
5.48
4.88
4.50
4.85
ChandteDbaa WEX 34.1--
4,90
5.12
5.22
5.23
5.19
5.48
4.87
4.89
4.94
CaNIERDeeegiet WEN---In-
4.83
5.14
5.10
5.13
5.07
5.30 4.79
4.82
4.75
CHINCIDeeogge 1RRN-66-8---
4.83
5.13
5.00
5.11
5.04
5.28
4.78
4.79
4.75
Ddkmeis02221 XC775 X171-12-- 4.61
4.99
4.93
5.26
4.91
5.13
4.52
4.59
4.49
4.60
4.60
4.90
4.93
5.25
4,90
5.13
4.53
4.59
4.49
Ahei
Ch
RGMM
1000
Pircau
dese
Araa
Ambck Method
Anack Mechod
Figure 18: Evaluate the payoff matrix on both the CIFAR-10 datasct's 's evaluation set and test set using the SAPS method.
Payoff Matrix on Evaluttio Set (CIFAR10 TOPK)
Payoff Matrix Test Set (CIFARIO- TOPKK
Evalnation Set Size
Tex Size
6.14
6.15
6.13
6.14
6.13
6.14
6.14
6.13
6.14
5.14
5.14
6.14
6.15
6:13
6.15
6.15
6.14
5.14
6.53
6.54
6.64
6.54
6.54
6.53
6.53
6.53
6.53
Dufinss Model
6.55
6.53
6.54
6.55
6.5 6.53
6.56
6.53
0.55
GesbttADNu WI 5.80
5.79
5.79
5.80
5.88
5.R0
5.79
5.79
5.80
ChandiADe WIRN WWN S4L1
-
5,78
5.79
5.78
5.79
5.79
5.79
5.79
5.80
5.74
5.84
5.83
5.84
5.83
5.83
5.84
5.84
5.86
5.84
5.82
5.83 5.84 5.84 5.84
5.63
5.83
5.83
xOrFS12.
5.56
5.55
5.57
5.57
5.55
5.57
5.56
5.55
ARGi
Squm
doe
Amri
RoaM
Method
Figure 19: Evaluate the payoff matrix on both the CIFAR-10 dataset's evaluation set and test set using the TOPK method.
A.6 ImageNet results
In this section, we provide supplementary payoff matrices (on the evaluation set) and payoff matrices (on the test set) for other methods applied to the ImageNet dataset, along with corresponding results such box plots.
22

---

Published in Transactions on Machine Learning Research (01/2026)
Calculated Nash Equilibrium Strategy (CIFAR100 -PPS)
Defender's Nash Strategy
Attacker's Nash Stralegy
1.0
Probabilitt 64
Probability
ChecitaDaes 3482 14,
Debandent2zz2 fitt XOTLL
Pany B02 Moriitstt WRN199
AND
KGSM
PCD
PIFGSM
SPSA
Spuar
dlop
Addepaiin Motteie
Figure 20: Obtain the Nash at tack and Nash defense stralegies on the CIFAR-100 dataset using the APS method.
Calculated Nash Equilibrium Strategy (CIFAR100 RRANK
Defender's Nash Strategy
Attacker's Nash Strategy
120
1.00
13-
Probabiiity
Probability
Clmaig4Daiee WEN 34,
Debmeenntacct... liMt XOTEL1
APGD
CW
RGSM
PCD
PIFOOd
SP5A
deae
Figure 21: Obtain the Nash aakkkk and Nash defense strategies on the CIFAR-100 dataset using the RANK method.
Figures 32 to 36 respectively present the Nash attack strategies and Nash defense strategies derived on the validation set for five different CP methods (APS, RANK, RAPS, SAPS, TOPK). Figures 37 to 41 are box plots showing the results on the ImageNet test set using APS, RANK, RAPS, SAPS, and TOPK as CP methods. Figures 42 to 46 illustrate the sizes of prediction sets for five different CP methods (APS, RANK, RAPS, SAPS, TOPK) on the evaluation set and test set under various attacks and their corresponding defense models. This used verify that the process commutative. It can be observed that the sizes of prediction sets generated on the two datasets are basically consistent, indicating that our evaluation set is fully commutative with the test set..
23

---

Published in Transactions on Machine Learning Research (01/2026)
Calculated Nash Equilibrium Strategy (CIFARI00 RAPS)
Defender's Nash Strategy
Attacker's Nash Stralegy
1.0
Probabilitt 64
Probability
ChecitaDDes 1482 14,
CatutiDeounphdd.c WRB 3-10
Pebandent2222... fiit XOTLL
Pany B0DOR derittttt WRN199
AND
KGDM
PCD
PIFGSM
SPSA
Spuar
dlop
Addepaii2 Motteie
Figure 22: Obtain the Nash attack and Nash defense strategies on the CIFAR-100 dataset, using the RAPS method.
Calculated Nash Equilibrium Strategy (CIFARI00- TOPK)
Defender's Nash Strategy
Attacker's Nash Strategy
0.71
Probubility
Probability
0.14
Adkalllroziiitcan..... RNN28
Delmokhuzzzz..c RGGFLL
Paep Pagstenii Aasttenn Wartio 16
APOD
Clp
PGDM
DN
PDD
PIPCSM
SPS1
Spaar
clop
Cheiiddas wKl
CacaEtDRoosyle did WRX
Figure 23: Obtain the Nash attack and Nash defense strategies on the CIFAR-100 datasct using the TOPK method.
Table 5 demonstrates the performance of various CP methods under the Nash defense strategy the ImageNet dataset. The table presents the average values and standard deviations of coverage, size, and SSCV for a range of attack strategies.
24

---

Published in Transactions on Machine Learning Research (01/2026)
Table 4: Performance of various Conformal Prediction (CP) methods under the Nash defense strategy the CIFAR-100 dataset. The table presents the mean and standard deviation of coverage, size, and SSCV against a range of attack strategies. All results are averaged over 20 independent random splits.
Attacks
Indicator
APS
RANK
RAPS
SAPS
TOPK
Coverage
0.996 (0.003)
0.997 (0.001)
0.996 (0.003)
0.997 (0.001)
0.997 (0.001)
Clean
Size
49.198 (2.038)
62.059 (1.023)
48.881 (1.960)
61.097 (0.985)
62.304 (1.010)
SSCV
0.100 (0.001)
0.097 (0.001)
0.100 (0.001)
0.097 (0.001)
0.097 (0.001)
Coverage
0.980 (0.003)
0.977 (0.002)
0.979 (0.002)
0.977 (0.002)
0.978 (0.003)
FGSM
Size
53.423 (1.117)
62.123 (1.021)
53.238 (1.045)
61.535 (0.994)
62.305 (1.005)
SSCV
0.094 (0.008)
0.077 (0.002)
0.093 (0.008)
0.077 (0.002)
0.078 (0.003)
Coverage
0.981 (0.003)
0.978 (0.002)
0.981 (0.003)
0.978 (0.002)
0.979 (0.003)
PGD
Size
52.955 (1.181)
62.113 (1.022)
52.757 (1.106)
61.496 (0.991)
62.302 (1.008)
SSCV
0.095 (0.007)
0.078 (0.002)
0.095 (0.007)
0.078 (0.002)
0.079 (0.003)
Coverage
0.996 (0.003)
0.997 (0.001)
0.996 (0.003)
0.997 (0.001)
0.997 (0.001)
APGD
Size
49.999 (1.845)
62.092 (1.022)
49.706 (1.763)
21.233 (0.985)
62.302 (1.010)
SSCV
0.100 (0.001)
0.097 (0.001)
0.100 (0.001)
0.097 (0.001)
0.097 (0.001)
Coverage
0.996 (0.003)
0.997 (0.001)
0.996 (0.002)
0.997 (0.001)
0.997 (0.001)
CW
Size
52.307 (1.283)
62.099 (1.020)
52.218 (1.265)
61.798 (1.001)
62.303 (1.007)
SSCV
0.100 (0.001)
0.097 (0.001)
0.100 (0.001)
0.097 (0.001)
0.097 (0.001)
Coverage
0.898 (0.009)
0.898 (0.006)
0.897 (0.008)
0.898 (0.008)
0.896 (0.008)
PIFGSM
Size
55.500 (1.156)
62.133 (1.026)
55.291 (1.099)
61.610 (0.990)
62.305 (1.008)
SSCV
0.143 (0.091)
0.005 (0.004)
0.127 (0.068)
0.006 (0.005)
0.007 (0.006)
Coverage
0.991 (0.002)
0.988 (0.001)
0.991 (0.002)
0.988 (0.001)
0.988 (0.001)
GN
Size
57.335 (1.154)
62.166
(1.022)
57.225 (1.113)
61.725 (0.970)
62.302 (1.008)
SSCV
0.100 (0.000)
0.088 (0.001)
0.100 (0.000)
0.088 (0.001)
0.088 (0.001)
Coverage
0.996 (0.003)
0.997 (0.001)
0.996 (0.003)
0.997 (0.001)
0.997 (0.001)
Square
Size
49.391 (1.940)
62.068
(1.022)
49.086 (1.862)
61.198 (0.988)
62.305 (1.010)
SSCV
0.100 (0.001)
0.097 (0.001)
0.100 (0.001)
0.097 (0.001)
0.097 (0.001)
Coverage
0.995 (0.003)
0.996 (0.001)
0.995 (0.003)
0.996 (0.001)
0.996 (0.001)
SPSA
Size
49.487 (1.909)
62.060 (1.022)
49.184 (1.829)
61.131 (0.986)
62.308 (1.006)
SSCV
0.100 (0.001)
0.096 (0.001)
0.099 (0.001)
0.096 (0.001)
0.096 (0.001)
Coverage
0.991 (0.002)
0.979 (0.029)
0.991 (0.002)
0.996 (0.004)
0.985 (0.024)
Nash
Size
57.335 (1.154)
62.166 (1.022)
57.225 (1.113)
61.814 (0.967)
62.302 (1.006)
SSCV
0.100 (0.000)
0.080 (0.025)
0.100 (0.000)
0.096 (0.004)
0.086 (0.019)
25

---

Published in Transactions on Machine Learning Research (01/2026)
Distribution of Prediction Set Size under Nash Attack
(GP Method: APS, Dataset: CIFAR 100)
Masiuun Daloaa (57.355)
70
Expected Size
02
00
Alepan 1010
Cher Wett St,
Anteto 18
Callea
Defense Strategy
Figure 24: Box plots of size for different defense models using the APS method on the CIFAR-100 dataset.
Distribution of Prediction Set Size under Nash Attack (CP Method: RANK, Dutaset: CIFAR100)
Maxianaa
80
Expected Size
70
65
Adkyttatttelion 2010
Demert tgle Arara.r
Doe Wel
Defriin Strtngy
Figure 25: Box plots of size for different defense models using the RANK method on the CIFAR-100 dataset.
Distribution of Prediction Size under Nash Attack (CP Method: RAPS. Dataset: CIFAR100)
Minimen Miee Size et Ditmars (57.223)
70
Experte Size
00
56
Adbroir wororaatiiia....ooo
west
wwghtttaeasttee... Miterr
Defiase Straegy
Figure 26: Box plots of size for different defense models using the RAPS method on the C1FAR-100 dataset.
26

---

Published in Transactions on Machine Learning Research (01/2026)
Biviriuuiion of Predictio Set Siee under APGO Attack
Datsibatioo of Predicioo Sei Sice under GN Attack CIFAR10)
Ditehuiiion of Preditiion Se: Sise under Nark Attack CIFARI00)
CITAR10))
Defece Sriner
Figure 27: Box plots of size for different defense models using the TOPK method the CIFAR-100 dataset under various attacks.
Payoff Matrix on Evaluation Set (CIBAR100 APS)
Payoff Matrix on Test (CIFAR100 APS)
Evaluation Net Size
Tist Set Size
Addeppiliteccasssteeee... KN1B-
88 54 76 66
68.62
66.33 39 68
12 64.63 64 58 64.
Acdegallo2220Meeerr... 55.27 65. 66.95 $9.07 66. 68.51 65.02 64 64.93
Chey222DDes WAN 50.47
50,78
52.87
55.87
52.55
55.05 50.02 49.97 49.88
Dufess Modkl
Cher WRN 50.85 51 53.34
56.68 53.02 55.61 50.41 50.36 50.26
DebteiDee 409.70
53.57
47.71
Cebieeeeooppool.ccoocooooooooo. WRN
Deberake -5.119
51.86
53.49
64.15
55.25
Desnekiinnciikk 51..56
51.20
66.57
67.0
68
72.45
68,
70.01
20
31
66
PwegtozRebaeeeee WHNNLLE
56.97
67 43 60.04
73.04
68.55
70.54
68, 66 68 66.58
APhb
Arrb
Attick Method
Anack Method
Figure 28: Evaluate the payoff matrix on both the CIFAR-100 datasct's evaluation set and test set using the APS method.
Payoff Matrix on Evaluation Set (CIFAR100 RANK)
Payoff Matrix on Test Set (CIFAR100 RANK)
Evaluation Set Size
Test Set Size
Mirpliii22iiiie RNIR
72.04
72.04
F2.04
72.05
72.06
22.05
72.07
P2.03
02 72.03
AdkgaBiin RNI
72.05 72.04 72.06
72.07 72.05
72.07 72.03 72. 24 72.04
Gle WRN_3UU
63.18
63
63.21
63.26
63.21
03 26 63.15 63 63.15
Defisse Model
Can20040e WRN
63.27
63.33
63.35
63.37
63.39
63.29
63.29
Deinnkkiieesiikk XCIFL -64.20
64.20
64.23
64.52
64
6426 64.18
64. 64.18
Dengaiiiiddaasmms.... WHNNTLLS
77.20
19
77
77
77.21
77
23
77.18 77,
77.18
Pesgtczaaoaatsen. WHNNLIS 77.21
77
20
77.23
77
77.24
77.20 77. 77.20
Amrck Mehhod
Attrk Method
Figure 29: Evaluate the payoff matrix on both the CIFAR-100 dataset's evaluation set and test set using the RANK method.
27

---

Published in Transactions on Machine Learning Research (01/2026)
Payoff Matrix Pvaluation Set (CIFAR100
Payoff Matrix
Test (CIFARIOO RAPS)
68.60
64.50
Chesbi24DNus WEN -5.0-550.25
50.
52
55.67
52.32
54.83
49.79
49.72 49.05
Deline Model
51.01
56.45
52.79
55.
40
50.18
50.13
50.03
Delene
Cabl2iDeoomloo WAN 34-0-44--0-----
488.61
53.14
53
57,89)
52,74
55.39
48.08
47.93 47.58
-49.45
54.29
54.06
59.12
53.40
R8
48.81
48 09 48.35
Dete XOTICI XCTTCI2
-5.28
28..25555 51.95
53.57
64.22 53.10
56.31
00 50.99 50.50
Dexneski2ee ii XCILI
51.63
03.63.555
52
53.82
64.97
53.34
56.55
51.35
51. 36 51.27
67. 18
68
72.50 19 mi.0 ns 66..35 66.37 se 28
DesgooCRroeetnn WRNNLIS 57 67
68.11
F300 6R 70.58 66.73 66, 66.
W9SA
dees
C9
Road
TRYCOM
Sqpan
Attack Method
Amad: Method
Figure 30: Evaluate the payoff matrix on both the CIFAR-100 da dataset's 's evaluation set and test set using the RAPS method.
Payoff Matrix en Evaluation Set (CIFAR100- TOPK)
Payoff Matrix on Test Set (CIFAR100 TOPK)
Evaluation Set Size
Test Set Size
72.10 72.1 72.11 72 72 72.12 72.11 72.11 22.11
A60gg RNI
72.10 72.11
72.10
72.10
72 72.11
72.10
72 22.11
ChesstitDee W 63.61 63.60 63.50
63, 63.61 63.50 63.61 63.58 63.59
Dadesse Model
Dicc XCIT -4.55
64.60
64.60
64
64.59 64.60 39
64.59 64.60
Dexnnkii20C3... XCTLL 664.99 64.61
64.58
64.60
64.61
64.59
64.60
64. 64.60
Dempar/iiedrattteen... WINNH 77.27 17,7771 77
27
77 29
77 77.27 77.27
77.27
Peegtic2hiuuussmn. WHNNLLIS 77.28
77
28
77.27
27
7,26 77, 27 77.27
APCD
Attack Methed
Attakk Method
Figure 31: Evaluate the payoff matrix on both the CIFAR-100 dataset's evaluation set and test set using the TOPK method.
Calculated Nash Equilibrium Strategy (IMAGENET APS)
Defender's Nash Strategy
Attacker's Nash Strategy
1.00
1.20
probability
Probability
Dhadrraaaaa WiN 59.2
Sabondattizzaiee be XGTMI22
66
AFOD
CW
RGM
G8
PGD
FRRSSM
0
daa
Figure 32: Obtain the Nash attack and Nash defense strategies on the ImageNet dataset using the APS method.
28

---

Published in Transactions on Machine Learning Research (01/2026)
Table 5: Performance of various Conformal Prediction (CP) methods under the Nash defense strategy on the ImageNet dataset. The lable presents the mean and standard deviation of coverage, size, and SSCV against a range of attack strategies. All results are averaged over 20 independent random splits.
Attacks
Indicator
APS
RANK
RAPS
SAPS
TOPK
Coverage
0.955 (0.001)
0.937 (0.011)
0.955 (0.002)
0.938 (0.008)
0.937 (0.011)
Clean
Size
269.403 (4.520)
257.710 (4.929)
269.351 (4.577)
257.284 (4.756)
257.565 (4.827)
SSCV
0.086 (0.003)
0.037 (0.011)
0.086 (0.003)
0.038 (0.008)
0.037 (0.011)
Coverage
0.977 (0.001)
0.963 (0.005)
0.977 (0.001)
0.962 (0.004)
0.963 (0.005)
FGSM
Size
269.117 (4.453)
257.699 (4.942)
269.065 (4.537)
257.111 (4.755)
257.564 (4.826)
SSCV
0.081 (0.002)
0.063 (0.005)
0.081 (0.003)
0.062 (0.004)
0.063 (0.005)
Coverage
0.964 (0.001)
0.951 (0.003)
0.964 (0.001)
0.951 (0.003)
0.951 (0.003)
PGD
Size
266.248 (4.478)
257.689 (4.944)
266.194 (4.567)
257.077 (4.754)
257.565 (4.826)
SSCV
0.072 (0.004)
0.051 (0.003)
0.072 (0.005)
0.051 (0.003)
0.051 (0.003)
Coverage
0.955 (0.002)
0.936 (0.011)
0.955 (0.002)
0.938 (0.008)
0.936 (0.011)
APGD
Size
273.920 (4.421)
257.713 (4.928)
273.868 (4.478)
257.329 (4.757)
257.564 (4.826)
SSCV
0.086 (0.003)
0.036 (0.011)
0.086 (0.003)
0.038 (0.008)
0.036 (0.011)
Coverage
0.986 (0.001)
0.974 (0.004)
0.986 (0.001)
0.974 (0.003)
0.974 (0.004)
CW
Size
278.106 (4.541)
257.706 (4.934)
278.054 (4.597)
257.318 (4.756)
257.565 (4.826)
SSCV
0.090 (0.003)
0.074 (0.004)
0.090 (0.003)
0.074 (0.003)
0.074 (0.004)
Coverage
0.901 (0.003)
0.900 (0.003)
0.901 (0.003)
0.900 (0.003)
0.900 (0.003)
PIFGSM
Size
271.377 (4.892)
257.689 (4.944)
271.325 (5.014)
257.065 (4.752)
257.566 (4.827)
SSCV
0.138 (0.017)
0.003 (0.002)
0.143 (0.019)
0.002 (0.001)
0.003 (0.002)
Coverage
0.987 (0.001)
0.976 (0.003)
0.987 (0.001)
0.975 (0.002)
0.975 (0.003)
GN
Size
253.404 (4.189)
257.686 (4.947)
253.355 (4.246)
257.015 (4.757)
257.566 (4.826)
SSCV
0.092 (0.002)
0.076 (0.003)
0.092 (0.002)
0.075 (0.002)
0.075 (0.003)
Coverage
0.986 (0.001)
0.942 (0.018)
0.986 (0.001)
0.949 (0.020)
0.944 (0.026)
Nash
Size
278.106 (4.541)
257.715 (4.929)
278.054 (4.597)
257.328 (4.758)
257.565 (4.827)
SSCV
0.090 (0.003)
0.042 (0.018)
0.090 (0.003)
0.049 (0.020)
0.044 (0.026)
Calculated Nash Equilibrium Strategy (IMAGENET RANK)
Defender's Nash Strategy
Attacker's Nash Strategy
Probabiiit
Probubility
Cheaatsthei WRN. .0.2
deoeedeezizzzppllee XCMMM
Poog
APGD
CW
FGRM
PGD
PPGSI
Figure 33: Obtain the Nash attack and Nash defense strategics on the ImageNet dataset using the RANK method.
29

---

Published in Transactions on Machine Learning Research (01/2026)
Calculated Nash Equilibrium Strategy (IMAGENET RAPS)
Defender's Nash Strategy
Attacker's Nash Strategy
Probabillty
Probubility
ChocataiDas WiN 80.2
Ddbaedeezzzaiile XOmMI XOeMMme
APGD
GW
RGM
MGD
PIGSS
Figure 34: Obtain the Nash attack and Nash defense strategics the ImageNct datasct using the RAPS method.
Calculated Nash Equilibrium Strategy (IMAGENET SAPS)
Defender's Nash Strategy
Attacker's Nash Strategy
Probability
Probability
Comaatttaea.. WRN 20.2
Dddaatieiiizzzigee sommzz
APGD
CW
RGM
RDD
PIFGMM
Figure 35: Obtain the Nash attack and Nash defense strategies the ImageNet dataset using the SAPS method.
30

---

Published in Transactions on Machine Learning Research (01/2026)
Calculated Nash Equilibrium Strategy (IMAGENET TOPKK
Defender's Nash Strategy
Attacker's Nash Strategy
1.00
100
16
Probability
Probability
Deb anbredattzzza... de XCTMI o
Ohaddtaaaa, Wil 50
APOD
CW
RGM
GB
PGD
PIRPIS
dkun
Figure 36: Obtain the Nash attack and Nash defense strategies on the ImageNet dataset using the TOPK method.
Distribution of Prediction Set Size under Nash Attack Method: IMAGENET)
(278.196)
Expeced Siae
400
350
Stratgyy
Figure 37: Box plots of size for different defense models using the APS method on the ImageNet datasct.
Distribution Prediction Set Size under Nash Attack (CP Method: RANK, Dataset: IMAGENET)
500
Mainun Sie Delases (257711)
450
Experted Size
300
250
Chocouttaa KBV 29.2
Augietiuuen stor
Defence
Figure 38: Box plots of size for different defense models using the RNAK method on the ImageNet datasct.
31

---

Published in Transactions on Machine Learning Research (01/2026)
Distribution of Prediction Set Size under Nash Attack (CP Method: RAPS, Dataset: IMAGENET)
Moa (278059)
Expected Site
400
380
500
Checitsone Wat VRL.
Inlendo airtaat
Gelone
Defense Strategy
Figure 39: Box plots of size for different defense models using the RAPS method on the ImageNet dataset.
Distribution Prediction Set Size under Nash Attack (CP Method: TOPK, Dataset: IMAGENET)
500
Minmmnn Mann Sim Delmaan (22 560)
Expocted Size
350
300
250
Calvey
Defense Strategy
Figure 40: Box plots of size for different defense models using the TOPK method on the ImageNet dataset.
Distribution of Prediction Set Size under Nash Attack (CP Method: SAPS, Dataset: IMAGENET)
Maiman Mon Siee Delases (357 308)
Expected Stee
400
350
Dasion
NCD
Defence Straegy
Figure 41: Box plots of size for different defense models using the SAPS method on the ImageNet dataset.
32

---

Published in Transactions on Machine Learning Research (01/2026)
Payoll Matrix Evaluation (IMAGENET-AI) Evaluation Set Size
Payoff Matrix Test Set (IMAGENET APSS)
Test Set Size
Cosstt240an 20,2
470.41
519.1
506.73 502.8 81 512.19 522. 43 460,09
Chanki WRX_5 471.26
520. 80
508.51 502 56 511.92 52188 470.10
Dulens Modil Modil
Dderddanann2Lggg XOTMII- 408.89
973.11
281.76
260
85
306.87 331.09 09.00
417.90 90
Delessescooiaatk RCTMII- 400899
374, 65) 283.60
262
308.38 333.2
408
PengS12sRerutt 281.80
256.44
270.08
275.52 273.56
Nrngh 276.1
276.81 281. 272.30
274.96
MPED
ROaM
ARGo
Prodd
dess
Figure 42: Evaluate the payoff matrix on both the ImageNet dataset's evaluation set and test set using the APS method.
Payoff Evaluation
RANK)
Payoff Matrix on Set (IMAGENET RANK)
Evaluation
Test Set Size
504.30 504.51 504.51 504.54 504.35
Chan80 WBN 51.2 504.36 504.5 S1 504.52 504.51
504.52
504. 504.35
Defense Model
Deloraa Modal
Derradedtttuuggp.ecre -69.7712222277000 269.61 269.41 260.37 280 289.48 289.70
263.14
263.12 263.12
Pregenteiattu 253.12
263. 263.14 263.13 263.13 26312 263.12
APOD
PROM
PiRcau
Arci
REod
Amack Method
Allack Mithod
Figure 43: Evaluate the payoff matrix on both the ImageNet dataset's evaluation set and test set using the RANK method.
Payoff Matrix on Evaluation Set (IMAGENET RAPS)
Matrix (IMAGENET- RAPS)
Evaluation Set Size
ChadiC4Duu WROL 20 470.27 519.02 508.54 502.67
512.05
522.39 20
498.95
471.13
50R. 38
Delinee Modal
Delens
306.70
406.82
De Dekeaaderzzazziaaak. XC31MM2 407.88 87 373.53
53,
282.50 281.73
307.20
332 407.59
22t.liitttt-88.88888 278.82 282.14 273.26 256.77 270.43 275.87 273.90
Pergh 276.94 281.77 272.63 256.36 269.99 275.32 272.55
Road
AFCO
intaaa
Anack Method
Ataak Method
Figure 44: Evaluate the payoff matrix on both the ImageNet dataset's 's evaluation set and test set using the RAPS method.
33

---

Published in Transactions on Machine Learning Research (01/2026)
Payoll Matrix Evaluation (IMAGENET- SAPS)
Payoff Matrix on Test Set (IMAGENET SAPS)
Evaluation Set Size
Test Set Size
Ch 50.2 503.85 503.86 503 46 503.39 503.49 503.57
5C0 84
CheniiMDer WAN
503.96
500. 57 503.47
503.40
503.49
503.50 503.86
Dutesse Modk!
Delese
Drtendden X0rFMI2 269.40 269.21
208.49
268.36
268.62
268.74 269:38
Debensdes202ZLgge
262.97
262.79 262.59 262.73 262.93
PerggatiCaaatee 262.97 262.57 262.79 262.70 262.76 262 74 262.92
APDD
deal
APGo
RLOM
RRRMM
Attock Methed
Attack Method
Figure 45: Evaluate the payoff matrix on both the ImageNet dataset's evaluation set and test set using the SAPS method.
(IMAGENET
Payoll Matrix on Test Set (IMAGENET TOPK)
Test Set Size
504.70 504. 304 70
304.09
504.70
504.70 504.70
504.70 504 504.70 504.70
504.70
504.70 504.60
Delosse Model
Delene
Siraskki2122.gaa.oo - 269, 269.80 80
269.79 269.80 269.79 269.79
263.22 263.22 263.21 263.21 26321
Preggreeentte 263.22 263. 22 263.22 263.21 263.21 263.1 263.21
APRD
APOD
Rood
mRoal
Atteck Methrd
Attikk Method
Figure 46: Evaluate the payoff matrix on both the ImageNet dataset's evaluation set and test set using the TOPK method.
34


Reviews:
[{"review_id": "cqQUQJV5eZ", "reviewer": "Reviewer_phk5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper tackles the challenge of obtaining reliable uncertainty quantification under adversarial attacks, which frequently break the exchangeability assumptions required for conformal prediction. The authors introduce a framework that integrates conformal prediction with a discrete zero-sum game between an attacker and a defender. By analyzing this interaction, they derive a Nash equilibrium defense strategy that is proven to maintain valid coverage while minimizing worst-case prediction-set size within a predefined attack space. Experiments show that the defense consistently achieves both the desired coverage and smaller, more stable prediction sets", "claims_and_evidence_yn": "Yes", "claims_explanation": "The theoretical claim about preserving coverage using max-quantiles is correct and clearly justified.\n\nThe proposed method and its formulations are correct.\n\nExperiments on CIFAR-10/100 and ImageNet support the claims", "audience_yn": "Yes", "audience_explanation": "The paper connects adversarial robustness and conformal prediction, two active research areas in machine learning.\n\nThe game-theoretic formulation would interest researchers in reliable, uncertainty quantification, and robust deep learning.\n\nIts practical relevance to safety-critical domains (such as autonomous driving and medical imaging) makes it appealing to a wide range of applications.", "requested_changes": "1. The current framework guarantees coverage only for attacks included in the predefined attack set. Adding experiments or analysis on attacks not in the calibration set would strengthen confidence in the method’s practical robustness.\n\n2. Discuss assumptions on known finite attack space. Since the main theoretical guarantee relies on this assumption, adding a brief discussion of limitations and potential extensions would improve clarity.\n\n3. Provide more intuition behind the max-quantile guarantee. A short intuitive explanation or toy example could help readers unfamiliar with conformal prediction understand why the guarantee holds.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "fZR2a5oaZn", "reviewer": "Reviewer_g7zA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "### Strengths\n- The paper formalizes the problem of selecting a robust defense model for Conformal Prediction as a two-player, zero-sum game from a finite, predefined set of attackers and defenders.\n- The paper presents a fairly straightforward motivation, and I truly appreciated the discussion in Section 2 for clarity and formalization of Conformal Prediction. The paper was also relatively easy to understand.\n\n---\n\n### Weaknesses / Questions\n- The theorem stated in the paper and nash equilibrium holds true only if the attacker attacks from a known set of attacks. The issue is attackers often pick attacks which are unknown - which breaks the guarantee / size of the prediction set. Can the authors provide experiments for how the Nash defense holds up against an unknown style of attacks?\n- Why do the authors split the dataset into train and calibration? Why not use only one dataset? \n- Can you explain why Chen-2024-Data consistently does well in Figures 4 and 5?", "claims_and_evidence_yn": "Yes", "claims_explanation": "Yes, majority of the claims seems to be supported by clear evidence. However, a few major questions remain underanswered as shown in the Weaknesses.", "audience_yn": "Yes", "audience_explanation": "Yes, these findings would definitely be interesting.", "requested_changes": "Critical changes are already discussed in Weaknesses. Regardless, there are some minor revisions that need to be made:\n\n- If the captions to each figure (especially to those in the Appendix) were more informative, it would be appreciated.\n- The paper’s citations are not properly formatted, which significantly impacts readability. We kindly request the authors to address this issue to enhance the presentation quality.", "broader_impact_concerns": "Not applicable.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vyXI0e15tm", "reviewer": "Reviewer_kaFT", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a new model and methodology for training and evaluating deep learning models with conformal predictions robust against adversarial attacks. Deep learning models are known to be easily disturbed by adversarially constructed test input, and thus is often unreliable in safety-critical situations; traditional methods used to introduce robustness also suffers from a variety of drawbacks, including large complexity cost, inability to generalize to other adversarial attacks, etc.\n\nTo solve the aforementioned concerns, the authors employ game-theoretical ideas to train and select classifiers that serve as best strategies against a given set of adversarial attack strategies. By formulating the training-evaluating process as a two-player zero-sum game between the deep learning model as the defender and the adversary as the attacker, the authors propose a training paradigm involving solving for the Nash equilibrium and using the randomized strategy of selecting according to the optimal distribution on the family of available classifiers against the adversarial attack, provided that the strategy space for both the defender and the attacker is finite, i.e., the two-player zero-sum game in question is finite.\n\nThe authors provide experimental results evaluating the performance of their strategies on a variety of datasets, and across multiple attack/defense strategies.\n\nWhile the authors' work points out a very promising and interesting direction for research, I am somewhat skeptical of the novelty and the extent of contribution of this submission. Despite reasonably pointing out some significant drawbacks of prior work on adversarially robust conformal predictions, the authors proceed to present a somewhat simple paradigm that does not seem to properly address these concerns, or still exhibits severe limitations, calling my doubts to the technical soundness and novelty of their claims.", "claims_and_evidence_yn": "No", "claims_explanation": "Expanding on my opinions in the summary section: I find the authors' claims on the novelty and contribution of their game-theoretical paradigm rather insufficiently substantiated for a large part.\n\nThe authors' claims, in my understanding, are as follows:\n- Prior works crucially rely on the exchangeability assumption, which is difficult to satisfy in real-world and adversarial scenarios;\n- Prior methods to introduce robustness to conformal predictions suffer from an inability to generalize against unknown attack strategies.\nBoth of which the authors claim to address with their training paradigm.\n\nHowever, none of these crucial points seem to be properly and completely addressed by this submission:\n- The algorithmic framework on page 5 trains, calibrates, and evaluates exclusively on the training data, split into three partitions. The testing data does not seem to play any part before the final classifier-strategy is selected. Intuitively, this calls for the same concerns as prior training paradigms - that should the training set prove insufficiently representing, the optimal strategy can still perform suboptimally on unknown data.\n- The authors' two-player zero-sum game theoretic formulation continues to rely on the assumption that the strategy space for both the model and the adversary are finite, which is a very limiting assumption. When presented with unknown attack strategies outside of the considered attack strategy space, the theoretical guarantees obtained by the framework can fall apart similar to prior works.\nThe submission does not seem to address these observable drawbacks, and from the current manuscript I cannot find confidence in the authors' claims of novelty and contribution.", "audience_yn": "Yes", "audience_explanation": "I do believe that the (un)reliability of machine learning models in safety-critical scenarios is a very important research direction for researchers in the field, and is one that I am quite excited for. Despite the submission's potential lack of novelty, I think the topic of choice and the game-theoretical direction that the authors propose is indeed informative for the audience of TMLR as a whole.", "requested_changes": "As illustrated above, I am unsatisfied with the authors' claim of contribution and novelty; I believe the limitations and drawbacks of their training paradigm and framework should be properly addressed and discussed before it can be properly re-evaluated for acceptance.", "broader_impact_concerns": "I do not believe a broader impact statement is required concerning the nature of this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)