Adversarial generalization of unfolding (model-based) networks
Authors: Vicky Kouni
Venue: NeurIPS 2025 poster ()
Keywords: unfolding networks
adversarial generalization
adversarial Rademacher complexity
Primary area: theory
TL;DR: The paper studies the generalization performance of unfolding networks in the presence of adversarial attacks during training and test time.

Abstract: Unfolding networks are interpretable networks emerging from iterative algorithms, incorporate prior knowledge of data structure, and are designed to solve inverse problems like compressed sensing, which deals with recovering data from noisy, missing observations. Compressed sensing finds applications in critical domains, from medical imaging to cryptography, where adversarial robustness is crucial to prevent catastrophic failures. However, a solid theoretical understanding of the performance of unfolding networks in the presence of adversarial attacks is still in its infancy. In this paper, we study the adversarial generalization of unfolding networks when perturbed with $l_2$-norm constrained attacks, generated by the fast gradient sign method. Particularly, we choose a family of state-of-the-art overaparameterized unfolding networks and deploy a new framework to estimate their adversarial Rademacher complexity. Given this estimate, we provide adversarial generalization error bounds for the networks under study, which are tight with respect to the attack level. To our knowledge, this is the first theoretical analysis on the adversarial generalization of unfolding networks. We further present a series of experiments on real-world data, with results corroborating our derived theory, consistently for all data. Finally, we observe that the family's overparameterization can be exploited to promote adversarial robustness, shedding light on how to efficiently robustify neural networks.

Adversarial Generalization of Unfolding (Model-based) Networks
Vicky Kouni
LAMSADE, Paris Dauphine PSL Research University vasiliki. kouni &dupidda ceciiinne.sll..... psl.
Abstract
Unfolding networks are interpretable networks emerging from iterative algorithms, incorporate prior knowledge of data structure, and are designed to solve inverse problems like compressed sensing, which deals with recovering data from noisy, missing observations. Compressed sensing finds applications in critical domains, from medical imaging to cryptography, where adversarial robustness is crucial prevent catastrophic failures. However, solid theoretical understanding of the performance of unfolding networks the presence of adversarial attacks is still in its infancy. In this paper, we study the adversarial generalization of unfolding networks when perturbed with 2z-norm constrained attacks, generated by the fast gradient sign method. Particularly, the-art overaparameterized unfolding networks and deploy a new framework to estimate their adversarial Rademacher complexity. Given this estimate, adversarial generalization error bounds for the networks under study, which are tight with respect the attack level. To our knowledge, this is the first theoretical analysis on the adversarial generalization of unfolding networks. We further present a series of experiments real-world data, with results corroborating our derived theory, consistently for all data. Finally, we observe that the family's s overparameterization can be exploited promote adversarial robustness, shedding light on how to efficiently robustify neural networks.
choose family of state-of-
provide
Introduction
The advent of deep unfolding networks (DUNs) [17] ushered a new paradigm for inverse problems [37], by transforming iterative optimization algorithms into trainable neural architectures [39]. The starting point of DUNs is that many of these algorithms can be written compactly in a neural network formulation:
X++1 nonlinearity(linear, piterrriiyitiinaarrrassmmmsss// bias_ hias_term).
In this paper, we are interested the inverse problem of Compressed Sensing (CS) [44], modeling plethora of modern applications, from medical imaging and speech processing, to communication systems and cryptography, where robustness crucial for ensuring safe and reliable inference.
CS deals with recovering data from missing, noisy observations, given that the structure of the data sparse via some fixed transform (e-g. wavelets for images). To address CS, a popular approach relies on formulating it LASSO optimization problem, and then using some iterative proximal algorithm to solve it [8]. Since the output of the proximal algorithm a given iteration can be written as in (+), the algorithm's 's iterations can be treated as the layers of DNN, so that the algorithm is "unfolded" into said DNN [17]. While all parameters are fixed in the proximal algorithm, its unfolded counterpart is treated as structured DNN, with unknown and thus learnable parameters, e.g., the sparsifying transform. As such, the sparse data model is inherited by DUNs [4, 24, 42], rendering them as interpretable DNNs with superior reconstruction quality in reduced time [40, 41].
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Interestingly, learnable sparsifying transforms seem to improve the robustness of DUNs against additive noise [25]. with recent empirical studies also focusing on DUNs' robustness against gradient- based adversarial attacks [13, 46]. Despite these advances, a solid mathematical understanding of the performance of DUNs in the presence of adversarial attacks, both during training and test times, remains elusive. Generally, adversarial robustness of standard DNNs is rigorous research topic [18, 31J, with adversarial generalization [29] being at the forefront of recent research interest, as it can be leveraged to control and analyze the robustness of DNNs against adversarial attacks like the fast gradient sign method (FGSM) [16], the projected gradient method (PGD) [27], and their variants [22, 26]. A key tool in this direction is the adversarial Rademacher complexity (ARC) [1], which quantifies model complexity under worst-case perturbations, and enables a structured way of studying the adversarial generalization of DNNs.
Our contributions in the context of related work. Motivated by recent advances in DUNs and adversarial generalization, in this paper, we seek to address the following core question:
What is the generalization performance of DUNs in the presence of adversarial attacks? (*)
To that end, main contributions follows:
From a broader family of DUNs, we select a state-of-the-arr representative, parameterized by an overcomplete sparsifier, leading to an overparameterized regime. Then, we perturb that DUN with FGSM attacks considered both during training and test times constrained in the Io-norm. We differentiate our approach from related work in the following aspect: we prove the Lipschitz continuity of the attacked DUN with respect to the learnable overcomplete sparsifier, and deploy this result as a new cornerstone to estimate the ARC of the DUN.
upper-bound the adversarial generalization error of the examined DUN; to our knowledge, these are the first theoretical results on the generalization performance of adversarially perturbed DUNs. Specifically, our main Theorem, stated informally below, provides a convincing partial answer to (*), and highlights how the overeompleteness of the learnable sparsifier, the number of layers, and the level of the adversarial FGSM-based attack, ripple out to DUN adversarial generalization. To the best of our knowledge, our result improves on state-of-the-art related work [12, 49] in terms of the number of layers and the attack level (cf. Table 1).
We leverage the ARC estimate
Theorem (cf. Theorem 3). With high probability, the adversarial generalization error of the examined DUN roughly scales like (og(E) with L being the total number of layers, N controlling the overcompleteness of the learnable sparsifier, and being the attack level.
We evaluate our mathematical results with numerical experiments on real-world data. Our findings are three-fold: a) the empirical adversarial generalization of the DUN conforms with the theoretical one, consistently for all data, b) overcompleteness promotes adversarial robustness: the higher the N. the more adversarially robust the DUN is to increasing attack levels, c) comparisons with a bascline DUN learning an orthogonal sparsifier highlight the superiority of overcompleteness over orthogonality in the context of adversarial robustness, with our adversarially perturbed DUN outperforming the baseline in all scenarios.
Overall, our proposed study provides the first theoretical framework for understanding and improving the adversarial robustness of DUNs in CS, with direct implications for safety-critical applications. In medical imaging, for example, our results could help ensure that MRI reconstructions remain reliable even under adversarial perturbations, reducing the risk of misdiagnosis. Furthermore, by showcasing that overparameterization via learnable overcomplete sparsifiers improves robustness, our work offers concrete design principles for developing resilient, interpretable networks in real-world deployments, where accuracy, efficiency, and security must coexist.
2 Related work
To place our contributions in context, we review relevant literature in the next paragraphs. (Adversarial) Robustness in DUNs. DUNs solve the CS problem in the presence of noise added during the observation process; the same holds true when perturbing DUNs with adversarial attacks. Thus, studying the (adversarial) robustness of DUNs is reasonable. As a starting point, early work [25] highlights the robustness to Gaussian noise added to the observations, for DUN learning an

---

Table 1: Comparison of our adversarial generalization error bounds to related work's. Bold letters indicate the
method yielding the tighter - which is desirable upper-bounds, in terms attack level & and number of layers
L. For [12, 49], fr denotes the upper bound on the spectral norm of the parameter matrix at the kth layer output. For our work, the upper bound on the spectral norm of the parameter matrix shared across all layers is vB.
Ours (Lipschitz continuity surrogate
Method
Sunogate PGSM-bassd loss [12)
Covering numbers [49)
FGSd- loss covering numbers)
Adv. gen. enror bounds
Ote &))
Ot (logiile)
overcomplete sparsifier, i.e., a sparsifying transform with more rows than columns, as opposed to a DUN learning an orthogonal sparsifier. This behavior is consistent with the model-based regime, where overcomplete sparsifiers promote robust reconstruction of inverse problems [7]. More on the adversarial robustness of DUNs, [46] develops new, DUN-specifie, adversarial attack, which is compared to FGSM attacks, to elaborate empirically on the robustness of the examined DUN, Additionally, [13] explores experimentally the DUN robustness in the presence of a PGD attack [27]. Despite these advances. a theoretical investigation on the adversarial robustness and generalization of DUNs still in its infancy, motivating the need for rigorous framework that can explain and quantify the behavior of DUNS in the presence of adversarial attacks.
Adversarial learning and generalization. To develop such framework for DUNs, we turn to the well-established field of adversarial learning. which aims to improve the robustness of neural networks, by solving a maximization problem that identifies worst-case attacks. A key focus in this area is adversarial generalization, which pertains to the ability of adversarially trained DNNs to generalize well test-time adversarially perturbed data, and is quantified via the adversarial generalization error. This quantity incorporates a maximization over input perturbations, reflecting the nature of adversarial training. To study the adversarial generalization error, various theoretical tools have been proposed, including on on-average stability [48], minimax theory [45], compression arguments [2]. PAC-Bayesian theory [43], and adversarial Rademacher complexity (ARC) [1]. The latter is prominent one, due to its elegance and long-standing connection with machine learning [3]. The ARC is particularly suitable for our study, as it aligns well with the structured architecture of DUNs and offers meaningful complexity measures connected to generalization performance.
Adversarial Rademacher complexity. ARC is a fundamental tool in studying the adversarial generalization of DNNs, since the adversarial generalization error bounds can be upper-bounded in terms of adequate upper-bounds for the ARC. Nevertheless, the appearance of the max operation in ARC's defimition, complicates the derivation of upper bounds for it. Typical ways of circumventing this problem include the usage of optimal attacks or of surrogate adversarial loss function [1. 12, 20, 50], employing a dual formulation of the maximization problem [35, 36), or working directly with the covering numbers of a DNN's adversarial hypothesis class [29, 49]. Similarly to [12, 49], we work with surrogate FGSM-type loss and upper bound the ARC via the covering numbers of the DUN's adversarial hypothesis class. Nevertheless, we differentiate our approach by proving and using as a cornerstone the Lipschitz continuity of the examined DUN w.r.t. the parameter matrix. As depicted Table I, we derive a tighter upper-bound for the ARC, and thus for the adversarial generalization error, both in terms of attack level and number of layers (cf. Theorem 3 and Corollary 4), which is highly desirable. For a unified comparison, we consider the case of parameter matrices having boper-boundeddeee spectral norms'. Our findings not only advance the theoretical understanding of DUNs under adversarial attacks, but also provide concrete tools for designing robust architectures for inverse problems, bridging critical gap between empirical results and formal guarantees.
3 Background and problem formulation
Notation. For matrices A2 E N we denote by [21 E R2NNN their concatenation with respect to the first dimension, and [Ar [2]] E Nx2N their concatenation with respect to the second dimension. We write Omen Ro for the zero matrix and Lnxn for the n identity matrix. For x E R. > O, the soft thresholding operator S, : R + R is defined as Sxxx) = sign(x) ma, |xf - r). For x IR", S,(:) acts component-wis and is i-Lipschitz with respect x. The covering number
In the special case of low-rank parameter matrices, [12] exhibits layer-independent adversarial generaliza- tion error bound, thereby rendering it tighter than ours in terms of the number of layers.
3

---

N(T, d, f) of metric space (T, d) at level > 0, is defined as the smallest number of balls with respect to the metric required to cover T. When d is induced by some norm || ll. we write IT. II.
An ADMM-based DUN for CS. CS deals with recovering data x E R from missing, noisy observations y Ax + I.",, N 2 n, so that Wx E EN is sparse. ADMM [6] is one of the most celebrated iterative algorithms solving the CS optimization problem: min zllAx >lB + Xl[Wx||I
n, by assuming that there exists a fixed transform W RR
> O. The oulput of ADMM at the kth iteration resembles the output of the kth layer of a DNN: it consists of a ReLU-type nonlinearity, i.e., the noft-thresholdigg operator, applied on an affine transformation of the input data. Then, unfolding ADMM relies on casting its iterations as layers of the said DNN.
To fully formulate ADMM as a trainable DNN, W can be unknown and layer-dependent, so that
it is learned from S = (A) Lid. Dr, for unknown D. Overall, unfolding ADMM gives rise to the family of ADMM- DUNs [25, 42], parameterized by (Wa) for L total layers. A prime representative from the family of ADMM based DUNs is the state-of-the-art ADMM- DAD [25], which enjoys a sharing parameter property, i.c., W = W1 for less trainable parameters. To our knowledge, ADMM-DAD is the only ADMM-based DUN parameterized by an overcomplete sparsifier with N regime- - with experimental results indicating a correlation between overcompleteness and robustness. This overcompleteness motivates to set ADMM- -DA as paradigm for studying the adversarial robustness and generalization of DUNs. The layer outputs of ADMM-DAD are given by
... Wy, thus allowing
thereby leading to overparameterized
f' = l'b + SMp))(
(1)
(4) = (Ou + b) + 1" SAI + b), k 2....,
(2)
for p 0, 1 R - [NNNN M M] R M := Mw = PW(A A + pW Wy E RNN
= [INxY: ONxwl E 2NNx 1" = [-INx [NSCV] E R2NXN b := bw = W(A A + pW Wy-WW'' A E
R With slight abuse of notation, we write the composition of L layer outputs
fv f o. ....o f'()
(3)
and call it the intermediate decoder. Then, ADMM-DAD implements the final decoder
() Tw (+)))) = 8 x,
(4)
where T Tw(u) = Awu + (A" + pW Www - E IR", with AN = I-p(A W pp(A A +
PWT E RRx2N More details on ADMM and ADMM-DAD can be found at Appendix A.
Definition 1 (Parameter class of ADMM-DAD), We let Fg to be the class of all overcomplete sparsifiers W e R Non such that S = WT W is invertible and [|S S for 0 < B < co.
Remark 2. Due to the invertibility of S [23], holds a s 11S112-2 and [| S VB. for some 0 < a S B < co.
The standard hypothesis class [23] of all the decoders implemented by ADMM-DAD is H (h : RM + R² h(y) (y), W Fz). Given H² and the training dataset S, ADMM-DAD aims to solve the CS problem by implementing hw) = 8 z x. We work towards that direction by minimizing (over W) the training mean-squared error (MSE): (h)in(h) 3 Σi (l/ xIl2. Then, the generalization error- measuring the generalization performance of the network - is defined as GF(h)= L (h) Lmme (k)), with Lmue Bus))-(llyy give the counterparts for all errors under the adversarial learning setting.
xl12) being the true error. Below, we
Ad Adversarial learning and generalization. In the presence of adversarial attacks ō during training and inference times, the adversarial train MSE and adversarial true error are given by Lmain ا ssx (1 + 6,) - xil2 and (h) fhen, we aim to estimate the adversarial generalization error:
(aaa))-mmmajppa (l 6) xlf),, respectively.
GE(h) [[rain(h) Erme(()).
(5)
The appearance of the max operation poses extra difficulty in estimating GE. To overcome this, standard approach relies on considering adversarial attacks being the solution to the inner maximiza- tion problem. Similarly to [10], we rely on the FGSM, which is a so-called white-box attack, in the
sense that the adversary has complete knowledge of the targeted model (and so o depends on W),
and we choose p = 2. Under this framework, FGSM yields == o &PGSM
(1 so that for (1,
4

---

known 6, we can discard the max operation in GE and rewrite it as GE(h) Lwain (h) Luve (h)
: E (1, + + ) - Bu..-elllyyy" - x1131. By attacking ADMM-DAD with 6, we essentially perturb the input CS observations so the perturbed intermediate and final decoders are
fy + 6) = o f +),
(6)
(y + 6) = Tw( + 6)).
(7)
respectively. The choice of p 2 leads to natural perturbation model, operating directly on the observations y [32], and can provide workable environment for studying adversarial generalization [10, 47]. Finally, we define the adversarial hypothesis class of ADMM-DAD as
H = : - R" h(v) = ( +6) nyw E H,, W E F.. 5 = o 6f
(8)
Our goal is study the adversarial generalization of ADMM-DAD, by delivering adversarial generalization error bounds over H¹ To do so, we employ the adversarial Rademacher complexity
Rs(H = sup Σ e,hly)) im1
heftL S
with e being a vector with i.i.d. entries taking the values #1 with equal probability. While prior approaches estimate (9) using covering numbers of an adversarial hypothesis class [49] and FGSM- based surrogate loss functions [12], our method introduces distinct and principled refinement. We also bound the ARC of an adversarial hypothesis class parameterized by FGSM 12 no-m constrained attacks, deploying covering numbers. Nevertheless, our key innovation lies in establishing the Lipschitz continuity of the perturbed final decoder (7) with respect to the parameter matrix W. This structural property, which to our knowledge has not been previously exploited this context, forms the foundation of our analysis and enables tighter ARC upper bounds (cf. Table 1) via the covering numbers of HL. Due to the interpretability of unfolded architectures, we expect that similar results can be derived for other classes of adversarial attacks, under slight modifications. Below, we make a set of typical in standard and adversarial learning scenarios assumptions that will hold throughout the rest of the paper, and render our proofs and arguments relatively simple and accessible.
Assumptions. (a)) With high probability, have ||x/]l2 S Bin. for some B > 0, I....... s. For 6 generated by the FGSM under an no-mo constraint, and for any E H with high probability over yr chosen from D. it holds I[i(y))l2 Bouts for some Bout > 0, i I, .... (b) For the soft- thresholhing operator, we follow similar settings for nonsmooth functions [5] and write S (x) for [x] > a/p,
and (x) 0 for [x] A/p. (c) There exists some 0 such that [VV xIIi1l2 2 K, for any W FB. k I.....L L. Boundedness assumptions for the gradient of the loss are standard when theoretically studying the adversarial robustness of DNNs [10, 12, 30], and are numerically supported [15, 33, 34]. by imposing adequate constraints to avoid the case of VVV (Y) - -I13il2 = 0.
4 Main results
We address this paper's 's research question (+), by delivering adversarial generalization error bounds for ADMM-DAD in the form of Theorem 3 (with proof found in Appendix C.5) and Corollary 4.
Theorem 3 (Adversarial generalization error bounds for ADMM-DAD). For L 2 being the total number of layers, let Hl be the adversarial hypothesis class defined in (8) and ö adversarial attack generated by the FGSM, with 1|ö112 S E, for attack level > O. Assume there exist pair- samples (o); itd. Dd, with y; Ax; e, |lellz S 17. for some n > O, and Assumptions (a) (c) hold. Then with probability at least 1 Ç, for all E HL, the adversarial generalization error of ADMM-DAD defined in (5) is bounded as
1o 2 VBLipp
Nn
2 log(4/
GB( 50
log
exp.
1
(10)
VsBaat
s
with Lipl. defined in Theorem 6 being the Lipschitz constant of the adversarially perturbed final decoder (7) implemented by ADMM-DAD and exp denoting the natural exponent.
5

---

As we will see in Theorem 6, L enters at most exponentially and & at most linearly in the definition of -. Hence, due to the appearance of the logarithm in Theorem 3, we easily obtain: Corollary 4 (Growth rate). !ewe consider the dependence of the adversarial generalization error bound (10) only on L,, N, S, 6, and treat all other terms as constants, it roughly holds that
GE( so NLlog(e)
(1I)
S
Significance of Theorem 3 & Corollary 4. Our results: a) are informative, by including all elemental factors, i.e,, N, I,, E, that determine the DUN's architecture and performance, b) highlight how overcompleteness N ripples out to DUN adversarial generalization, i.e., although our bounds grow as N increases, the growth is at the reasonable rate of NN, c) are tighter- - which is desirable- - and thus more realistic, than those of state-of-the -arrt-aarrt related work [12, 49] w.r.l. and (cf. Table 1).
The path to proving Theorem 3. We give a sequence of results, each of which serves as a crucial component in deducing Theorem 3. We account for the number of training samples in S and thus pass to matrix notation, i.e., capitalize all vectors. Based on Assumption (a), a simple application of the Fauchy-Schward inequality yields ||Y||s I VsBm. [[k(()) & VSB and ||Al|s Vse.
Proposition (Bounded outputs), Let k E N, and fw()) be the perturbed intermediate decoder implemented by ADMM-DA DAD and defined in (6). Then. for any learnable overcomplete sparsifier W E Fg. we have
v(1 2Byp)', (Y F S (IlY AlA PY VB
(12)
where y 2-pli'AAb- a as in Remark v= 1 + V2, and A == A FGSM
with II
JAFGSMII= S VS6.
w
Why is Proposition 5 important? The upper bound on ADMM-DAD's outputs constitutes the first instance depicting how L and ripple out to the DUN adversarial robustness; then, we deploy this bound to prove Lipschitz continuity of ( with respect to W.
Theorem 6 (Lipschitz continuity of the perturbed decoder w.rt. parameter). Let Oly()) be the perturbed final decoder implemented by ADMM-DA -DD and defined in (7), L 2, and learnable overcomplete sparsifier W g. Then, for any W,, W2 Tp. we have
U1 (2) Thyn (YI1F= := 11htr, (Y + A1) hhy, (Y + A S Lip Wall2-2, (13)
where A, := FGGS with |IA S Vse, i= 1,2, and Lipschitz constants Lip,, depending exponentially on the number of layers L and linearly on the atlack level g:
VSE
Lipl,* =2yp (1lll==
Vis + 2B(Bin + Bua)) x²
vy v1111113-2)
(14)
41 Σ km/
Σ (rn)-**a He + - + Vos 1
(m))))
with r= 1 + 2Byp. y as in Proposition 5, B as in Definition 1, v = (1 + V2), and H₂ a constant calculated explicitly and defined in (49) of Appendix C.2.
Why Theorem 6 important? We provide an explicit formulation of Lip with direct dependence on L. and 6, allowing us to tightly upper-bound the ARC (9) with respect to L. and & (cf. Theorem 8).
While Theorem 6 enables tightness in the ARC bounds, the passage allowing the ARC's estimation is accomplished by means of the celebrated Dudley's inequality [9, Theorem 5.23), [11, Theorem 8.23]. This a powerful probabilistic tool, which upper-bounds a stochastic process (like the ARC) on a space, to the integral of the covering numbers of this space. We work towards that direction and define M := ((h6y))) ... Ih(y,)) Raxx h E Hf4 10tt/ + +.)... +. +5,) E RRa.. hw he H., W Fg). corresponding to H.. Since M and Hf are parameterized by W, we rewrite (9) as 1 MEM 5
Rg(HH² = E sup ΣΣ (hh())) = E sup
ΣΣ ca Ma.
(15)
hefe

---

so that we can estimate the covering numbers of M instead of H²:
Proposition 7 (Upper-bound covering numbers). For the covering numbers of M it holds:
1+ 2 VBLiph Le Nn
N(M (Il,,)
(16)
1
with - defined as in Theorem
Thanks to (16), overcompleteness N is also included in the framework we are setting up. Then, a simple application of the Dudley's 's integral inequality upper-bounds the ARC in terms of (16):
Theorem 8 (ARC estimate). Let HD be the adversarial hypothesis class of ADMM-DAD and defined in (8). Then, for the adversarial Rademacher complexity Rs(H') defined in (9) it holds:
4 V2
Vilust
2 VBLip
Rs(H S
Nn
1
dt.
(17)
Why is Theorem 8 important? The ARC is an essential tool for thoroughly explaining adversarial generalization. The explicit dependence of the ARC estimate on elemental quantities like N, L, stresses how these ripple out to the adversarial generalization of ADMM-DAD, Especially, by definition of Lip,,, due to the appearance of the logarithm, and since (17) can be proven to be integral-free (cf. Appendix C.5), the ARC estimate roughly scales like (NLlog(e) (cf. Corollary 4).
To connect the ARC to the adversarial generalization error bound and deduce Theorem 3, we deploy [38, Theorem 26.5). The latter upper-hounds the generalization error of network, to the Rademacher complexity of the network's 's hypothesis class, when composed with the loss || 11- To remove || Il2 and work solely with the Rademacher complexity, we employ [28, Corollary 4], which further requires to calculate the Lipschitz constant of || 11 It is easy to check that ((-) = || 11 is Lipschitz continuous, 2m 2B Therefore, by (17) and [28, Corollary 4] we deduce:
with Lipschitz constant LiPHn
Roll 112 044 z V2(2 + B s so { ViRus
VBLip,
Nn
1
dt
(18)
We combine (18) and [38, Theorem 26.5), to give adversarial generalization error bounds for ADMM- DAD, stated formally in Theorem 3 and Corollary 4, thus answering this paper's s research question (*). The proofs of Proposition 5, Theorem 6, Proposition and Theorem 8 can be found at Appendices C.I. C.2, C.3 and C.4, respectively. Moreover, [11, Theorem 8.23], [38. Theorem 26.5] and [28, Corollary 4] are formally stated as Theorem B.6, Theorem B.7 and Lemma B.8, respectively. In the next Section, assess the validity of our theory with series of experiments real-world data.
5 Experiments
We train and test ADMM-DAD on two real-world image datasets: CIFARIO 10 (50000 training and 10000 test 32 x 32 coloured image examples) and SVHN (73257 training and 26032 test 32 x 32 colored image examples). For both datasets, we transform the images into grayscale ones and vectorize them. We fix m/n 25%, and alternate the overcompleteness. N, and the number of layers L. We consider a standard CS setup, with an appropriately normalized random Gaussian A E Re and noisy observations of the form y Ax e, with being zero-mean Gaussian noise, with standard deviation std = 10-2 To generate the adversarial attack ö, we employ FGSM from [15], under an 12-norm constraint, and attack ADMM-DAD with during training and test times, with varying attack levels 8. To highlight the adversarial robustness of ADMM-DAD against more powerful attacks, we also employ [15] generate an ba-aaedd PGD attack with 10 iterations. We initialize the learnable overcomplete sparsifier W E R Nxn using a Xavier normal distribution [14]. We implement all models in PoTor [19] and train them using the Adam algorithm [21], with batch size 128. As evaluation metrics, we use the clean and adversarial test MSEs
Liea
mM x1112
(19)
7

---

Lo33 Lo33
Ohi A0k, 1+30
1210
Natie level
Matic bevel
Figure 1: Performance of ADMM-DAD (plotted logarithmic scale) on CIFARIO (left) and SVHN (right), for varying number layers L and attack levels of the FGSM, and overcompleteness N = 10n. Top: clean test MSE (19) and adversarial test MSE (20). Bottom: adversarial EGE (21). For both datasets, (21) increases
Theorem 3 suggests, and in fact scales at the rate dictated by Corollary 4, thus confirming our derived generalization theory. A similar increment is observed for both (19) and (20), but at reasonable rate, thereby highlighting the adversarial robustness of ADMM-DAD.
MCCE
Al.
Loda
Can
Figure 2: Performance of ADMM-DAD (plotted on logarithmic scale) on CIFARIO (left) and SVHN (right), for varying number of layers L and attack levels & of the PGD (10 iterations), and overcompleteness N 10n. Top: clean MSE (19) and adversarial test MSE (20). Bottom: adversarial EGE (21). Although our theoretical analysis focuses on FGSM, we observe rate dictated by Corollary 4, for both datasets, thus corroborating our derived generalization theory. Similarly, (19) and (20) also increase, but at reasonable rate, thus highlighting the adversarial robustness of ADMM-DAD under more powerful than FGSM attacks.
even for stronger adversarial attack PGD, (21) scales at the
Σ |/1y + 8,) 10--(/ x/112- F=1
Dusa()) (h) =
(20)
respectively, with D = (0 being a set of test data, not used during training, and the adversarial empirical generalization error (adversarial EGH)
EGE(h) (h) Erain (h)),
(21)
with (h) being the adversarial train MSE defined in Sec. 3. We also compare ADMM-DAD to state-of-the-art baseline DUN called ISTA-net [4], parameterized by an orthogonal sparsifier. We aim to use the structural difference between the two DUNs to showcase how the adversarial robustness and generalization of DUNs are affected, when employing an overcomplete sparsifier instead of an orthogonal one. For more experimental settings and details, we refer the reader to Appendix D.
Test and generalization errors for increasing 8. We measure the performance of ADMM-DAD with L = 5 and L = 10 on CIFARIO. 10, and L = 10 and L = 15 on SVHN, both with fixed N = 10n, terms of the clean test MSE (19), the adversarial test MSE (20), and the adversarial EGE (21). as & varies, for both the FGSM and the PGD. We report the results in Figure 1 for FGSM, and in Figure 2 for PGD, both corroborating Theorem Specifically, for both datasets and attacking methods, we observe that the adversarial EGEs, as these are depicted in the bottom of Figure I and Figure 2, increase, as both L and increase. Despite the appearance of other terms in (10), the adversarial EGES seem to scale at the rate of vL. log(e), like Corollary suggests. As illustrated at the top of Figure and Figure 2, both (19) and (20) also increase increases. This behavior is anticipated from the adversarial robustness perspective, since the higher the attack level, the more neural network "struggles" to infer correctly. Nevertheless, the increment of (19) and (20) on both
8

---

Cran, nobe
Clean nose lexal 0.01
Adk, nake
Adt. nole lovel 6.01
noise
nekee
AdV.
Figure 3: Robustness plots of 5-layer ADMM-DAN on CIFARIO, (left) and 10-layer ADMM-DAD on SVHN (right), for alternating overcompleteness N and different attack levels of FGSM. For both datasets, as N increases, the clean test MSEs (19) and the adversarial test MSFs (20) drop. Importantly, for the standard case of 8 0.01, the robustness gap of ADMM-DAD on both datasets is particularly small. All all, results highlight the beneficial role that N plays on robustifying ADMM-DAD against varying adversarial attack levels.
datasets seem to be at the reasonable rate of roughly square-root. Overall, the empirical behavior of ADMM-DAD matches its theoretical one. Interestingly, despite the fact that our theory hinges upon the FGSM, the similar adversarial robustness and generalization of the PGD "GD-attacked ADMM-DAD fuels us theoretically investigate this phenomenon more in the future, as we pinpoint in Sec. 6.
The role of N in adversarial robustness. We measure the adversarial robustness of 5- and 10-layer ADMM-DAD on CIFAR10 and SVHN respectively, for increasing N, and three different values of & for the FGSM, by means of (19) and (20). Given that DUNs operate in a regression setting (see (7)) so that standard classification-tyye type metrics like accuracy cannot be utilized the deployment of the clean and adversarial test MSEs (and their gap) as adversarial robustness metrics is a common practice [35, 36]. We report the results for both datasets in Figure which demonstrates an intriguing phenomenon: as N increases, for fixed E, both clean and adversarial test MSEs drop, indicating that the overeompleteness of the learnable sparsifying transform promotes ADMM-DAD's adversarial robustness. Of course, the MSEs increase as & also increases, but this is anticipated, since stronger attack in the CS observations tantamounts explaining the adversarial robustness of ADMM-DAD through the overcompleteness of the learnable sparsifier lays a fruitful ground, to identify sysperpprrameter" and properly fine-tune tune them, so as to boost adversarial robustness in the unfolding regime; we briefly mention this research line in Sec. 6.
DUN being less able to recover the data. Nevertheless,
A note on the robustness gap. Based on Figures and we examine the robustness gap of ADMM- DAD, i.e. the difference between (19) and (20). We notice that the robustness gap slightly increases for the CIFARIO with increasing of the FGSM albeit the gap's 's scaling from one attack level the next is reasonable- but the picture, e.g.. Figure 1, is much better for the SVHN, where both test errors grow proportionally. This indicates that ADMM-DAD enjoys adversarial robustness to increasing attack levels. Especially in the case of & = 0.01 which is still a non-negligible attack level - we deduce from Figure 3 that, for both datasets, the robustness gap is especially small, e.g.. ~ 10-4. Due to these interesting findings, explaining the robustness gap of DUNs, both from theoretical and practical side, could be an inspiring future work (see also Sec. 6).
Comparison to baseline. We compare a 10-layer ADMM-DAD to a 10-layer ISTA-net on both datasets, with mild overcompleteness for ADMM-DAD of N 10n. We present the results in Table 3, with the top comparisons refering to CIFARIO 10 and the bottom to SVHN. We observe that ADMM-DAD outperforms the bascline, consistently for all datasets, since exhibits smaller clean and adversarial test MSEs, as well as adversarial EGE: on the other hand, ISTA-net achieves errors being orders of magnitude larger than those of ADMM-DAD. Furthermore. we observe that the robustness gap (see paragraph above) exhibited by ADMM-DAD is smaller than the corresponding one of net.-net. This behavior highlights the beneficial role of overcompleteness in the unfolding regime, as opposed to orthogonality, when studying adversarial robustness.
Conclusion and future work
In this paper, we addressed the adversarial generalization of DUNs. These are interpretable networks emerging from iterative optimization algorithms, incorporate knowledge of the data model, and

---

Table 2: Comparison of ADMM-DAD with overcompleteness N = 10n the baseline, both with 10 layers. against different FGSM attack levels &, on CIFARIO (top) and SVHN (bottom). Bold letters indicate the DUN that scores the best performance in terms of all metrics (19), (20), (21). Overall, ADMM-DAD outperforms the baseline, highlighting the advantage of overcompleteness over orthogonality in the unfolding regime.
CIFARIO
test MSE
Adv.
EGE
DUN
0,01
0.3
0.01
0.1
0.01
0.1
ADMM-DAD
0.019
0.023
0.026
0.000
0.025
0.034
0.18-107
0.11-10
1.16-
ISTA-nel
0,021
0.07
0.229
0.023
0.947
0.229
0.70 10
0.55 10"
0.15 1.15-10" 100"
SVHN
Clea
Ads, test
AdV
DUN
0.01
0.1
0.01
0.1
0.01
0.1
ADMM-DAD
0.033
0,014
0.25
0,113
0.05
0.039
0.23.
10
3.77
17-4
5.77
1STA-net
0,028
0.036
0.676
0.030
0.054
0.703
0.89
10
0.78
10"
15 10
are designed to solve inverse problems like CS, which finds applications in safety-related domains. Thus, it is crucial to understand the adversarial robustness and generalization of DUNs. To that end, we selected an overparameterized representative from celebrated family of DUNs, serving as a paradigm to study the adversarial generalization in the unfolding regime; then, we perturbed this network with FGSM-based attacks under an (2-norm constraint. We proved that the attacked network is Lipschitz continuous with respect to the parameters a crucial intermediate step for estimating the DUN's adversarial Rademacher complexity. Then, we utilized this estimate to deliver adversarial generalization error bounds for the representative DUN. To our knowledge, these are the first theoretical results explaining the adversarial generalization of DUNs. Finally, we supported our theory with relevant experiments, and highlighted how overparameterizztion in the unfolding regime can promote adversarial robustness.
Our work opens promising future directions. Although provided solid mathematical explanation for the adversarial generalization in the unfolding regime, a question arises regarding the tightness of these upper bounds, for instance, with respect to the overparameterization. What is more, the generalization of our theoretical framework to broader classes of adversarial attacks like PGD- based with different norm constraints, could improve the understanding and impact of DUNs in real-world scenarios, e.g., when these are applied in CS-MRI. Finally, we empirically observed that overparameterization promotes adversarial robustness in the unfolding regime. Consequently, would be fruitful to theoretically study the robustness gap, in terms of the overparameterizaaion, as means of explaining the adversarial robustness of DUNs.
Acknowledgments and Disclosure of Funding
The research was mainly conducted during the author's affiliation with the Isaac Newton Institute for Mathematical Sciences of the University of Cambridge. The author would like thank the Isaac Newton Institute for Mathematical Sciences for supporting them during their INI Postdoctoral Research Fellowship in the Mathematical Sciences, especially during the program "Representing. calibrating & leveraging prediction uncertainty from statistics to machine learning". This work was funded by the EPSRC (Grant Number FP/V521929/1). Moreover, the author would like to thank P. Mertikopoulos for insightful comments and fruitful discussions.
References
[1] Awasthi, P., Frank, N., and Mohri, M. Adversarial learning guarantees for linear hypotheses and neural networks. In Int. Conf. Mach. Learn., pp. 431-441, PMLR, 2020.
[2] Balda, E. R., Behboodi, A., Koep, N., and Mathar, R. Adversarial risk bounds for neural networks through sparsity based compression. arXiv preprint arXiv:1906.00698, 2019.
[3] Bartlett, P. L. and Mendelson, S. Rademacher and Gaussian complexities: Risk bounds and structural results. J. Mach. Learn. Res., 3(Nov):463-482, 2002.
[4] Behboodi, A., Rauhut, H., and Schnoor, E. Compressive sensing and neural networks from statistical learning perspective. In Compressed Sensing in Information Processing, pp. 247-277. Springer, 2022.
10

---

[5] Bertoin, D.. Bolte, J., Gerchinovitz, S., and Pauwels, E. Numerical influence of ReLU'(0) on backpropagation. Adv. Neural Inf. Process. Syst., 14:468-479 2021.
[6] Boyd, S., Parikh, N., and Chu, E. Distributed optimization and statistical learning via the allernating direction method of multipliers. Now Publishers Inc, 2011.
[7] Casazza, G. and Kutyniok, G. Finite frames: Theory and applications. Springer Sci. & Bus. Media, 2012.
[8] Daubechies, I., Defrise, M., and De Mol, C. An iterative thresholding algorithm for linear inverse problems with sparsity constraint. Commun. Pure and Appl. Math., 57(11):443-1455, 2004.
[9] Dudley, R. M. The sizes compact subsets of hilbert space and continuity of Gaussian processes. J. Funct. Anal., 1(3):290-330, 1967.
[10] Farnia, F., Zhang, M., and Tse, Generalizable adversarial training via spectral normalization. arXiv preprint 181. 01.111111.7777222 2018.
[11] Foucart, S. and Rauhut, H. An invitation to compressive sensing. A mathematical introduction to compressive sensing. pp. 1-39. Springer, 2013.
[12] Gao, Q. and Wang, X. Theoretical investigation of generalization bounds for adversarial learning of deep neural networks. J. Statistical Theory and Pract., 15(2):51, 2021.
[13] Genzel, M., Macdonald, J., and März, M. Solving inverse problems with deep neural networks- robustness included? IEEE Trans. Patt. Anal. and Mach. Intell., 45(1): 111111991111 1134, 2022.
[14] Glorot, X. and Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proc. 13th Int. Conf. Artif. Intell. and Statist., pp. 256. JMLR Workshop and Conf. Proc., 2010.
[15] Goodfellow, I. et al. Cleverhans vO. 1: an adversarial machine learning library. arXiv preprint arXiv:1610.00768. 1:7, 2016.
[16] Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. arXiv preprint arXiv: 1412.6571... 2014.
[17] Gregor, K. and LeCun, Y. Learning fast approximations of sparse coding. Proc. 27th Int. Conf. Mach. Learn., pp. 399-406, 2010.
[18] Huang, S., Lu, Z., Deb. K., and Boddeti, N. Revisiting residual networks for adversarial robustness. In Proc. IEEE|CVF Conf. Comput. Vision and Patt. Recognit., pp. 8202-8211, 2023,
[19] Ketkar, N. Introduction to pytorch. In Deep learning with python, pp. 195-208. Springer, 2017.
[20] Khim, J. and Loh, P. L. Adversarial risk bounds via function transformation. arXiv preprint arXiv:1810.09579, 2018
[21] Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. arXiv preprint 112.9.1412.6980, 2014.
[22] Kong, X. and Ge, Z. Adversarial attacks on regression systems via gradient optimization. JEEE Trans. Syst., Man, and Cybern.: Syst., 2023.
[23] Kouni, V. and Panagakis, Generalization analysis of an unfolding network for analysis-based compressed sensing. arXiv preprint 2rXiv:2303.05582, 2023.
[24] Kouni, V. and Panagakis, Y. DECONET: An unfolding network for analysis-based compressed sensing with generalization emoo bounds. IEEE Trans. Signal Process., 71:1938-1951, 2023.
[25] Kouni, V., Paraskevopoulos, G., Rauhut, H., and Alexandropoulos, G. C. ADMM-DAD Net: A deep unfolding network for analysis compressed sensing. IEFE Inf. Conf. Acoust, Speech and Signal Process., pp. 1506-1510. IEEE, 2022.

---

[26] Lin, J.. Song, C., He, K., Wang. L., and Hoperoft, J. E. Nesterov accelerated gradient and scale invariance for adversarial attacks. arXiv preprint arXiv:1908.06230 2019.
[27] Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. In Int. Conf. Learn. Representations, 2018.
[28] Maurer, A. A vector-contraction inequality for Rademacher complexities. In Int. Conf. Algo- rithmic Learn. Theory, pp. 17. Springer, 2016.
[29] Mustafa, W., Lei, Y., and Kloft, M. On the generalization analysis of adversarial learning. In Int. Conf. Mach. Learn., pp- 16174-16196. PMLR, 2022.
[30] Najafi, A., Maeda, S. I., Koyama, M., and Miyato, T. Robustness adversarial perturbations in learning from incomplete data. Adv. Neural Inf. Process. Syst., 32, 2019.
[31] Ortiz-Jiménez, G., Modas, A., Moosavi-Dezfooll S. M., and Frossard, P. Optimism in the face of adversity: Understanding and improving deep learning through adversarial robustness. Proc. IEEE, 109(5):335-65, 2021.
[32] Raj, A., Bresler, Y., and Li, B. Improving robustness of deapnle -based image reconstruc- tion. In Int. Conf. Mach. Learn., pp. 7932-7942. PMLR, 2020.
[33] Rauber, J., Brendel, W., and Bethge, M. Foolbox: A python toolbox to benchmark the robustness of machine learning models. In Reliable Machine Learning in the Wild Workshop, 34th Int. Conf. Mach. Learn., 2017.
[34] Rauber, J., Zimmermann, R., Bethge, M., and Brendel, W. Foolbox native: Fast adversarial attacks to benehmark the robustness of machine learning models in pytorch, tensorflow, and jax. J. Open Source Softw., 5(53):2607, 2020.
[35] Ribeiro, A., Zachariah, D., Bach, F., and Schön, T. Regularization properties of adversarially- trained linear regression. Adv. Neural Inf. Process. Syst., 36:23658-22 -2670, 2023.
[36] Ribeiro, A. H. and Schön, T. B. Overparameterized linear regression under adversarial attacks. IEEE Trans. Signal Process., 71:601-614, 2023.
[37] Scarlett, J., Heckel, R., Rodrigues, M. R., Hand, P., and Eldar, Y. C. Theoretical perspectives on deep learning methods in inverse problems. IEEE J. Sel. Areas Inf. Theory, 5(3):4334453 2022.
[38] Shalev-Shwartz, S. and Ben-David, S. Understanding machine learning: From theory to algorithuss Cambridge Univ. Press, 2014.
[39] Shlezinger, N., Whang, J., Eldar, Y. C., and Dimakis, A. G. Model-based deep learning. Proc. IEEE, 111(5):465 499,,,499999200 2023.
[40] Song. J., Chen, B., and Zhang, J. Memory augmented deep unfolding network for compressive sensing. In Proc. 29th ACM Int. Conf. Multimedia, pp. 4249 4258, 2021.
[41] Song, J., Chen, B., and Zhang, Dynamic path-controllable deep unfolding network for compressive sensing. IEEE Trans. Image Process., 32:2202-2214, 2023.
[42] Sun, J., Li, H., and Xu, Z. Deep ADMM-Net for compressive sensing MRI. Adv. Neural Inf. Process. Syst., 29, 2016.
[43] Sun, T. and Lin, J. PAC-bayesian adversarially robust generalization bounds for graph neural network. arXiv preprini arXiv:2402.04003 240.04438. 2024.
[44] Tanner, J. and Vary, S. Compressed sensing of low-rank plus sparse matrices. Appl. and Comput. Harmon. Anal., 64:254-293, 2023.
[45] Tu, Z., Zhang, J., and Tao, D. Theoretical analysis of adversarial learning: A minimax approach. Adv. Neural Inf. Process. Syst., 32, 2019.
[46] Wang, Y., Wu, K., and Zhang, C. Adversarial attacks on deep unfolded networks for sparse coding. In IEEE Int. Conf. Acoust.. Speech and Signal Process.. pp. 5974-5978. IEEE, 2020.
12

---

[47] Wang, Y., Zhang. K., and Arora, R. Benign overfitting in adversarial training of neural networks. In Int. Conf. Mach. Learn., 2024.
[48] Wang, Y., Liu, S., and Gao, X. S. Data-dependent stability analysis of adversarial training. Neural Networks, 183:106983, 2025.
[49] Xiao, J. et al. Bridging the gap: Rademacher complexity in robust and standard generalization. In 37th Annu. Conf. Learn. Theory, pp. 5074-5075. PMLR, 2024.
[50] Yin, D., Kannan, R., and Bartlett, P. Rademacher complexity for adversarially robust general- ization, In Int. Conf. Mach. Learn.. pp. 7085-7094. PMLR, 2019.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
All the claims found the abstract and introduction are properly supported in the theoretical results of Sec. 4 and the experimental results of Sec. 5. Additionally, proofs for our derived theory can be found in Appendix C and experimental extensions in Appendix D.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Throughout the paper, we state the limitations of our work, among which the assumptions imposed on the examined unfolding model.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
14

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: For the course of our theoretical results presented in Sec. 4, we state mini- mal and justified assumptions, to clarify every possible dependency of the problem. All associated proofs can be found in complete form at Appendix C.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Thcorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
In Sec. 5, we provide details on the experimental setup of the paper leading to the corresponding experimental results. We also employ example datasets used in papers that are close to our work, while in Sec. D, we outline more experimental details, including choice of hyperparameters for reproducibility purposes.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
15

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Upon acceptance, will provide link public github repository with pytorch code, and sufficient documentation for reproducibility all the experimental results that accompany the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cc cc/
public/guic /eddddeSuuuuumm ssionPolicy) for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
hht////oeebbmsssseeee ssioomooissi
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permirted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We briefly describe the main experimental setup in Section 5, and then further elaborate on all details in Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes].
Justification:
We run all of our experiments multiple times and report the error bars.
Guidelines:
16

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, trainjtest split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: For the course of our experiments, we present detailed descriptions on the computer resources in Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: httperrrwpp....puiiittttiiaaaaaa
Answer: [Yes]
Justification: We preserved anonymity in our submission. Our submission abides by the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
17

---

Justification:
We discuss potential societal impacts of our work in Appendix E.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negatíve societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
Our paper does not pose any such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All original owners of assets are properly credited, e.g., we cite the original paper of the unfolding network we investigate, while for our experiments, we cite the public papers and code that are used as baselines for comparison.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
18

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi wwwoe. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We do not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: No crowdsourcing or human subjects involved in the experiments con- ducted for this paper.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No crowdsourcing or human subjects were involved the experiments con- ducted for this paper.
19

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The methods developed during the present research do not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy htttps//neurrps.. / for what should or should not be described.
20

---

Appendices
A Unfolding ADMM to ADMM-DAD for solving Compressed Sensing
In this Section, we introduce the inverse problem of Compressed Sensing (CS), and present two methods to solve it: the model-based iterative algorithm ADMM and its unfolded counterpart, namely, ADMM-DAD, For the sake of completeness, we restate parts of the background presented in Sec. 3.
A.1 Preliminaries Compressed Sensing
CS deals with recovering data x E R" from missing, noisy observations y Ax+eE R"" m < n, |lellz 1/, O, under the assumption that there exists some fixed sparsifying transform W R 2 n. In a typical CS scenario, for additive Gaussian noise e of sufficiently high standard deviation, e.g., of the order of 10-2, have approximate recovery of x, so that the error between the original and reconstructed data is upper-bounded by a quantity involving 7]. Under the sparsity assumption, the optimization problem associated to CS is the so-called LASSO
min X 2 ||Ax yl1 A||Wa|l1-
In the usual synthesis sparsity model, W is usually an orthogonal sparsifying transforms, í.e., W R"xe with WWT Ixx (e.g. W may be the discrete cosine transform). However, when W is overcomplete, namely, N > n, one operates in the much more flexible analysis sparsity model, which shown offer more advantages than its synthesis counterpart. For an analytic comparison between the two sparsity models, we refer the interested reader to [23, 24]. Under the analysis sparsity model, the optimization problem of CS is called generalized LASSO.
A.2 Model-based approach: the ADMM
Various algorithms can solve the (generalized) LASSO problem, one of which is the celebrated alternating direction method of multipliers (ADMM). ADMM introduces dual variables E R so that the LASSO problem is equivalent to
min |Ax
>1k + A|k
subject to W 0.
For p > (penalty parameter), N iterations, initial points 22,22, v²), and Sup(( being the soft-thresholding operator introduced in Sec. 3, ADMM produces the following iterative scheme:
*+ (AAA p w) y pWT (
c+1 (Watt + +
A-1 Wxttl , =+11
which is known [6] to converge to solution p* of the (generalized) LASSO's equivalent formulation, ie., ||Ax* - ylI I - p* and Wx - - as lc - co. Although ADMM can equally address the LASSO problem under both the synthesis and the analysis sparsity models, continue our setup with an overcomplete sparsifier W e R Nxn due to the advantages of analysis over synthesis sparsity.
A.3 Data-driven approach: the ADMM-DAD
To reformulate the iterative scheme of ADMM as deep unfolding network (DUN), we substitute the x-update into the z- and v-updates, then the z-update into the r-update, and introduce the intermediate variable = [ e R2XX1 that
Out b - SApp(Ou + b)
O + ONxyb + (Su + b),
with
θ UNNN MIM] E R
21

---

M: == Mw PW(A'A A + +WW WY W E BRA b == bw()) W(A A pW' W A
Finally, we introduce I' = [ ONXN] E 2NN and /" = [- NxN: [Nxx] E R2Nx so that the iterative scheme of ADMM is compactly written in the following angle-variable form:
[+1 = l'O + b) + 1" ( + b). 2
To enable learning scenario, we assume that the sparsifying transform W is unknown and learned from set of i.i.d. training samples, i.e., S {(x;,))) ye)) drawn from an unknown distribution" D'. Then, the iterative scheme of ADMM can be interpreted as a neural network with L E N layers, coined ADMM Deep Analysis Decoding (ADMM-DAD) [25].
B Auxiliary Theorems
In this Section, we state Theorems, which will be used later on the proofs of our main results. Specifically, we start with two well-known Theorems from numerical linear algebra.
Theorem B.1. Let A RR be invertible and B Raxa_ For sub-multiplicative matrix norm || on Roxa if holds ||A- ||]]| < 1, then + B E Raxa is invertible. Moreover, we have
|| 1- -|---1|-1100 -|8|
IIA
B) ()+BB)--
(22)
Proof. A + not invertible, then there exists some x # such that Ax Bx = 0. By assumption,
x=0
is invertible, thus -x = A-'Bx. Bx. Hence, ||x|| |x|| S ||A |[B||
||I||
= 1 > ||^ |[]],
which contradicts our assumption,
A + B is invertible. We also have: A
A + B = A(I + B) Since A + B and / + A- B are invertible, we get (A B) -
Due to the invertibility of + B we get
A BI
- (1 (1+ =1 -1-- -1- B(I + A B)-
[[Z 1| B(I A B)
<|/||| + ||-+|||ABB// - ||
<l + ||~-||| || || |1(1 + A -)
1
|177 - B) || - 1 ||A ||BII
We apply the latter estimate to [I(A B) = V|^ ||| |1(1 + A B) || and the proof follows.
Theorem B.2. For sub-multiplicatiie matrix norm || || R ifA, B E Rxx are invertible, then
|| - --
|| -|1 || ||A BlI
(23)
Proof. Since B-1 --1 = B BA- (AA
B B)A-1 we deduce, by sub-multiplicativity of the norm || Il. that ||B A A || S B || I[A BBII I|A
) BA-
BA
As part of strategy for proving Appendix C.2 the Lipschitz continuity of the perturbed final decoder (7), we provide below two intermediate results, which serve in a similar way to Proposition 5: a) in Proposition B.3 show that the gradient with respect the input- of the layer outputs, is upper- bounded by quantity involving the number of layers k < L, b) Theorem B. 3.4 showcases the Lipschitz continuity of the final decoder (4) in the clean, not contaminated by adversarial attacks regime- with respect to the parameter matrix W. In fact, the Lipschitz constants of the decoder depend exponentially on the total number of layers L..
FFrrmall speaking, this distribution over x; and for fixed A,e, we obtain y, Ax;
22

---

Proposition B.3 (Bounded gradient outputs), E N, and (w( be the intermediate decoder of ADMM-DAD defined in (3). Then, for any learnable overcomplete sparsifier W E ETB. we have
K IIV y & 11All--2 VB Σ +( 28yp)' 2
(24)
where v ++ V2. y= a-plA Ale B as in Definition and a as in Remark 2.
Proof. We prove (24) via induction. Firstly, we notice that I/! + 1"112--2 S 11/'12--2 + 11/" 1 v2= Then, for k 1:
pwwpww/Ww.... w) 112-22 IV ( Bl vlAll222 A +
which holds by definition of (I). The invertibility of S W W and Theorem B.I imply that
p|l 112-2
II(A 4 + -> = + + v 1- p||S 1122~21AA All2-2
P ==
a- -=Al'AAAll Alle-2
where in the last inequality we used the fact that B-1 I|S $115~1112-22 112-22 S a- due to the overcompleteness of W [23]. Hence,
VVII 11l2--2r VB.
Suppose now that (24) holds for some k E N. Then, for k 1:
IV 1 (Il
sr + 2 + ,1V
Sy (1 28yp)
-lAl12-22 vB Σ
v(1 + 28ypr + 11lll2-22 vD)
250
=|AAll--ry
BB
Σ
(111 28yp)/
F=
which complements the proof.
Theorem B.4 (Lipschitz continuity of final decoder w.r.t. parameter [23, Corollary 3.11)). Let h Hl be the standard hypothesis class of ADMM-DAD:
H-- (h Rd T R" h-y)= H4,,), W E Fgl.
L. 2 be the total number of layers, and learnable overcomplete sparsifier W e Fs. with Fg in Definition 1. Then, for any W.. W2 E Fs. we have:
[[w,, w. A (Y)) Tw, (A (Y ExllW2 Will2-2.
(25)
where
1-1 r/1 + 28y0)
Σg = = VB KL + 1yAl--2221 28yp)
Σ
(26)
23

---

v 1+ V2, y as in Proposition 5, B as in Definition 1, and
L Σ
K1. yG
+
G yG
Σ
(27)
with G y(1 + 28yp).
Two key concepts which the estimation of the (adversarial) Rademacher complexity relies the covering numbers and Dudley's 's integral inequality. Specifically, in order to estimate the supremum of stochastic process which is essentially how the ARC is defined over space, can employ Dudley's inequality [9, Theorem 5.23], [11, Theorem 8.23], which further requires the estimation of the covering numbers of the said space.
To that end, we state below an intermediate result, which provides an upper-bound on the covering numbers of the a-radius ball in the space of all N x real-valued matrices. From this space, pass to the estimation of the covering numbers of the parameter space Fe (cf. Appendix C.3). Then, by using the Lipschitz continuity of the perturbed decoder (cf. Theorem 6), we will be able estimate the covering numbers of the adversarial hypothesis class H² (8) by means of the Lipschitz constants and TB.
Lemma B.5 (Covering numbers [23, Lemma 3.12]). For O < co, the covering numbers of the ball N (a) = (X Non [|Xil2-2 a) satisfy the following for any 1 O:
Hbu
2a N
NC [H (a)
(0).11-12-2..)) 12)-2, & (1
Now, we formally state Dudley's s integral incquality, which we employ later on in Appendix C.4 to upper-bound the ARC.
Theorem B.6. Let (Xi)er be a random Gaussian process a metric space (T,d) d) with sub-gaussian increments. Then,
B sup X, 44 V2 MTT122
(28)
1ET
where A(T) suP V.
The next Theorem allows connect the ARC and the adversarial generalization error (5). Although this Theorem has been proven in the standard hon-adversarial learning regime, we can still deploy it for our framework, since by definition of 4 and due to the lack of the max operation in ARC, Theorem B.7 constitutes precise tool for delivering adversarial generalization error bounds for ADMM-DAD.
Theorem B.7 (Generalization error bounds [38, Theorem 26.51). Let
Let'H H be a family of functions, S the training set drawn from D and & a real-valued bounded loss function satisfying [E(h, 2)) S c, for all E H, E Z. Then, I (0, 1), with probability at least 1 - we have for all h E H
loo((4T)
Line L (h) 2Rs(( H)) + 4c v
(29)
where
Rs(E H4h= B sup Σ
elhhy)..).
LeSHL 1=1
the Rademacher complexity of the hypothesis class when composed with the loss function, and Rademacher vector, that is, vector with i.i.d. entries taking the values 11 with equal probability.
24

---

To remove the dependency on the loss function E(-) and work solely with the Rademacher complexity of the hypotehesis class, we employ the well-known contraction principle for vector-valued functions:
Lemma B.8 ([28, Corollary 4)). Let H be of functions h x R,, f R R" a K K-Lipschitz function and [xil Then
V2KE sup Σ MaH i=
sup GEH
(x))(()),
(30)
where (4) and (Ex) are Rademacher sequences.
C Proofs of Main Results See.. 4
We dedicate this Section to the proofs of all theoretical results presented in Sec. 4.
C.1 Proof of Proposition 5
Proof. We prove (12) via induction. Firstly, we notice that |// + 1"I12-2 & 11/1l2-2 + 1/2122-2 s 1 v2 =v. Then, for = 1:
(Y A)ll= S vIlBIp S M/AIb222 Y All VAKKAAA A + +WW wy 112-2-
(31)
which holds by definition of (1). The invertibilityy of S W w and Theorem B. imply that
1115-1112-22 IIA A + -) + 122) 1- plls -112--21AA7A1p
p
p-plIAAAllaa :=y y.
(32)
where in the last inequality we used the fact that f-1 S 11---112--2 S a= due to the structure of W [23]. Substituting (32) into (31) yields ( I (||ll> + [|lll>)1A|ll-222 VB. Suppose now that
(12) holds for some & N. Then, for k +
I (Y A) 5001002--21 + A)lle + [1B[F)
sr((1 (1[++[[Mb==))llfW + A)lle + +[[B1())
A-1
sy (1 2Byp) MIAIl--201X AllF)y VB
((1 2Byp) + +1A112-2211Y + Allry vB
1=0
=||Y Alle/IAl1a-21 VB Σ
1'1 28yp).
which concludes the proof.
C.2 Proof of Theorem
Proof. Henceforth, we write f*) 1,, M,, B, to denote the dependence on W, (similarly for W2). Firstly, prove Lipschitz continuity of the perturbed intermediate decoder defined in (3). Due to the explicit form of the matrices @, M, B, the I-Lipschitzness of Sap(( Proposition 5, and the introduction of mixed terms, we obtain
IU + -f A
<11' + A) B₂) (1 4-1 1 A) B²)
25

---

-100f:100-40-001-755000000000000 Bi)le
[1 (Y A) B2)
-/00 - + +)+ + B1)llF
=|0 100---00000110001-----0000010010 11-10y (Y+ A)lr + 1|B2 - Billr
Alll + 1|B2 Bille =|10: + 11 -
V2(11B - BillF
+ +10021 A) +
O1 AIlF)
=|00 0111--2111 ( A) 112122221111 110-::00.__..11111---0------1110 1 A)lle
+ 22010>1l2-211/1 / (8 + -) (1 A)IF= + ||0 11112-1111 (x (Y A)llF)
+ +(1 + v2)] Bille
(Y s sv(10= 01ll2-2111 (Y + A) + 110112-21112 -Y A) - 11 + A)llF + +1IB3 +lBB2-Bill)) Billo)
-
10 - - A)ILe
sx 2 M22 Mill2-211f 11 A|+l
(33)
(on)
+ (1 + +Bypp)ll (Y A) (Y A)ll= + [|B, - Billz
(9)
with v = + v2 implied as in Appendix C.I. Since the proof becomes rather technical, for the sake of readability, we separate it into corresponding subsections from that point on.
C.2.1 Upper-bounding (s*)
IM M1lla-2 1IWWWA A DW, -WWTAA'A A + +W WW Ila-2
=PIlW(AAA A WW, W2(A"A A -WW(W))
+ W2(A" + +WW (' + pW? w/)'
S P pW? W2) (A - p. WW
(
+p + W/) W p w,) </llllb
According to Appendix C.I, we have
IIAA A + D - -I(AAA A pW, WW)
P
:=
PlAlA All--2
Therefore, for (f1), we introduce mixed terms obtain
[W AWW WI(A +
=|IW>(A A + pw?
pWW W W2(A A + pW WW)
W2(ATA + PW? W.)- - -
<1WW512-21K4a (ttl:///ll"++-///>>>/>//> W
+IWW1I2-221IA7 A > W,ll2-2
26

---

52y VBlls W1ll2-2.
For the term (f), due to Theorem B.2, we get
IIW((AAA + AA"A + +W/
SBI(A A + +W, - " wi)
<BplI(ATA A +DW] "WW/w))''l--2lIIA"" pw? 112 W2 w/ Will2-2
$26 yllls Will2-2,
where in the last inequality used the following derivation:
|W3 W2 W/ Willa-2 [I/ W2 - W3 W, + W W, w/ Wille-2 $ 22 vBlls
Will2-2-
Overall, for (**), it holds:
[| Mill2-2 s2yp VB + 2Byp)ll Will2-2-
(34)
C.2.2 Upper-bounding (9)
The introduction of mixed terms and Theorem B.1 yield
[B2 Bille =I| A W - WI(A pW) ( ADlk--2
<|AAlb--al//11+///333'''' pW, (s) Wi(A A PWT 112-2
p/ W2) Wi(A ( - A1ll--2
1/A11=-11//1//1//33/4444A4110 W A2(ATA+ PW/ Wi)
1 A W
W2(A"A+ "/) A
A WWW
12
W2(A A + + w.) W2) W1(A A + pW7 (;) A1l2-2
<|AA11-22 |LYTIA VOIIAA pW W2) - (A A p W1 - yl122 Will2-12
VBYIA= - + IAA +WW Wi(A A WWW W,)- '1l2-2
511AI12-2 (IYII VBOIIC + W 1 12 + 1 W3 WT Will2-2
+yIlW2- -yilW2-Will--e+ W -BYIA2- - Aill
+IAilleiIa"
- (A + PW7 Wi)-'1l-2
<11AIl2-2 (I7II(( VBOlICA A + +pw "11 + +WW Ws) 112211222 W.w, w Will--2
+YIlW2 +>llW2-Will-2) -BYIAz
Aille A
11o W2 - w/ W/lll-222
27

---

All in all, for [B: Bille, we obtain:
| - Bylle 11 (1 -Will--2
(35)
- +E(1 + 2Byp)1lW2 -will-22 Willz-2)
(0)
where E == Vse.
C.2.3 Upper-bounding (o)
=0- -1.111:. to make notation more compact for the time being. By definition of A,, i= 1,2, We write (() = we have
V,0U,0),) (,, XX)
V)(U)(,)X))
(36)
Consequently, for (36), due to Assumptions (a) and (c) of Sec. 3, get
V V,
V,(((())
V V,00 (Y,
SE
12
+ +W0.://0..0000000000000000000000
E
+
( XOIle
IV - -
22
2F(Bin B II (YLb
|,(( Vr(U)())
(37)
(T.))
where in the last inequality used the derivation Vy) ())))) - , for all W E Tg. Now, for (T.1), we get
())
12 - -
$ $RN0000-WNUUU000--0ff0-----vvv
+20
< 2(B - " A
$ 2/1A112-222 /I W
Wilk2-2
+ +1x2p)y)
2(B B (V)
(38)
(T:2)
where in the last inequality we used Proposition B.3 and Theorem B.4 the Lipschitz continuity of f (Y) with respect to W, with Σg being the Lipschitz constants up to an arbitrary layer k.
28

---

C.2.4 Upper-bounding (T.2)
According to Assumption (b) of Sec. 3, and due to the chain rule for composite functions, for the gradient of the softtthreshodding operator with respect to Y calculated at 0 := 0 (Y) + B(Y), for any FB. we have:
@ ( A> A/p 00
Vy(SN)/(0))
(39)
A/p.
We calculate VyB(Y) W(AA= (o) AA, and assume without loss of generality that the first clause of (39) holds, since otherwise simply get rid of an extra v2 term. The, the introduction of mixed terms, and the application of Theorem B. and Proposition B.3 yield
- 7
+ 0110://0000000000000000000000000
<||02(1' -1-="""00 10 1 VyBi())l
+ +02(1' +
51 + 2, -V) 4ryp VBO + 2Byp) (MIIsII1 -
+
W2(4 (,M W WI(A'A p Willl--2
Sx1 (, 4vyp VOLL +2BYy)IVy (OYLU/I W1llz-2
+ w|lb=:///WK(da"++,,////>/>>/-//-
W2)
+ + - W (/)ll//) W1)ll2-2
W2)
Sy( + 20 (Y) ( 4vyp VOL 2Byp)I (C3Ll/l Will2-2
+ 11AAl1222 W/ Will2-2 + /BpY21W2 Will2-2
Sy(1 2 - - -,S1 (MIle U
+ +-8211A112 BBly vB 2 PP)) Wilka-1 Σ (11 2Byp)
+ +yl1A|12-2(1 2By Wille-2
1V/ (1) vr ((D)l Bu12-l9llll--. Will2-2 Σ (rr)
+ + - Willz-2)
where we set 1 28yp. Now, for all & 2 1, we define
29

---

G =
(40)
k-1 D: G.
Do =.
(41)
im0
4 = G(8ry"p8114122220D00 + y1A112-2
(42)
so that
|V s GIVY VyIl +Zell22 -Wlll-2-
(43)
We prove via induction that
C2
Zez L2 I. k=1
(44)
First, notice that for 1, holds
|| [VrBHg,0) Vrsu (1Il Mlll
(I + (B)) - PVy
S vlIB2 BillF
@ryl1All2-211W3 Will2-2
= =ZIlW2 Will2-2
=C Will2-2-
SO Ci has indeed the form described in (44). Suppose that (44) holds for some L E N. Then, for L + 1:
|V s - ZL.1l Will2-2
$ (GCL W Will--2
G
G GL-RZe + Z14z |1 - Wille-2
k=1
GL-KZz - WW2-Will--
= CzilllW2 - Will2-2-
which proves that for any L E N, it holds
IV IV ( Vy (Yll s C,llW, Will2-2-
(45)
We combine the results from Sec. C.2.3 - C.2.4 to deduce:
E(Bin
Bout)
|| ||Al|2-2ry VBIDa (1All2-21y (BEADe-1
|| Ailr s2
(46)
x2
(B.,. Bow CC& lll Will2-2.
Hence, applying (46) in (35) of Sec. C.2.2 yields
30

---

||B: Bills S /111|| ||1X|1112222|||| + 2 vB
E(Bm + Boul)
||A112 (BD.)
(47)
x²
11All2-277-VIIDDDa (B + Bown
lw: Will2-2.
Now, we plug (34) and (47) in (33) the beginning of this proof to obtain
10 + A1) 150 Az)lle 1 (1) 11
A)lle + 11A112- &
y M||l|lr + rE + vB
E(Bin
Boml)
1|All2-22y VBDa
(48)
x²
(1A1l--2 DD-1 (Bmn + Bou CC, |Ilv Will2-2-
In order to treat all layers in a uniform manner, we set P( + A) 12 + A) = Y A. Similarly to our derivation for Ct (44), we
H₂ = y11All2-2 4r 4r
r + rE vB
E(Bin + BBuu)
1|A112-22y (BD,
x²
(49)
DD (B + Bou (C: c.))
kz 1.
with Σg. Dr. C defined in (26), (41), (44), respectively. Now, it is a matter of calculations to prove via induction that
Ki
Gl L2
(50)
First, for due to (1) and (47), we have
-)) + A2)lF
<||/'8, + 1' (B)) - 11 SAp < v Bille
Sry||All2-2 A| rE 2 E(Bin Bout)
||A||2-
VBI2A
x²
(1All2-277 De-1 (Bin Bin |lw
Will2-2
SHillW2 Will2-2
=KGIlW2 - Will2-2-
so K" has indeed the form described in (50). Let us suppose that (50) holds for some N. Then, for L+ 1:
1 1 (Y +2) ( f( ADII GY + -2)- ADlI= + HeallW, Will2-2
(GK, HI+D)) Will2-2
31

---

G
G + Herl I12
Will2-2
1+1
Σ
G H2 I1W2- Will2-2
Ki
Il2 Will2-2-
Therefore, for any L E N, it holds
(2) A(Y + ADilI s K2IlW Willa-2,
(51)
with K2 defined in (50) This means that the perturbed intermediate decoder (6) is Lipschitz continuous with respect to W. For the perturbed final decoder (7), the affine map Tw is by definition Lipschitz continuous, with Lipschitz constant satisfying
LiPTe [/Tw,ll2-2 11T,, LipTng 2yp VB.
(52)
Therefore, we introduce mixed terms to get:
[[T (fk (Y + -ww
AD)II
=I7 (w: (Y A2)) Uw. (Y A)) Tw, (Y AD) - Tw. (Y ADDle
3/ www.12-21lfff (Y A2)) fw, (Y A,)DE 11T Tw,, Tww.l2-221f4, (Y A
s2yp |BK W2 - Willz-2 + [VIAll2-220III E)y VBDu) I1Tw: - -Tw,l2-22 1l2-2
<2yp J0K, Will2-2
2p M1AIl2-2(1X1I1- E)y (DL II(A A + pW: W2) W (ATA pWW Willz-2
2yp NBK, 22 -llllll-2222111111 E) (BD1 II2 Will2-2-
(53)
Overall, the perturbed final decoder is Lipschitz continuous, and we denote its Lipschitz constants with Lip to indicate the dependence on both L and e. Consequently, we have proven that, for all L 2,
1 (Y + A1) 11g (Y A2)ll: Lip Wallz-2-
(54)
where
E Liph =2yp vD (m() ( rE + 2B(Bm. + Bout)
x -
H, + (
+ +E) E
with E. Vss, == (1 v2), = 1 28yp, y as in Proposition 5. as Definition and Hę defined in (49).
32

---

C.3 Proof of Proposition 7.
Proof. By Definition we have TB implies that
B (vB). Then, the application of Lemma B.5 for Tg 1:2--
vB
Nn
NO -.2.1-11--220 1
(55)
Therefore, due Theorem 6, the covering numbers of M are bounded follows:
/
Lo Nn
N(M.II 11,,)) N(Lip, (l)-2,)) N(F 1/Lipp
1
(56)
C.4 Proof of Theorem 8
Proof. The ARC has sub-gaussian increments, so we can use Dudley's integral inequality (28) to upper bound it in terms of the covering numbers of the set M defined in Sec. 4. To that end, we first calculate
a Euh((rs)
A(M) sup
B
sup Ref
E
(x,))
hejgt.
sup Reff
Bout
(57)
Then, we combine Proposition and Theorem B.6 to get:
4 V2
ViReat ?
Rs(H4
log N(M 1)F
J
Virgat
2 VVLipph Lo
Nn
1+
dt
(58)
which is the desired estimate.
C.5 Proof of Theorem
Proof. Due (18), and the inequality [11, Lemma C.9]
b - s Vlogle(1 + b/aa),
log 1 +
a,b> 0,
(59)
the following holds for the ARC:
Nn
2 VBLip 1
Rs(1l 12 - H44 so
log
exp
(60)
VsB
According Assumption (a), deduce that the loss function || 112 is upper-bounded by (Bin + B B The result follows by substituting (60) Theorem B.7, with the aforesaid c.
33

---

Adv. EGE
Overcompleteness factor
(a) CIFARIO
Noise level = 0.01
Noise level 0.1
Noise level 1
12000
Adv. EGE
22/00
12500
Overcompleteness factor
(b) SVHN
Figure 4: Adversarial generalization of ADMM-DAD measured terms of the adversarial empirical generaliza- tion error (21), alternating overeompleteness N and different attack levels both datasets, (21) increases as N also increases, like Theorem suggests, thus confirming our derived adversarial generalization theory.
D Experimental details
To encourage reproducibility of results, hereby complement the experimental settings of
Sec. 5.
We follow standard CS setup and employ normal Gaussian observation matrix A mx which we normalize as A/ Ym for the CIFAR 10 dataset, and as A fxx for the SVHN dataset. The ex- perimental parameters and p have been calibrated accordingly, account for the different structural specifications of each dataset. Therefore, for CIFAR 10, we set
and 2 = 10-4 while for SVHN we alternate p and A depending on the value of N. Particularly, for N [10, 20, 30, 40, 50], we == [10-5 14 10 10-3 10 and p = [100, 1. 1, 10], respectively. For all implementations, we employ the Adam algorithm [21], which constitutes stochastic opti- mization method that adaptively estimates lower-order moments of the gradient of the adversarial training MSE. All of Adam's parameters are set to their default values, except for the learning rate éi. Specifically, for the CIFAR10 dataset, we train the 5- and 10-layr ADMM-DAD with 10-5 and fi- 10-4, respectively. For the SVHN dataset, we train the 10- and 15-layer ADMM-DAD with e, 14 and er = 10-3, respectively. We train all models on all datasets using carly-stopping with
34

---

respect to the adversarial empirical generalization error (adversarial EGE) (21). We repeat all the experiments least 10 times and average the results over the runs. For the comparisons with the baseline iST we set the best hyper-parameters proposed by the original authors. For the course of our experiments, we have utilized a node of H100 GPUs.
Adversarial generalization error with alternating N. For the sake of completeness, we present Figure 4 the scaling of the adversarial EGE (21), corresponding to the clean test MSEs (19) and the adversarial test MSEs (20) depicted in Figure 3, for increasing N, and three different values of g. Similarly to our discussion in Sec. 5, we observe that the adversarial EGE increases as and increase, for both datasets, thereby corroborating our theoretical derivations for the adversarial generalization of ADMM-DAD.
E
Impact Statement
Our work contributes to the theoretical understanding of adversarial robustness in DUNs, which designed to solve inverse problems like CS. While the research is primarily theoretical, provides key insights that could help improve the reliability and robustness of neural networks in high-stakes applications, such as medical imaging, Given the theoretical and exploratory nature of our study, it does not pose any foreseeable societal risks in the near term. Instead, lays the groundwork for future robust machine learning systems, enjoying enhanced interpretability and resilience to adversarial attacks.
35


Reviews:
[{"review_id": "gUlhE6eh4I", "reviewer": "Reviewer_2Rop", "summary": "This paper makes a theoretical contribution to the generalization of Deep Unfolding Network (DUN) under an adversarial setting. In particular, the paper focuses on a specific DUN, namely ADMM-DAD, solving the inverse problems in compressed sensing. The paper’s main result is bounding the adversarial generalization error bounds of ADMM-DAD. The paper presents several theoretical results, including the Lipschitz continuity of the ADMM-DAD decoder and an upper bound for the adversarial Rademacher complexity, to describe how the authors derived their main result. With experiments, the paper validates the theoretical findings.", "questions": "1. Please capitalize the title properly.\n2. Figure 1 would look better if a log scale were used. In particular, the authors observed that the empirical generalization error (EGE) grows at the rate of $\\sqrt{L\\log(\\varepsilon)}$, so the log-scale would show the relation between $\\log(\\varepsilon)$ and EGE better.\n3. Is it possible to use more powerful attacks in the experiments? In particular, I’m curious about the robustness of ADMM-DAD against more powerful attacks. Because the attackers would consider attacks stronger than FGSM, such an investigation would add more practical value to this work.", "limitations": "I don’t find any specific limitations or negative societal impacts of this work.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "### Strengths\n**Quality**: The experiments were performed on real-world image datasets, namely CIFAR-10 and SVHN, which is essential to demonstrate the practical value of the findings.\n\n**Clarity**\n1. The paper presents the theorems in separate boxes, which helps readers capture the main contributions.\n2. Also, the paper emphasizes the meaning (or importance) of each theoretical finding. This also clarifies the key takeaways from each result.\n\n**Significance**: The paper investigates the problem of adversarial learning in a relatively new application, i.e., DUN.\n\n**Originality**: To the best of my knowledge, the paper presents novel ideas.\n\n---\n### Weaknesses\n**Significance**\n1. The contribution is limited to a particular application of DUN, i.e., the inverse problem of compressed sensing.\n2. FGSM might be too weak to represent the solution that maximizes the error. However, I also agree that theoretical analysis becomes extremely difficult for more sophisticated attacks.", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the Reviewer for their kind words, we highly appreciate them.\n\nRegarding the Reviewer's questions:\n1. *\"The contribution is limited to a particular application of DUN, i.e., the inverse problem of compressed sensing.\"*\n\nOur choice of a compressed-sensing-tailored DUN is attributed to the fact that compressed sensing (CS) models a plethora of modern applications. For instance, CS is employed in secure data transmission and cryptographic protocols [1], communications in 5G networks [2], and tomographic imaging scans [3], to name but a few. In fact, all these applications must be addressed within a framework guaranteeing safe and reliable inference, to prevent catastrophic failures. Compared to other inverse problems, CS is particularly vulnerable to adversarial perturbations, due to its reliance on sparse and highly undersampled data, making it a natural stress test for robust model design. Notably, CS methodology has inspired the original paper introducing DUNs [4], with CS-based DUNs already being integrated into real-world CS pipelines, e.g., accelerated MRI in hospitals, where inference must be fast, interpretable, and trustworthy. Overall, due to its power of accommodating a wide range of applications, CS is an excellent paradigm showcasing the unfolding regime, with particular interest in how DUNs can be strategically designed to be both efficient and resilient.\n\n2. *\"Please capitalize the title properly.\"*\n\nWe thank the Reviewer for the elegant suggestion; we will capitalize the title of our paper accordingly, to meet the conference's formatting.\n\n3. *\"Figure 1 would look better if a log scale were used. In particular, the authors observed that the empirical generalization error (EGE) grows at the rate of $\\sqrt{L\\log(\\varepsilon)}$, so the log-scale would show the relation between $\\log(\\varepsilon)$ and EGE better.\"*\n\nWe will encompass the Reviewer's suggestion in our revised manuscript, by using a log-scale in Figure 1, in order to better indicate the logarithmic scaling of our derived bounds.\n\n4. *\"FGSM might be too weak to represent the solution that maximizes the error. However, I also agree that theoretical analysis becomes extremely difficult for more sophisticated attacks. Is it possible to use more powerful attacks in the experiments? In particular, I’m curious about the robustness of ADMM-DAD against more powerful attacks. Because the attackers would consider attacks stronger than FGSM, such an investigation would add more practical value to this work.\"*\n\nDue to the fact that ADMM-DAD recasts the reconstruction problem of compressed sensing as a nonlinear regression one, we aim to deploy attacks of broader utility, rather than based on a classification paradigm. To that end, we choose FGSM, which relies on the gradient of the loss function and is a state-of-the-art attack, deployed for delivering adversarial generalization error bounds, as well as a baseline attack accommodating both classification and regression settings. Apart from FGSM, PGD-based attacks can also be used in a regression setting, so we harnessed time during the rebuttal period to conduct relevant experiments, in order to assess the adversarial robustness of ADMM-DAD against more powerful attacks - yet complying with regression. Due to our theoretical results being based on FGSM, we illustrated only these attacks; however, in order to highlight the potential of our model to be adversarially robust against different attacks, we provide in the table below the results from adversarially training and testing ADMM-DAD in the presence of an $l_2$-norm PGD, e.g., with 10 iterations, for CIFAR10 (top) and SVHN (bottom):\n```\n     |Clean MSE  | Adv MSE  |EGE×10⁻⁵\n-----|-----------|----------|---------\nε\\L     |5/10      |5/10     |5/10\n.01 |.0287/.0185|.0294/.019 |.8/2\n.05 |.0302/.0234|.0311/.0242|1.9/1.5\n.1  |.0331/.0261|.034/.027 |2.6/1.8\n.5  |.035/.0264 |.0347/.028|4.2/4.5\n1   |.036/.0281 |.037/.0292|5.7/6.3\n----|-----------|----------|---------\n ε\\L |10/15     |10/15    |10/15\n.01 |.0135/.0102|.0148/.011|30/186\n.05 |.0149/.0113|.0151/.0129|38/203\n.1  |.0151/.0136|.0163/.0145|39/211\n.5  |.0142/.0128|.0145/.0137|86/218\n1   |.016/.0145 |.0173/.0155|87/230\n```\nIn the first revision oppportunity, we will gather these results in a figure, similar to the results presented in Figure 1 of the submitted manuscript, to showcase the scaling of the test and empirical generalization errors, as the noise level of the PGD attack increases.\n\n[1] Yuge, N., et al., 2025. Privacy Preserving Deep Unrolling Methods and Its Application to Image Reconstruction. IEICE Trans. Fundam. of Electron., Commun. & Comput. Scie., 108(3), pp.304-312.\n\n[2] Xu, B., et al., 2024. Deep Unfolding Beamforming and Power Control Designs for Multi-Port Matching Networks. IEEE Trans. Wireless Commun.\n\n[3] Yang, L., et al., 2025. Distillation guided deep unfolding network with frequency hierarchical regularization for low-dose CT image denoising. Neurocomputing, p.130535.\n\n[4] Gregor K, LeCun Y. Learning fast approximations of sparse coding. Proc. 27th Int. Conf. Mach. Learn., pp. 399-406, 2010."}, {"review_id": "sWOCSOfa8f", "reviewer": "Reviewer_mH3W", "summary": "The paper presents a study that offers a theoretical analysis and understanding of unfolding networks under adversarial attacks. In particular, the authors address the problem of adversarial generalization by considering FGSM with an L2 norm. The main objective is to characterize the error bounds of the network in the presence of such attacks. The primary tool used for this analysis is the adversarial Rademacher complexity, which is employed to quantify model complexity under worst-case perturbations. Based on this, insights regarding the lipschitz properties and error bounds are derived. An experimental analysis is also provided to support the theoretical findings.", "questions": "Considering the weaknesses outlined above, the main questions I would like to raise for the authors are:\n\n- Can you clearly address the main limitations of the proposed analysis from a practical perspective? Specifically, what are the concrete benefits and implications of this study in real-world settings? Additionally, could you clarify the practical significance of the assumptions made in your theoretical framework, and how restrictive they are in practice?\n\n- Why is the analysis limited to FGSM with an L2 norm? FGSM is traditionally associated with the L-inf norm, as it relies on the sign of the gradient. Could you clarify what is meant by an “FGSM L2” attack in your setting? Did you intend to refer to a one-step L2 norm PGD attack without random initialization? Furthermore, could this analysis be extended to other adversarial settings (e.g. multistep attacks as PGD), which are also relevant and widely studied in the community?", "limitations": "As reported above, a more thorough discussion of the practical limitations of the work is required.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 1, "significance": 2, "originality": 3, "strengths_and_weaknesses": "STRENGTHS\n\n- The paper presents strong theoretical insights which, in my understanding, provide useful bounds that could help advance the understanding and potential certification of model robustness.\n-Despite the complexity of the theoretical content, the soundness of the work appears to be respected.\n\n\nWEAKNESSES\n\n- The paper would benefit from a clearer introduction that better motivates the purpose of this theoretical study. While the analysis appears rigorous, a stronger contextual explanation of its practical relevance, particularly in the domain of DUNs, is needed. For example, in the first section, the authors should better clarify the importance of addressing from a practical perspective the question: \"What is the generalization performance of DUNs in the presence of adversarial attacks?\"\n\n- After the bullet points listing the contributions in the introduction, the authors include a discussion that effectively resembles a \"Related Work\" section. However, it is currently embedded in the introduction, which makes the flow of the paper a bit more complex. In future versions, the authors should consider putting this part into a dedicated. \n\n\n- Overall, the paper is difficult to follow. Although it is clear that the authors made a significant effort, a structure that could introduce the problem to a broader audience and highlights the practical implications of the findings would be appreciated.\n\n- In Theorem 3, the authors introduce assumptions that should be further discussed from a practical perspective, specifically, how critical these assumptions are, and how they distinguish theoretical results from practical applicability. Additionally, it should be clarified why the proposed analysis is limited to FGSM with an L2 norm. Can it be extended to other adversarial settings?", "was_revised": true, "final_justification": "I thank the authors for the very clear and constructive rebuttal. While my previous comments mainly concerned the presentation and the correctness and strength of the posed assumptions, the authors have clearly clarified these points in their response. \nConsidering these, my current score has improved to borderline accept.", "rebuttal": "We would like to thank the Reviewer for their constructive comments, which we appreciate.\n\nRegarding the Reviewer's questions:\n- *\"The paper would benefit from a clearer introduction that better motivates the purpose of this theoretical study. While the analysis appears rigorous, a stronger contextual explanation of its practical relevance, particularly in the domain of DUNs, is needed. For example, in the first section, the authors should better clarify the importance of addressing from a practical perspective the question: \"What is the generalization performance of DUNs in the presence of adversarial attacks?\"  \nIn Theorem 3, the authors introduce assumptions that should be further discussed from a practical perspective, specifically, how critical these assumptions are, and how they distinguish theoretical results from practical applicability. Additionally, it should be clarified why the proposed analysis is limited to FGSM with an L2 norm. Can it be extended to other adversarial settings?  \nCan you clearly address the main limitations of the proposed analysis from a practical perspective? Specifically, what are the concrete benefits and implications of this study in real-world settings? Additionally, could you clarify the practical significance of the assumptions made in your theoretical framework, and how restrictive they are in practice?\"*\n\nA core practical challenge of the proposed analysis is that it relies on Rademacher complexity, which is a worst-case complexity measure, leading to bounds holding uniformly over the hypothesis class. Still, despite the fact of being worst-case estimates, our adversarial generalization error bounds provide a convincing partial answer to how robust DUNs are in the presence of adversarial attacks. Importantly, they constitute significant theoretical baselines, as they can be leveraged to deliver tighter adversarial generalization error bounds - we leave this idea as a promising future work and briefly mention it in lines 327-329 of Sec. 5.\n\nRegarding the concrete benefits and implications of our study in real-world settings: CS, which DUNs are designed to address, finds application in domains, e.g., medical imaging, communications, cryptography, where safe and reliable inference is of utmost importance. For example, by leveraging the inherent overparameterization of DUNs to mitigate the effect of adversarial attacks, we provide stronger theoretical foundations for using DUNs in security-critical systems, protecting them against malicious manipulation. in fact, our result that overparameterization (via overcomplete sparsifiers) improves robustness offers a new training paradigm: instead of blindly increasing model size, one can strategically design DUNs to be efficient and resilient, thereby reducing reliance on trial-and-error adversarial training. All in all, to better position our paper w.r.t. the practical importance of addressing our main research question, we will add in the revised version the following paragraph, at the end of the contribution bullet points:\n\n*\"Overall, our proposed study provides the first theoretical framework for understanding and improving the adversarial robustness of DUNs in CS, with direct implications for safety-critical applications. In medical imaging, for example, our results could help ensure that MRI reconstructions remain reliable even under adversarial perturbations, reducing the risk of misdiagnosis. Furthermore, by showcasing that overparameterization via learnable overcomplete sparsifiers improves robustness, our work offers concrete design principles for developing resilient, interpretable networks in real-world deployments, where accuracy, efficiency, and security must coexist.\"*\n\nOn to the practicality of Assumptions (a) – (c): real-world images, e.g., CIFAR10 and SVHN, are naturally bounded in the $l_2$-norm. Then, we empirically observe the boundendess, w.r.t. the $l_2$-norm, by some constant $\\mathrm{B}_\\mathrm{out}>0$, of the output of the adversarially perturbed decoder. In fact, such boundedness assumptions for the input-output data are common when studying generalization [1-3]. Moreover, we are inspired by a recent study [4] stating that, from the applied perspective, even if the gradient of a ReLU-type activation - the soft-thresholding operator can be written as the sum of two shifted ReLUs - is zero, the training behaviour of the net is not affected. Finally, according to the code snippet creating an $l_2$ FGSM attack, e.g., from the state-of-the-art package cleverhans [5], developed for generating popular adversarial attacks, a division by zero is avoided. That is, for a model $f$ and input-output data $(Y,X)$, one can ensure that $\\|\\|\\nabla_Y\\ell(f(Y),X)\\|\\|_F\\neq0$, with $\\ell(\\cdot)$ being the chosen loss; thus, the $l_2$ attack is well-defined.\n- *\"After the bullet points listing the contributions in the introduction, the authors include a discussion that effectively resembles a \"Related Work\" section. However, it is currently embedded in the introduction, which makes the flow of the paper a bit more complex. In future versions, the authors should consider putting this part into a dedicated.\"*\n\nTo increase readability of our manuscript, we will revise it to include all related work, starting from line 78 of the submitted manuscript, in a separate Section.\n- *\"Overall, the paper is difficult to follow. Although it is clear that the authors made a significant effort, a structure that could introduce the problem to a broader audience and highlights the practical implications of the findings would be appreciated.\"*\n\nWe appreciate the effort the Reviewer put on thoroughly reading our paper, so we address the aforesaid concern in detail in the answer of the 5th bullet-point.\n- *\"Why is the analysis limited to FGSM with an L2 norm? FGSM is traditionally associated with the L-inf norm, as it relies on the sign of the gradient. Could you clarify what is meant by an “FGSM L2” attack in your setting? Did you intend to refer to a one-step L2 norm PGD attack without random initialization? Furthermore, could this analysis be extended to other adversarial settings (e.g. multistep attacks as PGD), which are also relevant and widely studied in the community?\"*\n\nOur choice of an $l_2$ FGSM attack, instead of $l_\\infty$, relies on two main reasons. Firstly, as is usually done in the compressed sensing (CS) literature [6], for the (Gaussian) noise $e$ added to the CS observations $y$, i.e., $y=Ax+e$, it holds $\\|\\|e\\|\\|_2≤\\eta$, $\\eta>0$ - this detail can also be found in Sec. A.1 of the paper's supplemental material. As we move from the model-based to the data-driven CS, one may notice that perturbing ADMM-DAD with adversarial noise tantamounts to adding adversarial noise to the CS observations. As such, it seems natural to constrain the adversarial noise in the $l_2$ ball, complying with the investigation of the adversarial robustness of models solving the CS problem [7-8]. What is more, we are inspired by relevant settings [9], where a model's adversarial generalization is addressed under $l_2$ FGSM attacks. Particularly, an $l_2$ FGSM attack $\\delta$ pertains to normalizing the gradient of the loss to have $\\varepsilon>0$ Euclidean norm. Overall, we stress our choice of an $l_2$ attack in lines 169-173 of the submitted manuscript. Due to the costly, iterative nature of PGD, and our aim to be consistent with out theoretical framework, we preferred to demonstrate only the results from FGSM, found in Figure 1 of the submitted manuscript. However, we trained and tested ADMM-DAD with an $l_2$ PGD attack, to showcase the adversarial robustness of ADMM-DAD against more powerful attacks. As we realise that also referring to the PGD, e.g., with 10 iterations, results would better position our paper, for the Reviewer's convenience, we present them in the table below, for CIFAR10 (top) and SVHN (bottom):\n```\n    |Clean MSE  | Adv MSE  |Adv. EGE×10⁻⁵\n----|-----------|----------|---------\nε\\L      5/10      5/10       5/10\n.01 |.0287/.0185|.0294/.019 |.8/2\n.05 |.0302/.0234|.0311/.0242|1.9/1.5\n.1  |.0331/.0261|.034/.027 |2.6/1.8\n.5  |.035/.0264 |.0347/.028|4.2/4.5\n1   |.036/.0281 |.037/.0292|5.7/6.3\n----|-----------|----------|---------\n ε\\L  10/15       10/15      10/15\n.01 |.0135/.0102|.0148/.011|30/186\n.05 |.0149/.0113|.0151/.0129|38/203\n.1  |.0151/.0136|.0163/.0145|39/211\n.5  |.0142/.0128|.0145/.0137|86/218\n1   |.016/.0145 |.0173/.0155|87/230\n```\nFor illustrative purposes, we will create a plot - similar to Figure 1 - from the previous table, and place it in Sec. 4 of our revised manuscript.\n\n[1] Behboodi, A., et al., 2022. Compressive sensing and neural networks from a statistical learning perspective. Compressed Sensing in Inf. Process., pp. 247-277\n\n[2] Kouni, V., Panagakis, Y., 2025. Generalization analysis of an unfolding network for analysis-based Compressed Sensing. Appl. & Comput. Harmon. Anal., 101787\n\n[3] Joukovsky, B., et al., 2021. Generalization error bounds for deep unfolding RNNs. Uncertainty Artif. Intell., pp. 1515-1524\n\n[4] Bertoin, D., et al., 2021. Numerical influence of ReLU’(0) on backpropagation. Adv. Neural Inf. Process. Syst., 34, 468-479\n\n[5] Papernot, N., et al., 2016. Technical report on the cleverhans v2. 1.0 adversarial examples library. arXiv preprint arXiv:1610.00768\n\n[6] Foucart, S., Rauhut, H., 2013. An invitation to compressive sensing. A mathematical introduction to compressive sensing, pp. 1-39\n\n[7] Raj, A., et al., 2020. Improving robustness of deep-learning-based image reconstruction. Int. Conf. Mach. Learn., pp. 7932-42\n\n[8] Wang, Y. et al., 2024. Benign overfitting in adversarial training of neural networks. Int. Conf. Mach. Learn.\n\n[9] Farnia, F., et al., 2018. Generalizable adversarial training via spectral normalization. arXiv preprint arXiv:1811.07457."}, {"review_id": "isaoNPvKcG", "reviewer": "Reviewer_YdTx", "summary": "This paper investigates the adversarial generalization of unfolding networks, where the adversarial perturbations are generated by the FGSM method, constrained in $l_2$-norm. They provide the adversarial generalization error bounds through the lens of adversarial Rademacher complexity and covering numbers. Experimental results on real-world data validate the theoretical findings.", "questions": "Refer to the weaknesses.", "limitations": "It is necessary to consider more general forms of $l_p$-norm adversarial attacks for arbitrary values of $p\\geq 1$.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strength:**\n+ This paper provides the first generalization analysis on the adversarial generalization of unfolding networks.\n+ A series of experimental results corroborate the theoretical findings.\n\n**Weakness:**\n+ **Scalability of the theory.** This paper mainly focuses on $l_2$-norm constrained attacks. How difficult is it to extend the generalization analysis to the $l_p$ norm attack for arbitrary $p\\geq 1$? Furthermore, FGSM is a well-known $\\ell_\\infty$-norm-based attack. It remains unclear why FGSM is specifically chosen to generate adversarial perturbations in this work, rather than employing other classical $l_2$-norm attacks such as the Carlini & Wagner (C&W) method. Please clarify this.\n+ **The novelty of theoretical techniques.** What are the differences and improvements in the analytical techniques presented in this paper compared to those in (Xiao et al. 2021), especially for addressing outer maximization of adversarial losses?\n+ **Comparison of the adversarial generalization bounds. (Table 1)** For a fair comparison, the proposed bounds should explicitly reflect the dependence on the norm of the parameter matrix. Additionally, the factor $\\rm{Lip}_h^{L,\\epsilon}$ should be given in the main text.\n+ **Experiment.** There is a lack of a strong connection between the theoretical findings and the experimental section. In particular, the experimental section overlooks how the empirical error relates to the derived generalization bound. Is it possible to provide a direct comparison between the derived bounds and the observed generalization (e.g., augmenting some graphs with the values ​​suggested by the bounds)?\n+ **Attack methods.** It would improve the experimental part if more attack methods were considered in this paper, such as PGD, C&W, etc.\n\nThe overall content of the paper is relatively dense, and the excessive use of symbols affects the overall readability.\n\nXiao et al. Bridging the gap: Rademacher complexity in robust and standard generalization. In 37th Annu. Conf. Learn. Theory, pp. 5074–5075. PMLR, 2024.", "was_revised": true, "final_justification": "My main focus is on the improvement and scalability of the theoretical results, which have been adequately addressed in my opinion.", "rebuttal": "We would like to thank the Reviewer for their valuable comments, which we appreciate.\n\nRegarding the Reviewer's questions:\n\n- *\"**Scalability of the theory**. This paper mainly focuses on $l_2$-norm constrained attacks. How difficult is it to extend the generalization analysis to the $l_p$ norm attack for arbitrary $p\\geq 1$? Furthermore, FGSM is a well-known $l_\\infty$-norm-based attack. It remains unclear why FGSM is specifically chosen to generate adversarial perturbations in this work, rather than employing other classical $l_2$-norm attacks such as the Carlini & Wagner (C&W) method. Please clarify this.\"*\n\nWe opt for an $l_2$ FGSM attack, as it is more relevant to the noise model of compressed sensing (CS). In the CS literature [1], with (Gaussian) noise added to the CS observations, i.e., $y=Ax+e$, it holds $\\|\\|e\\|\\|_2\\leq\\eta$, $\\eta>0$; this detail is also found in Sec. A1 of our supplemental material. As we point out in lines 169-172 of our submitted manuscript, adversarially perturbing ADMM-DAD is equivalent to adding adversarial noise to $y$. Thus, it seems natural to transfer the noise model of standard CS to the adversarial setting. In fact, it is usual to investigate the robustness of models solving the CS problem against $l_2$, e.g., PGD, attacks [2-3]. However, $l_2$ attacks like C&W are crafted to promote a misclassification effect; this, in turn, is quite irrelevant to our setting, since ADMM-DAD aims to find a nonlinear function mapping $y$ to $x$, i.e., solves a regression task. Since FGSM is a state-of-the-art attack applicable in regression, and serves as a baseline for theoretical investigation and experimental comparison, we deem its choice, with under an $l_2$-norm, to be the most appropriate for our setup. Extending these results to different normed attacks would require a distinct treatment. For example, $p=1$ leads to a sparse attack, which requires to examine its support, in order to leverage the full strength of the adversary. Although this is a promising direction for future research, it lies beyond the scope of the present paper.\n\n- *\"**The novelty of theoretical techniques**. What are the differences and improvements in the analytical techniques presented in this paper compared to those in (Xiao et al. 2021), especially for addressing outer maximization of adversarial losses?\"*\n\nThe outer-maximization shell of our approach is completely different to that of (Xiao et al. 2021), which considers the adversarial generalization error of a hypothesis class incorporating the max operation. To cope with the latter, the authors propose to uniformly cover the perturbation set appearing in that class. Although we also deploy covering numbers for estimating the adversarial generalization error, we follow an intrinsically different trajectory: we tackle the maximization by working with a surrogate loss [4-5]. Our key innovation compared to (Xiao et al. 2021) is that before employing covering numbers, we establish the Lipschitzness of the function implemented by the adversarially perturbed DUN, w.r.t. the parameter matrix. This Lipschitzness result, serving as a structural property, enables tighter adversarial generalization error bounds, via a refined covering number treatment and analysis. We describe this framework in lines 176-184 of the submitted manuscript, to better highlight our analytical technique.\n\n- *\"**Comparison of the adversarial generalization bounds. (Table 1)** For a fair comparison, the proposed bounds should explicitly reflect the dependence on the norm of the parameter matrix. Additionally, the factor $\\text{Lip}\\_h^{L,\\varepsilon}$ should be given in the main text.\"*\n\nWe will revise our manuscript according to the Reviewer's suggestion, and incorporate the bound $\\sqrt{\\beta}$ of the spectral norm of the parameter matrix in Table 1, which will read as $\\mathcal{O}(\\sqrt{L\\log(\\beta\\varepsilon)})$. Moreover, we will transfer the factor $\\text{Lip}\\_h^{L,\\varepsilon}$, presently found as eq. ($\\ddagger$) in Sec. C.2. of the supplemental material, to appear in Theorem 6 of Sec.3.\n\n- *\"**Experiment**. There is a lack of a strong connection between the theoretical findings and the experimental section. In particular, the experimental section overlooks how the empirical error relates to the derived generalization bound. Is it possible to provide a direct comparison between the derived bounds and the observed generalization (e.g., augmenting some graphs with the values ​​suggested by the bounds)?\"*\n\nFrom the theoretical perspective, we employ adversarial Rademacher complexity (ARC) - defined in eq. (9) of the submitted manuscript - of the hypothesis class, to provide a convincing partial answer to how adversarially generalizable DUNs are. ARC is a fundamental tool in studying adversarial generalization, via estimating the adversarial generalization error $\\tilde{\\text{GE}}(h)=|\\mathcal{\\tilde{L}}\\_{\\text{train}}(h)-\\mathcal{\\tilde{L}}\\_{\\text{true}}(h)|$, which in our FGSM context, is $$\\tilde{\\text{GE}}(h)=|(1/s)\\sum_{i=1}^s\\|\\|h_W(y_i+\\delta_i^{\\text{FGSM}})-x_i\\|\\|_2^2-\\mathbb{E}\\_{(x,y)\\sim\\mathcal{D}}\\|\\|h_W(y+\\delta^{\\text{FGSM}})-x\\|\\|_2^2)|.$$ Adversarial generalization error bounds based on ARC hold uniformly over the hypothesis class, thereby establishing ARC as a worst-case complexity measure. We also apply techniques from asymptotic analysis to show that $\\tilde{\\text{{GE}}}(h)\\leq\\mathcal{O}(\\sqrt{(NL\\log(\\varepsilon)/s)})$; this is presented in Corollary 4 of the submitted paper.\n\nFrom the practical perspective, a standard approach of empirically examining the generalization of neural nets in the standard learning regime [6] relies on approximating the true loss with the test loss. Similarly, we approximate $\\mathbb{E}\\_{(x,y)\\sim\\mathcal{D}}\\|\\|h_W(y+\\delta^{\\text{FGSM}})-x\\|\\|_2^2)$ with the adversarial test loss (given in eq. (19) of the submitted manuscript) $\\tilde{\\mathcal{L}}\\_{\\text{test}}(h)=(1/d)\\sum\\_{i=1}^d\\|\\|h(y'_i+\\delta_i')-x'_i\\|\\|_2^2$, for test data $\\{(y'_i,x'_i)\\}\\_{i=1}^d$. Then, the adversarial empirical generalization error $\\tilde{\\text{EGE}}(h)=\\tilde{\\mathcal{L}}\\_{\\text{test}}(h)-\\mathcal{\\tilde{L}}\\_{\\text{train}}(h)$, depicted in eq. (20) of the submitted manuscript and used to examine ADMM-DAD's adversarial generalization, approximates $\\tilde{\\text{{GE}}}(h)$. Given that we primarily aim to gain first theoretical and empirical insights in DUN adversarial generalization, our respective bounds do not need to be tight - although it is a promising future direction, mentioned in lines 327-329 of our submitted manuscript.\n\nConsequently, from the mathematical viewpoint, we employ the powerful notion of the ARC, albeit being a worst-case - complexity measure, to deliver adversarial generalization error bounds. From the practical side, we leverage the common - in standard learning literature - technique of approximating the true generalization error with its empirical counterpart, and adapt it accordingly for the adversarial learning regime, to experimentally elaborate on our theory. As such, there is no gap between our theoretical and our experimental results. Nevertheless, we understand that elaborating more on approximating the adversarial true error with the adversarial test error will better position our paper, so we will incorporate a version of the above discussion at the first revision opportunity, after eq. (20) found in Sec. 4.\n- *\"**Attack methods**. It would improve the experimental part if more attack methods were considered in this paper, such as PGD, C&W, etc.\"*\n\nGiven that the only well-known attacks suitable for regression are FGSM and PGD, and to be consistent with our theory and due to the costly nature of PGD, we preferred to adhere to FGSM for our experiments, illustrated in Figure 1 of the submitted paper. However, as we realise that perturbing ADMM-DAD with PGD would add value to our work, for the Reviewer's convenience, we indicate in the following table the results obtained after training and testing ADMM-DAD with $l_2$-norm PGD, e.g., with 10 iterations, for CIFAR10 (top) and SVHN (bottom):\n```\n    |Clean MSE  | Adv MSE  |Adv. EGE×10⁻⁵\n----|-----------|----------|---------\nε\\L      5/10      5/10       5/10\n.01 |.0287/.0185|.0294/.019 |.8/2\n.05 |.0302/.0234|.0311/.0242|1.9/1.5\n.1  |.0331/.0261|.034/.027 |2.6/1.8\n.5  |.035/.0264 |.0347/.028|4.2/4.5\n1   |.036/.0281 |.037/.0292|5.7/6.3\n----|-----------|----------|---------\n ε\\L  10/15       10/15      10/15\n.01 |.0135/.0102|.0148/.011|30/186\n.05 |.0149/.0113|.0151/.0129|38/203\n.1  |.0151/.0136|.0163/.0145|39/211\n.5  |.0142/.0128|.0145/.0137|86/218\n1   |.016/.0145 |.0173/.0155|87/230\n```\nBased on these values, we will create a plot similar to Figure 1, and add it in Sec. 4 of the revised manuscript.\n\n[1] Foucart, S., Rauhut, H., 2013. An invitation to compressive sensing. A mathematical introduction to compressive sensing, pp. 1-39\n\n[2] Raj, A., et al., 2020. Improving robustness of deep-learning-based image reconstruction. Int. Conf. Mach. Learn., pp. 7932-42\n\n[3] Wang, Y. et al., 2024. Benign overfitting in adversarial training of neural networks. Int. Conf. Mach. Learn.\n\n[4] Gao, Q., & Wang, X., 2021. Theoretical investigation of generalization bounds for adversarial learning of deep neural networks. J. Stat. Theory & Pract., 15(2), 51.\n\n[5] Yin, D., et al., 2019. Rademacher complexity for adversarially robust generalization. Int. Conf. Mach. Learn., pp. 7085-7094\n\n[6] Jakubovitz, D., et al., 2019. Generalization error in deep learning. Compressed sensing and its applications: 3rd Int. MATHEOn Conf., pp. 153-193"}, {"review_id": "uo4f6EHGZk", "reviewer": "Reviewer_PrVf", "summary": "This work presents the first adversarial generalization bound for unfolding networks—a class of neural networks with structured architectures tailored for specific inverse problems. The authors assume perturbations are bounded in the $l_2$ norm and are generated using the Fast Gradient Sign Method (FGSM). Under these assumptions, they provide the first theoretical analysis of the adversarial generalization properties of unfolding networks.", "questions": "Can the bound be extended to the precise attack rather than FGSM attack?", "limitations": "Yes.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. This paper is well-written and easy to follow. \n2. The proposed bound is tighter than before.\n\nWeaknesses\n1. The authors assume the FGSM attack, which is a weak attack in practice. It seems that the compared bound [1] did not assume the FGSM attack but used the precise maximal value. \n2. It would be better to explain what the overcomplete sparsifier is.\n3. There are some typos: \n\n\tLine 120, $O_{n\\times n}\\in \\mathbb{R}^n$ should be $O_{n\\times n}\\in \\mathbb{R}^{n\\times n}$\n\n\tLine 169, it lacks a right bracket.\n\n\tEquation 10, why is there a $log(exp(\\cdots))$, which can be simplified as $\\cdots$?\n\n\n[1] Xiao, J. et al. Bridging the gap: Rademacher complexity in robust and standard generalization.\nIn 37th Annu. Conf. Learn. Theory, pp. 5074–5075. PMLR, 2024.", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the Reviewer for their kind words and helpful comments; we appreciate both.\n\nRegarding the Reviewer's questions:\n\n1. *\"The authors assume the FGSM attack, which is a weak attack in practice. It seems that the compared bound [1] did not assume the FGSM attack but used the precise maximal value.\"*\n\nDepending on the context, FGSM attacks can be less powerful than other known attacks, e.g., Carlini & Wagner. However, our choice of attack is motivated by two main reasons. Firstly, ADMM-DAD operates in a regression setting, where given input-output data, it aims to learn a nonlinear function solving the compressed sensing problem. Due to the nature of the problem at hand, we opt for an attack that does not explicitly rely on a classification paradigm (thereby rendering it quite irrelevant to our setting), but rather deploy an attack with broader utility; the characteristic of FGSM being based on the gradient of the loss function renders it as a good candidate for our study.\n\nSecondly, due to the appearance of the max operation in the definition of the adversarial generalization error, i.e., $$\\tilde{\\text{GE}}(h)=|\\frac{1}{s}\\sum\\_{i=1}^s\\max\\_{\\|\\|\\delta\\|\\_p\\leq\\varepsilon}\\|\\|h\\_W(y\\_i+\\delta\\_i)-x\\_i\\|\\|_2^2-\\mathbb{E}\\_{(x,y)\\sim\\mathcal{D}}(\\max\\_{\\|\\|\\delta\\|\\|\\_p\\leq\\varepsilon}\\|\\|h\\_W(y+\\delta)-x\\|\\|\\_2^2)|,$$ it is standard in related literature to work around this subtle point by relying on a surrogate loss function - in particular, of FGSM type. Similarly to [2], we rely on FGSM to discard the max operation and estimate a max-free adversarial generalization error: $$\\tilde{\\text{GE}}(h)=|\\frac{1}{s}\\sum\\_{i=1}^s\\|\\|h_W(y_i+\\delta_i^{\\text{FGSM}})-x_i\\|\\|_2^2-\\mathbb{E}\\_{(x,y)\\sim\\mathcal{D}}\\|\\|h_W(y+\\delta^{\\text{FGSM}})-x\\|\\|\\_2^2)|,$$ with $\\delta^{\\text{FGSM}}$ being the FGSM-based attack. Instead of working with a surrogate loss, the authors of [1] bypass the difficulty of the max operation by uniformly covering the perturbation set. Despite the fact of deploying standard covering numbers to estimate the adversarial generalization error of ADMM-DAD, we differentiate our approach from the one presented in [1] by turning to an FGSM surrogate loss, and then prove and use as a cornerstone of our study the Lipschitz continuity of the examined DUN w.r.t. the parameter matrix. This technical refinement leads to tighter upper bounds for the adversarial generalization error compared to the ones presented in [1].\n\n2. \"*It would be better to explain what the overcomplete sparsifier is.\"*\n\nA matrix $W\\in\\mathbb{R}^{N\\times n}$ is a *sparsifier*/*sparsifying transform*, when its application on a given vector $x\\in\\mathbb{R}^n$ renders the latter sparse. The overcompleteness stems from the structure of $W$ per se, regarding the number of rows compared to the number of columns; that is, when $N>n$, we call $W$ overcomplete, following the intuition that its row vectors can form an \"overcomplete\" (not orthonormal) basis for the ambient space.\n\n3. *\"There are some typos: Line 120, $O_{n\\times n}\\in\\mathbb{R}^n$\n should be $O_{n\\times n}\\in\\mathbb{R}^{n\\times n}$. Line 169, it lacks a right bracket. Equation 10, why is there $log(exp(...))$, which can be simplified as $...$?\"*.\n\nWe thank the Reviewer for spotting the first two typos, we will correct them accordingly in the revised version of our manuscript. However, the expression $log(exp(...))$ is written as $log(exp\\cdot(...))$ in the submitted manuscript, so that the parenthesis is multiplied by the exponential factor, and then the product is the input to the logarithm.\n\n[1] Xiao, J., et al., 2024. Bridging the gap: Rademacher complexity in robust and standard generalization. 37th Annu. Conf. Learn. Theory, pp. 5074–5075\n\n[2] Gao, Q., & Wang, X., 2021. Theoretical investigation of generalization bounds for adversarial learning of deep neural networks. J. Stat. Theory & Pract., 15(2), 51."}]

Author rebuttal:

Meta-review: This paper presents the first theoretical analysis of adversarial generalization for unfolding networks, focusing on ADMM-DAD in compressed sensing and deriving adversarial error bounds via Rademacher complexity, supported by experimental validation. Reviewers found the contribution original and technically solid, with strengths in the clarity of theoretical results and experiments on real-world datasets. Main concerns prior to rebuttal included the reliance on FGSM (a relatively weak attack), limited generality beyond compressed sensing, and presentation issues such as a dense introduction and unclear motivation. The rebuttal clarified assumptions, justified the attack choice, and provided additional PGD experiments, which addressed most concerns. Overall, the work is recommended for poster acceptance, with suggested improvements in clarity, broader adversarial settings, and tighter theory–experiment integration.
Decision: Accept (poster)