No Prior, No Leakage: Revisiting Reconstruction Attacks in Trained Neural Networks
Authors: Yehonathan Refael
Guy Smorodinsky
Ofir Lindenbaum
Itay Safran
Venue: ICLR 2026 Poster (2026)
Keywords: Deep learning theory
Implicit bias
Training reconstruction attack
Data privacy
Data protection
Deep learning security
Primary area: learning theory

Abstract: The memorization of training data by neural networks raises pressing concerns for privacy and security. Recent work has shown that, under certain conditions, portions of the training set can be reconstructed directly from model parameters. Some of these methods exploit implicit bias toward margin maximization, suggesting that properties often regarded as beneficial for generalization may actually compromise privacy. Yet despite striking empirical demonstrations, the reliability of these attacks remains poorly understood and lacks a solid theoretical foundation. In this work, we take a complementary perspective: rather than designing stronger attacks, we analyze the inherent weaknesses and limitations of existing reconstruction methods and identify conditions under which they fail. We rigorously prove that, without incorporating prior knowledge about the data, there exist infinitely many alternative solutions that may lie arbitrarily far from the true training set, rendering reconstruction fundamentally unreliable. Empirically, we further demonstrate that exact duplication of training examples occurs only by chance. Our results refine the theoretical understanding of when training set leakage is possible and offer new insights into mitigating reconstruction attacks. Remarkably, we demonstrate that networks trained more extensively, and therefore satisfying implicit bias conditions more strongly -- are, in fact, less susceptible to reconstruction attacks, reconciling privacy with the need for strong generalization in this setting.

Published as a conference paper at ICLR 2026
No PRIOR, No LEAKAGE: REVISITING RECONSTRUC- TION ATTACKS IN TRAINED NEURAL NETWORKS
Yehonatan Refael
Guy Smorodinsky"
Department of Electrical Engineering Tel-Aviv University, Israel re refaelkall im@ma 1. tau. ac.
Faculty of Computer and Information Science Ben-Gurion University of the Negev, Isracl smo amorodin@@pos www.pptttt..........oo ac.il
Ofir Lindenbaum 1
Itay Safran
Faculty of Engineering
Faculty of Computer and Information Science Ben-Gurion University of the Negev, Israel safrani eafrani@bgu.. ac.
Bar-Ilan University, Israel
ofir. Lindenbaumbbuu.ac
ABSTRACT
The memorization of training data by neural networks raises pressing concerns for privacy and security. Recent work has shown that, under certain conditions, portions of the training set can be reconstructed directly from model parameters. Some of these methods exploit implicit bias toward margin maximization, suggesting that properties often regarded as beneficial for generalization may actually compromise privacy. Despite compelling empirical demonstrations, the reliability of these attacks remains poorly understood and lacks solid theoretical foundation. In this work, we take a complementary perspective: rather than designing stronger attacks, we analyze the inherent weaknesses and limitations of existing reconstruction methods and identify conditions under which they fail. We rigorously prove that, without incorporating prior knowledge about the data, there exist infinitely many alternative solutions that may lie arbitrarily far from the true training set, rendering reconstruction fundamentally unreliable. Empirically, we further demonstrate that exact duplication of training examples occurs only by chance. Our results refine the thcoretical understanding of when training-soe set leakage is possible and provide new insights into mitigating reconstruction attacks. Remarkably, networks trained more extensively, and therefore satisfying implicit bias conditions more strongly, are, in fact, less susceptible reconstruction attacks, reconciling privacy with the need for strong generalization in this setting.
demonstrate that
INTRODUCTION
Neural networks have achieved remarkable success across wide variety of tasks, but their use raises fundamental concerns of privacy (Abadi et al., 20T6b] Runkel et al.) 2024 Fang et al. 2024 Tan cet al.) 2024 Bombari Mondelli 2025). Recent work has demonstrated that portions of the training data can be reconstructed directly from the parameters of trained model, even without access to gradients or queries (Haim et al. 2022 Buzaglo al. 2023b Loo et al. 2023). Unlike some previous methods that produce generie reconstructions resembling class prototypes or averages (Carlini et al. 2021 2019). the new techniques generate highly accurate and specific reproductions the original training data. Such reconstruction attacks highlight the risk that sensitive private information may leak from models, undermining their safe deployment in practice.
Despite the alarming success of these attacks, theoretical understanding of them remains limited. In particular, the attack introduced in (Haim et al. 2022), along with some subsequent work inspired by it, such as (Buzaglo al. 2023b), builds results concerning the implicit bias of gradient-
based optimization in training homogeneous networks (Lyu & Li 2020 Ji & Telgarsky 2020
'Equal Contribution.
CCo-last authors.
"Specifically, focus on the KKT conditions for binary classification defined in Theorem

---

Published as a conference paper at ICLR 2026
Intuitively, when optimization succeeds under certain assumptions, such networks trained using standard techniques do not merely converge to any solution that fits the training set; instead, they converge to specific solutions that satisfy additional constraints. Building on this observation, (Haim eet al. 2022) constructs an objective function that is minimized when these constraints are satisfied and carries out their attack by optimizing it. Nevertheless, despite its theoretical foundations, our understanding of the conditions under which the attack succeeds remains rudimentary, as rigorous analyses of this setting are scarce.
Recently, Smorodinsky et al. (2025) provided rigorous guarantees on the efficacy of such a recon- struction attack. However, these guarantees rely on restrictive assumptions, such as a univariate data distribution, which may limit their practical applicability. This leaves many fundamental questions open and motivates several follow-up directions to establish clear conditions under which such attacks can be provably effective or provably mitigated. A central question that arises is:
To what extent do the constraints imposed by the implicit bias of trained neural networks leak information about the training data?
In this paper, we address the above question by rigorously studying properties of the objective function used to fuel the reconstruction attack introduced by Haim et al. (2022). We propose method that, given the original training data, enables the construction of multiple other global minima for the objective function used in the attack. Additionally. we demonstrate that under certain conditions, these global minima are ubiquitous, and that solutions to this optimization problem exist that are substantially different from the original training data Moreover, we show that the closer the network is to a solution, the harder it becomes to successfully execute such attacks, perhaps contrary to previous common wisdom. Lastly, we empirically demonstrate that if an attacker initializes the attack far from the true training data, the reconstructed instances are significantly farther from the training
data as well. This highlights that additional prior knowledge (as used by Haim et al. (2022) is crucial for successful reconstruction attacks in this setting, and that the constrants imposed by the implicit bias alone do not necessarily reveal information about the training data.
The rest of this paper is structured as follows: After specifying our contributions in detail below, turn discuss additional related work. In Section 2 present the required background and notation used throughout this paper. In Section 3 we theoretically study the set of feasible solutions the objective function used in implicit-bias-drivnn privacy attacks, and in Section 4 we empirically validate and support our theoretical findings. Lastly, Section 5 jummmrrizes our contributions and discusses potential limitations and future work directions.
Our contribution. We prove that the attack presented by Haim et al. (2022) has inherent limitations. In particular, we demonstrate that prior knowledge about the data domain is necessary to accurately recover the true training examples, and we empirically show several scenarios which these attacks fail. More specifically:
In Subsection 3.1 we demonstrate that in the reconstruction attack devised by Haim et al. (2022), there are numerous potential candidate training sets that seem indistinguishable from each other. We provide simple, constructive techniques for generating alternative candidates, either by merging two data instances into a single instance (Lemma or by splitting single instance into two (Lemma 3). Moreover, we demonstrate that under the assumption that the training data does not span the entire domain, there exist infinitely many such alternative training sets whose distance from the true training set can be arbitrarily large (Theorem 4 Lastly, we relax this assumption, assuming the training data only approximately lies on a linear subspace, and we further analyze how far a point can be split (Theorem 5)
we consider the more realistic setting in which the trained model only approximately satisfies the implicit bias constraints, and present merging and splitting techniques analogous to those mentioned in the previous bullet point (Lemmas and 7). We then assume that the attacker has limited knowledge of the proximity to an implicit-bias
In Subsection 3.2
P is noteworthy that this result answers a question which was raised in Haim et al. 2022 : "On the theoretical side, it is not entirely clear why our optimization problem in Equation 5 converges to actual training samples. even though there is guarantee that the solution is unique, especially when using no prior." solutions are ubiquitous rather than unique, and in the absence of prior knowledge, the attack fails.
2

---

Published as a conference paper at ICLR 2026
solution and analyze the extent to which individual points can be split (Theorem Finally, we demonstrate that even if the attacker possesses additional information about the training procedure, under the assumptions of structured data and well well-trained model, it remains possible to split the points in a way that preserves data confidentiality (Theorem 9
In Section we complement our theoretical results with experiments that support our findings. Specifically, we model the attacker's prior as knowledge of the data domain boundaries, incorporated into the reconstruction attack through the attacker's 's initialization. We demonstrate, on both synthetic data and CIFAR, that the attacker's prior weakens, the effectiveness of the attack decreases accordingly, with convergence to solutions far from the true training set, as predicted by our theory.
ADDITIONAL. RELATED WORK
Reconstruction-bast privacy attacks. An emerging line of work studies the possibility of extract- ing training data directly from trained models, raising serious privacy concerns. Such reconstruction attempts have been demonstrated across a range of settings, including large generative models (Carlini
jet al. 2019: 2021 Nasr et al. 2023), diffusion models and diffusion architectures (Somepalli et al. Carlini et al.] (023), and federated learning (Zhu et al. 2019; He et al.] 2019; Hitaj et al.
2022
2017
Geiping et al. 2020 Huang et al. 2021] Wen et al.] 2022) scenarios. Several studies, most
notably Haim et al. (2022) Buzaglo et al. (2023a), have proposed optimization -base strategies
that leverage the implicit bias of neural networks to recover examples from the original training
data. Their approach frames data recovery as the problem of minimizing a suitably defined objective, whose solution can align with portions of the true training set.
complementary line of work, exemplified by Loo et al. (2023), studies attacks fine-tuned models that reconstruct downstream private training data, in contrast to the setting studied by |Haim et al. (2022), which we also examine in this paper. This scenario is particularly important, given the widespread use of fine-tuning on public models (e.g., LLaMA, DeepSeek, Mistral). Nonetheless, the theoretical guarantees presented by Loo al. (2023) are established under simplified and somewhat unrealistic settings that do not fully capture practical conditions, as the underlying proof assumes a particular infinite-width network model and data that lie exactly on the unit hypersphere.
A different perspective investigates the reliability of reconstruction attacks. The study in Runkel ct al. (2024) empirically shows that reconstruction is highly sensitive to initialization, leading to the generation of plausible samples not present in the original training data. This ambiguity makes it difficult for an adversary to verify whether a recovered image is an actual training sample. While their empirical findings are similar to ours, work provides rigorous theoretical guarantees that underpin this phenomenon, whereas their study remains purely empirical.
Differential Privacy. The current "golden standard" for privacy is the differential privacy frame- work Dwork et al. (2006b); Dwork & Roth (2014). Intuitively, differential privacy quantifies how much model's 's output can change when a single data instance is modified. thereby providing worst-case guarantee over all neighboring datasets. The smaller this change, the stronger the privacy guarantee. In practice, mechanisms enforce differential privacy by injecting carefully calibrated noise into data, queries, or training procedures (Dwork et al. 2006a (Abadi et al. 201 The composition and accounting results quantify the cumulative privacy loss (Kairouz et al., 2015 Balle & Wang 2018). These guarantees often come with utility trade-off's, which are studied in both ERM and deep learning (Chaudhuri et al. 201T Abadi et al.] 2016a). Unlike differential privacy, our approach does not rely noise injection. Instead, seek to better understand the underlying causes of implicit- implicit-bias-dee bias-driven privacy vulnerabilities, so that they can be circumvented when possible.
2 PRELIMINARIES, BACKGROUND AND NOTATION
Notation and terminology. We consider binary classification with data (x, y) E Rd x {-1, 1) and training set {(x,, v)) Let ((0-.) d + R be neural network with parameters 0 IR Write [ == max .r.. homogeneous 2-layer ReLU network is 4(0,x) = Σ fj [w]] x + b,],, where 0 = (w,, Uy. b The unit sphere is gd-1 = fx E Rd ||x||2 1). Let d(x) X2) denote the distance between vectors (Fuclidean by default), and for sets A, B define d( (A, B) =
3

---

Published as a conference paper at ICLR 2026
mD d(a, b). For R", its activation pattern is the binary vector whose j-th entry indicates whether neuron j is active on x. For neuron (w,.bz and point x, set Dx = (w, +b;; the signed distance to the hyperplane (w).)) + bj 0.
wwyll
Preliminaries. The following well-established result, which has recently attracted significant attention, shows that homogeneous neural networks trained with logistic exponential loss exhibit an implicit bias: they converge in direction to the solution of a max max-margin problem. This implicit bias is the key phenomenon underlying the attacks we study in this paper.
Theorem 1 (paraphrased version of Lyu & Li (020), Ji & Telgarsky (2020). Let Φ x) be a normalized homogeneous ReLU neural network. Consider minimizing the logistic (2 + log(1 [1 }) or the exponential (2 + e loss using gradient flow (which is a continuous time analog of gradient descent) over a binary classification set {(x,; yi y;)] IRd {-1, 1). Assume that there time to where L(O(to)) <
e e"))
i Then, gradient flow converges in direction to first order stationary
point (KKT point Vapnik 1995)) of the following maximum-maree
problem:
min- 2
1012 8.1 Vi E [m] (0;xx, 2 1.
A KKT point of Equation]
is characterized by the following set of conditions:
0 Vo fy. (0:x;)
(stationarity)
(1)
9: (0:x) 2 L, [n]
(primal feasibility)
(2)
A; 2 0, Vi E [m]
(dual feasibility)
(3)
A; if y + (0:x:) + 1, e [n]
(complementary slackness)
Utilizing the above result, Haim et al. (022) devised the following reconstruction attack, demonstrat- ing that one can reconstruct substantial subset of the training data from deployed model.
The core of the method is a minimization problem. It simultaneously adjusts the candidate training data X' and their associated Lagrange multipliers À to minimize composite loss function. This loss function is designed to enforce several of the KKT conditions that characterize an optimal solution for the classifier. The objective function is given explicitly by
X arg min (1
0 (oVoly.(000
max --xc, 0} + + (x)riorr"X"'
(5)
Lestaiionary
The stationary loss in Equation EqquationSppnalliiii penalizee deviations from the KKT stationary condition (Equation D where the non-negativity loss enforces the non-negativiyy constraint on the multipliers X;. as required by Equation and Lptior represents some prior knowledge we might have about the dataset. Lastly, note that the margin's ' value, as defined by the primal feasibility condition in Equation 2 at a KKT point, is normalized (Lyu & Li 2020) and thus equals 1. However, in practice, the margin width (0) may take on different scale, which, for later use, we denote by (0) = > 0 We remark that this method aims to reconstruct training data instances that are on the margin (namely, instances whose prediction value is or -p), since otherwise, by Equation 4 they do not factor in the constructed objective.
Although incorporating prior knowledge as an additional loss term to LKKT is a common technique, we deliberatcly exclude it from our analysis. Our rationale is that such priors are often heuristic and highly context-specific, rather than fundamental to the reconstruction method in Equation 5 Including a prior would confound the evaluation of the method's performance, as success would become dependent the specificity of the external knowledge, making it difficult to disentangle the contribution of the mathematical constraints imposed by Equations
'A network (0: x) called homogeneous if there exists such that for every b 0 and x, holds that & (b- 0: x) = (0; x).
*We say that gradient flow converges in direction to Ô lim
O(C)
|0(0
'The margin 70) scales linearly with [0]|s, where is the order of homogeneity, namely 7(0)
-(0)
4

---

Published as a conference paper at ICLR 2026
3 IMPLICIT-BIAS-BASH RECONSTRUCTION ATTACKS
In this section, we theoretically study the reconstruction attack introduced by Haim et al. (2022). We begin by formally defining our framework and the required definitions, which differ slightly from those in the practical setting studied in their seminal work.
Framework. An attacker, motivated reconstruct the training set S, is given a full access to the neural network (binary classifier) architecture 4 ( well as complete knowledge of the weights 0 2 However, the attacker has no knowledge the training samples X (x), (x)..... x., or the quantity n. In addition, we first assume no knowledge of the margin scale 7(0) in the following subsection, and in Subsection 3.2 we relax this assumption. With this information, the attacker then optimizes the objective in Equation to reconstruct the training set, yielding X' (x,,...,xx, x²,).
We begin by defining the KKT loss, which is used to assess the feasibility of a reconstructed set. As mentioned in the previous section, this definition omits the prior term.
Definition (KKT-loss). The KKT-loss is defined as.
(XKT(X1,...... AI,,,,....AA.) 7n Latationary (X1, (X,,..., X1, ......,, A1) + PoI A,), (6)
where n. 122 > O denote hyperparameters controlling the optimization process.
Remark 1. Note that I, the number of data samples in Definition] | does not necessarily equal n. This is justified, as by assumption, the attacker does not know the number of training samples n. Additionally, we do not incorporate the constraints in Equation 2 and Equation f into the definition of LKKT, as these were also not used by Haim et al.) (2022). A detailed discussion about this can be found in Appendix L
Following the definition of our loss function, we specify the corresponding set of examples that satisfy the conditions of Theorem which the minimization of the objective in Equation 6 aims to find.
Definition 2 (KKT set). Let (00:)) be a binary classification network with weights O that has converged to a KKT point (satisfying Equations 74. set of inputs $ (x)...., x,) is called a KKT set there exist nonnegative multipliers A X 2 such that LKKT(S, . A1) 0.
Clearly, the original training set S is KKT set by assumption.
3.1 CONVERGENCE TO AN EXACT KKT POINT
We now show that, perhaps surprisingly, once the prior component is removed from Equation 5 the objective LкKT admits infinitely many global minima. Moreover, certain minima yield reconstructed samples that differ substantially from the original training set, as measured by the minimum Euclidean distance between neighboring instances. Consequently, the reconstruction attack cannot reliably distinguish the actual training set from these alternative KKT sets. Remarkably, we further show that the distance between such minima and the original training set can be unbounded.
To this end, we present a constructive method for generating new KKT sets from given one. This method relies on two key lemmas that underpin the construction, after which we show how they can be used to explicitly generate broad family of KKT sets.
The proofs of lemmas and theorems in this subsection are relegated to Appendix A
Lemma 2 (Merge). Let be a KKT set and let x1, X2 E S be two points with identical labels and activation patterns, and with coefficients Xı, X2 > 0. Then, there exists
(0, 1) such that the set S' = (S (x1,x23) (x1s) is also KKT set, where X1.5 = ox1 + (1- (1-a)
Lemma (Split). Let S be KKT set and let X1 E S be a point with coefficient X1 > Then, for all a, B > 0 and for all Rd such that z =
and Z2 x, BB have the same activation pattern and classification as X1, the set S' == (S {x1 )) U [z), 22) is a KKT set.
'Many of our results can still hold, be extended, with only partial access to the network's weights. For clarity and simplicity, however, we assume full access in this work.
5

---

Published as a conference paper at ICLR 2026
The above lemmas offer constructive approach for discovering new global minima of the objective in Eq While the former allows merging two points to create new KKT set, the latter enables splitting single point into two. A more visual illustration of this concept can be found in Figure
We stress that the attack in Haim et al. (2022) also applies to certain non-homogeneous networks. Nevertheless, the primary focus of the attack and analysis on homogeneous neural networks, as they are biased toward meeting these conditions during training.
Remark 2.
While these technical results provide means to explore the loss surface of the objective in Equation 6 they do not provide quantifiable assessment of the extent to which KKT sets can be manipulated by merging and splitting. While merging imposes few constraints, it provides more limited alterations to the dataset. In contrast, splitting allows flexible alterations but further requires that the new points inherit the activation pattern and classification of the original point from which they are derived, which may constrain their distance to it. Conveniently, under the assumption that the set of training examples does not span the entire data domain, we can show that this distance is unbounded.
Theorem 4. Let S {xI,.... .xn] C Idd be the training set, and let (00:x) 2 2-layer neural network that was trained on and reached a KKT point, span X, & Rd, then for all > O, there exists a KKT set Sr such that d(S, S.) > r. Moreover, all points in S, are on the margin. That is, x0(0xx,] = ffr all x, E S,-
Interestingly, the above theorem implies that the KKT sets that we construct provide twofold defense: not only are they global minima of the KKT loss, but all points in these sets also lie on the margin. Consequently, even if the margin value is leaked, the attacker cannot use it to distinguish the actual training set from an arbitrary KKT set.
I and the above theorem pertains only to points on the margin. Thus, as with all other results in this subsection, it continues to hold even if arbitrarily many non- margin points are added to S. However, for simplicity, assume that all points in S lie on the margin.
We stress that, due to Equations
The assumption that the training examples do not span the entire data domain can be justified by certain real-world datasets that concentrate on low-dimensional structures or even unions of subspaces within the domain, property commonly exploited in practice (Elhamifar & Vidal, 2013). For example, this is evident in the MÄIST dataset, where most images consist primarily of black pixels. Still, in many cases, the data manifold is not proper subspace of the domain, but instead can be closely approximated by such a subspace. To provide meaningful guarantees in such cases, we present the following theorem.
Dec sion boundery haundary
Liee of possible samples
boundery Decision bouncary 2 Lne possible semples
X1
(a) Example of Lemma 2
The green point on the dotted line can replace the two points at its edges.
3 The blue point can be split into two points along the dotted line.
(b) Example of Lemma
Figure 1: Illustration of Lemmas 2 and 3 which demonstrate how splitting and merging can allow us to constructively explore the solution space of the attacker's optimization problem, presented in Definition [ An instance can be split into two instances along a valid direction (Subfigure Ta) or, conversely, two instances can be combined into single representative instance (Subfigure lbp. Note that the instances obtained through splitting or merging preserve the same activation pattern as the original instance(s).

---

= (xi)) (x. + av 3r3 is KKT set. Published as a conference paper at ICLR 2026
Theorem 5. Let S (x1,.... xn) C R be KKT set, let ((0xx) be 2-layer, width-k neural
network that was trained on S and reached a KKT point, and suppose that Y > and v E R satisfy (v,x,) S for all i. Then, for all x, E S and any a, not exceeding
D2(x,)1 Iw,l 1
min JE
A 7
In words, the theorem implies that each point x; can be split into two points, each at least away from x; (larger values of a and 3 may also be admissible). Note that d S n,, since otherwise one could find vector orthogonal and reducing the setting to the assumptions of Theorem 4 We stress that smaller values of y allow points to be split farther- situation that arises, for example, when the data manifold approximately lies on a linear subspace of the domain. Moreover, we prove that y S σd. where aa is the smallest singular value of the data matrix S written as a column matrix = (X1,-.. : :x,))::,))))))))))))))). (see Appendix C for further discussion). Geometrically, this means that the minimal splitting distance is controlled by the "thinnest" direction of the data cloud: the smaller the smallest singular value, the more freedom there is to alter points along directions perpendicular to those where the data is nearly flat.
[0,(x|)||\w.I
minse[k] 2,1=1
all data points, giving 7
3.2 ANALYSIS BEYOND THE IDEALIZED SETTING: APPROXIMATE KKT POINTS
In the previous subsection, we assumed that the model converged to KKT point. Since, in practice, trained network may only approximate such a stationary point rather than precisely attain it, this subsection provides more natural yet analytically tractable foundation for studying the associated privacy risks. To that end, will shortly formulate the following relaxation of the KKT condition. Definition ((E, 8)-KKT paraphrased from Lyu & Li 2020 Ji & Telgarsky (020
O satisfies (e, KKT if the following holds:
a) 10 Σx= Amh VO x:)xx c) An 2.
b) Vi, (0,x) 2 d) Vi, A, x,) - p) S
Following the above definition, the following is a natural generalization of KKT set.
Definition ((e, 8K-KKT sets). Let (f;x)) x) be homogeneous neural network, and let S (xI,.... ..... }. We say that S is an (e, OKKT set if there exist nonnegative X1,..., An such that is (KK.)
We note that optimizing the objective in Condition accuracy = produces (e, KKT sets for some 5 >> O. For brevity, when 8 is irrelevant, simply refer to these as <-KKT sets. This convention aligns with the setting in Haim et al. 22022,, who similarly disregard the 5 term, since it is not used in their attack.
The following are extensions of the merging and splitting lemmas from the previous subsection.
X1,X2 E S be point with Lemma 6 (Approximate-Ke) KKT merge). Let S be an e-KKT set and let x1, X2 coefficients X1, X2 > O. Then, there exists a E (0, 1) such that the set S' = =( (x1, (2)) U fax1 - xx)xx } is also K-KTT set.
Lemma 7 (Approximate-KKe coefficient Àι > O. Consider the two points: 21 X1 + av and z2 X1 By where a, B > O and v E TRd such that Z1. Z2 have the same activation pattern and classification as x1. Then, the set S' (S {x1)) 1) U {Z1, 2:2 is also an K-KKT set.
split). Let S be an s-KKT set and let X1 E be a point with
In the following two subsections, we examine the budget for splitting data points in the almost KKT setting. Each subsection assumes full knowledge of & but considers different level of knowledge the attacker has about
The proofs of lemmas and theorems are relegated to Appendix B
3.2.1 THE ATTACKER POSSESSES NO INFORMATION ABOUT 5
As a starting point, we first assume in this subsection that the attacker has no information about In such case, Lemma 7 can still be used as long the new points are in the same activation pattern as
7

---

Published as a conference paper at ICLR 2026
the original point. The following theorem provides minimal guarantee on the budget required to split data point in this setting.
Theorem 8. Let S= (x1,..., xn C IR be an e-KKT set for some & > O, let ( x) be 2-layer, width-k neural network that was trained on S and reached an (s, 8)KKK point, and suppose that y > and V E Rd satisfy (v, x1) S for all i. Then, for all Xg E S and any a, 3 not exceeding
|D,(x)]ll,,]
min
(7)
JEIN] E + TUo Ei=1
such that 2 == Xi + au and 22 = X; -B have the same classification as xs, S' = (5 ( {zz, Z2] is an K-KKT set.
Note that when O, the above equation simplifies to minje[k] [Dx(x:]] will Furthermore, if + 0,
then both a and 3 become unbounded, consistent with the analogous case Theorem S
3.2.2 THE ATTACKER HAS AN UPPER BOUND ON 5
In the previous subsection, we assumed, as an initial test case, that the attacker cannot know This assumption, however, is brittle in practice, some information about 8 may leak, or the attacker may be able to deduce an upper bound on its value from the training dynamics Because our splitting technique degrade the value of 6, this upper bound introduces additional constraint: split the training set beyond the attacker's bound, which limits the allowable budget for altering training instances. The following theorem establishes lower bound on the permissible amount of change in this scenario.
cannot
Theorem 9. Let S= (xi,.... } Rd be an (KKT set for some E, 8 > and let Φ x) be 2-layer, width-kk neural network that was trained on S and reached an (c, 5) -KT point. Suppose that 7 > 0 and v E Rd satisfy (v.x,) S Y for all i, and consider the dataset S' = (S {xx)) fxi a, BBv) for and B not exceeding Equation 7 Then, if Ad < p, S' is an (e, + AK)-KKT set, where
Ad = Ad(a + 31)
ΣE1 jEJ
l0y1
s s
Σ A
When y 0, the expression simplifies to 5 E + +1) = loyl. Thus, although 8 deteriorates with increased splitting, the effectiveness of the reconstruction attack diminishes as y decreases and the network approaches a KKT point, indicating that for well-traince networks on structured data, such attacks become unreliable.
Σ
EXPERIMENTS
In this section, we present experiments that complement our theoretical analysis. Our theory shows that the objective in [o]aation[]admiit ubiquitous global minima under certain conditions, but this alone does not guarantee that the attack will avoid converging the true training set. To examine this, we empirically test whether training-set leakage can occur without prior knowledge. We model the attacker's 's prior as knowledge of the data domain boundaries and incorporate this into the attack through the initialization distribution. For instance, when the domain consists of natural images, the prior encodes that the data lie within the valid pixel range (0, 1]d so the attacker initializes only within this range. This type of prior was also used by Haim et al. (2022), who employed both correctly scaled initialization and an optimization penalty to ensure solutions stayed within the natural-image domain.
No leakage without prior. We generated 500 synthetic training samples (250 per class) uniformly on the unit sphere 7783 784 and labeled them according to the sign of the first coordinate, We then trained 2-layer, width-1,000 ReLU network on this data for 500K epochs, achieving a final
For instance, large value of 5 may indicate that the training process terminated prematurely, leading to poor performance.
8

---

Published as a conference paper at ICLR 2026
training loss of 10-7. To assess reconstructability, initialize the candidate reconstructions x; on spheres of varying radii centered at the origin (each corresponding to a different level of prior information available to the attacker). For each setting, record the average distance between the best reconstructions and the actual training set over multiple runs.
We evaluate two types of prior, The first one is the magnitude of the data, and the second one is its geometry. Spheres with different radii have different geometries, such as different curvatures, different typical inner products, and different typical distances between two randomly sampled points.
Although all runs achieve similar KKT objective values (ranging from 330 to 332),, they yield markedly different reconstruction qualities, revealing strong dependence initialization and convergence to distinct minima. In the absence of prior information, near-optimal solutions thus do not reliably recover the original samples. Figure 2a depicts the distribution of the average distance between the training set and the best five reconstruction attempts. It demonstrates that the reconstruction error increases as the assumed radius deviates from the true domain of the data. Consequently, successful reconstruction appears strongly depend on prior knowledge of the data domain.
Beyond the theoretical framework. We trained the same 3-layer architecture on CIFAR as was done by Haim al. (2022), after shifting all training samples by various magnitudes, corresponding to different levels of prior information available to the attacker To reconstruct the training set, we minimized the objective in Equation without incorporating any additional regularization.
As shown in Figure 2b the results quickly deteriorate as the attacker's prior weakens. Moreover, the reconstructions clearly resemble averages of multiple training instances, indicating that the minimum reached by the attack's optimization procedure is in fact an interpolation of several training examples, as predicted by our theory.
Prior knowledge is essential for successful reconstruction attacks; KKT conditions alone do not necessarily reveal information about the training data. Experiments show that attack effectiveness is directly linked to the strength of prior knowledge: stronger/more accurate priors lead to stronger attacks, making an adaptive adversary that can estimate the true prior highly effective. Privacy risks are largely mitigated if the training data is shifted by secret bias, even when the general data domain known. Estimating and defending against such adaptive priors an interesting direction for future work.
For more experiments on various datasets and architectures, we refer the reader to Appendix F
SUMMARY AND DISCUSSION
In this paper, we demonstrated both theoretically and empirically that the objective function underly- ing the reconstruction attack of Haim et al. (2022) admits ubiquitous global minima. Consequently, reconstruction is generally unreliable without prior knowledge, which suggests new avenues for mitigating such attacks, for example, by shifting the training set with a secret bias. Notably, results indicate that the implicit bias induced by gradient methods can actually prevent leakage rather than facilitate it, which may counterintuitive in light of previous work.
Although our proposed defenses are theoretically motivated, they do not provably preclude recon- struction, since an attacker might still infer information about the data domain directly from the model. We leave the intriguing question of whether this is indeed possible and how to design provably secure defenses for future work. Expanding our findings to more modern, non-homogeneous neural networks is an important and interesting question for future work.
'While conceptually equivalent, this is not precisely how obtained our network. See Appendix D for further details.

---

Published as a conference paper at ICLR 2026
Blas 0. Average distance of best reconstructions
Aveenge estance
2
10
redii
(a) A comparison of the reconstruc- tion results based on initializations with different radii, for network trained with data sampled from the unit sphere, and measured by the average Euclidean distance of the 5 best constructions. The radius increases as the prior knowledge available the at- tacker weakens, significantly degrading the quality of the reconstruction.
was
(b) Reconstructed CIFAR images (odd columns) and their training set nearest neighbors (even columns). We trained a non-regularized model on data shifted by 0.5 (top image) and 5 (bottom image) until reaching almost- KKT point. The experiment demonstrates that attack effectiveness rapidly diminishes with increasing data shift, in- dicating a weaker prior for the attacker. While the top reconstruction captures a small subset of the training data, the bottom reconstruction fails entirely.
Figure 2: The left figure shows the attack on the unit sphere. The images on the right show the attack on CIFAR with shifted training data.
REFERENCES
Martin Abadi, Andy Chu, lan Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In ACM CCS, pp. 308-318, 2016a. doi: (0.11449977779992999
Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS '16, pp. 308-318, New York, NY, USA, 2016b. Association for Computing Machinery. ISBN 978 7781450341394.
Borja Balle and Yu-Xiang Wang. Privacy amplification by subsampling: Tight analyses via couplings and divergences. NeurIPS, pp. 6280-6290, 2018.
Simone Bombari and Marco Mondelli. Privacy for free in the overparameterized regime. Proceedings of the National Academy of Sciences, 122(15)ee423077212,, 2025.
Gon Buzaglo, Niv Haim, Gilad Yehudai, Gal Vardi, and Michal Irani. Reconstructing training data from multiclass neural networks, 2023a.
Gon Buzaglo, Niv Haim, Gilad Yehudai, Gal Vardi, Yakir Oz, Yaniv Nikankin, and Michal Irani. Deconstructing data reconstruction: Multiclass, weight decay and general losses. Advances in Neural Information Processing Systems, 36:51515-51535 2023b.
Nicholas Carlini, Chang Liu, Úlfar Frlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX Secu- rity Symposium (USENIX Security 19), pp. 267-284, Santa Clara, CA, August 2019. USENIX Association.
Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, and Colin Raffel. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2633-222222 2650. USENIX Association, August 2021. ISBN 978- 1-939939913 24-3.
10

---

Published as a conference paper at ICLR 2026
Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models, 2023.
Kamalika Chaudhuri, Claire Monteleoni, and Anand D. Sarwate. Differentially private empirical risk minimization. Journal of Machine Learning Research, 12:1069-1109, 2011.
Cynthia Dwork and Aaron Roth. The Algorithmic Foundations of Differrntial Privacy, volume 9 of Foundations and Trends in Theoretical Computer Science. Now Publishers, 2014. doi: 10.1561/ 0400000042.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise sensitivity in private data analysis. Theory of Cryptography Conference (TCC), pp. 265-284. Springer, 2006a. doi: 1007/ 1681878_14.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Proceedings of the Third Conference Theory of Cryptography, 06, pp. 265-284, Berlin, Heidelberg, 2006b. Springer-Verlag. Verlag. ISBN 3540327312.
Ehsan Elhamifar and René Vidal. Sparse subspace clustering: Algorithm, theory, and applications. IEEE transactions on pattern analysis and machine intelligence, 35(11):2765-21)) 2013.
Hao Fang, Yixiang Qiu, Hongyao Yu, Wenbo Yu, Jiawei Kong, Baoli Chong, Bin Chen, Xuan Wang. Shu-Tao Xia, and Ke Xu. Privacy leakage on dnns: A survey of model inversion attacks and defenses. arXiv preprint arXiv:2402.04011 04013, 2024.
Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. Inverting gradients-how easy break privacy in federated learning? Advances in Neural Information Processing Systems, 33:16937-16947, 2020.
Niv Haim, Gal Vardi, Gilad Yehudai, Ohad Shamir, and Michal Irani, Reconstructing training data from trained neural networks. NeurlPS, 2022.
Zecheng He, Tianwei Zhang, and Ruby Lee. Model inversion attacks against collaborative inference. In Proceedings of the 35th Annual Computer Security Applications Conference, pp. 148-162, 2019.
Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz. Deep models under the gan: information leakage from collaborative deep learning. Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pp. 603-618, 2017.
Yangsibo Huang, Samyak Gupta, Zhao Song, Kai Li, and Sanjeev Arora. Evaluating gradient inversion attacks and defenses in federated learning, Advances in Neural Information Processing Systems, 34:7232-7241, 2021.
Ziwei Ji and Matus Telgarsky. Directional convergence and alignment in deep learning. In
H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 17176-17186. Curran Associates, Inc., 2020.
Peter Kairouz, Sewoong Oh, and Pramod Viswanath. The composition theorem for differential 2015.
privacy. In ICML, pp.
Noel Loo, Ramin Hasani, Mathias Lechner, Alexander Amini, and Daniela Rus. Understanding reconstruction attacks with the neural tangent kernel and dataset distillation, 2023.
Kaifeng Lyu and Jian Li. Gradient descent maximizes the margin of homogeneous neural networks. In International Conference on Learning Representations, 2020.
Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Chen Eric Wallace, Florian Tramèr, and Katherine Lec. Scalable extraction of training data from (production) language models. arXiv preprint axXi:2331..7033,, 2023.
11

---

Published as a conference paper at ICLR 2026
Christina Runkel, Kanchana Vaishnavi Gandikota, Jonas Geiping, Carola-Bibiane Schönlieb, and Michael Moeller, Training data reconstruction: Privacy due to uncertainty?. 2024. URL https:
7/arxiv. wtt.asa.///aeeeoo.tttt..eata.aat 2412. 08544
Guy Smorodinsky, Gal Vardi, and Itay Safran. Provable privacy attacks on trained shallow neural networks, 2025.
Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffu- sion art digital forgery? investigating data replication in diffusion models. arXiv preprini arXiv:2212.0358 2022.
Qi Tan, Qi Li, Yi Zhao, Zhuotao Liu, Xiaobing Guo, and Ke Xu. Defending against data reconstruc- tion attacks in federated learning: An information theory approach. In 33rd USENIX Security Symposium (USENIX Security 24), pp. 325-342, 2024.
Vladimir Vapnik. Support-vector networks. Machine learning, 20:273-297, 1995.
Yuxin Wen, Jonas Geiping, Liam Fowl, Micah Goldblum, and Tom Goldstein. Fishing for data in large-batch federated learning via gradient magnification. arXiv preprint arXiv:2202.0085 2022.
Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. Advances in Neural Information Processing Systems, 32, 2019.
A PROOFS FOR SUBSECTION 3.1
In this section, we present all the proofs for Subsection 3.1 We begin with few additional notations that will be used throughout the appendix.
Given a neuron with weights w and bias b, define the shorthand (x(x = [(w, x) by]_ Let at denote subgradient of w + by] for sample x,, write of for the subgradient of [wj x; + b,)
A.1 PROOFS OF LEMMA AND LEMMA
We begin with proving an auxiliary lemma about the convex nature of the gradient of the neural network, and then we prove Lemmas and
Lemma 10. Let (00xx) x) be neural network with piecewise-lineal activations. Suppose that the point ax1 + (1 - a x2) is at the same activation pattern as X1 and X2. Then, (0 x) is convex with respect to
+ -0
Proof. Let (0; x) be a neural network with ReLU activations (the argument is the same for other piecewise linear functions), then since the gradient ( x) is with respect to 0, it remains an affine function some matrix A and vector c. We therefore have
each activation pattern, and can thus be written as Vo x) Ax + c for
a a (0; + (1 V) = a(Axi +c) (1- + cc)
A(axi + (1-a)xx) +c
= Vot(O; + (1-0)x))
We now turn to prove the lemmas.
Proof of Lemma Denote X' = Å₁ + X2 and
A1 It is easy to see that E (0, 1) and that A X > 0, which proves that Equations and hold. Moreover, since x; and x2 have the same activation
12

---

Published as a conference paper at ICLR 2026
pattern, have that X1.5 also has the same activation pattern since it is convex combination of the two, which proves that Equation 2 holds. To show that Equation holds, we compute
0
Asyi Vo
Auy, Vo + XIyy (o0(0;x)) Ao
11
Ag + )
1,1
Σ + -2xx2)
11,2
\1 x1
tY + 22 x2 )=))
X2
E + tvs. 0: + +22)
t1 + +
#12
E
+ (0: + - ax)x))
#12
+ (
11
where by the assumption that y1 1/2 1/.5. and is Lemma TO
Proof of Lemma Denote Aa
ad A .+ It is easy to see that X Ag 0, which proves a+3
that Equations and 4 hold. Moreover, since by assumption 2 and Zz split such that they have the same activation pattern as X1, this assures that Equation Situationhollde hoolds. Now, observe that
1. As + Aa
A a+3
Aia A1. a+3
AB da 32 \1X1, 2. Ax(x1 av) Aa(x1 - Bv) (Ag da)x1 a4gov a+
Then, by the above, we have
A + Ax(A((x + +av) Na BB8) +
= (Aa + da) (Aa + Az/c
A1 +c)
(1Vo((0:)))
(8)
Next, let see that indeed Equation 1 holds by computing
A Vob(0xx)
\ + Vo vIda (o0(0:2))
where * by Equation
Thus, the set S' is a KKT set.
A.2 PROOFS THEOREM AND THEOREM
We begin with proving the following auxiliary lemmas. Lemma 11 provides applicable technical constraints that any KKT network must satisfy, and Lemma shows that we can split the training samples in single direction, and that all points in the new KKT set will remain on the margin. Lemma [3 assures that we can split along this direction as far as we want and remain the same activation pattern the original sample.
13

---

Published as a conference paper at ICLR 2026
Lemma 11. Suppose 2-layer, homogeneous ReLU neural network with parameters O that satisfies the KKT conditions in Equations 4
Then we have
/ Auhk [ x
w
U
Xy
by = _ Ayyoo
Proof. We compute the derivatives with respect to 0 as follows
a
a Φ x) = 2yza
((0xx) = [w] x b,4
D 2 x) = 2joj.
Duy
Ow,
0b
If w/ I x + bj * 0 then o'j is well defined, and if w b; = 0 then of E [0. 1]. In any case, it holds
that of 2 O. Combining the above partial derivatives with Equation
we obtain
1/
Auyi W x + b,l_,
Wy U2
Ay
by U/
1=1
for all j [k:]. as required.
Lemma 12. Let S = { be KKT set such that all X; E S are on the margin, let p be the margin's 's value, and let & be vector that orthogonal all x; S. Then, for all B E R and for all 1 E [n] we have that ( x; + 3x) p.
Proof. Compute
& Σ
| X, + 3x)
U (w,, x + B8) + bil+
Σ
Asyoo (x,, (x,, X1 + (x) +by
j=1
j=1
1=1
n
Uj
1/
, bb
Σ
V
U/
1=1
f=1
U/ [w x1) + byl+
[00:x:) p.
Where is due to Lemma II and ** is due to the fact that & is orthogonal to all x,.
be a KKT set and let & be vector that is orthogonal to all X; S. Then, cy(xi + 38) ej (xi) for all I [n]] and for all R.
Lemma 13. Let S =
Proof. Compute
cj(Xi + 38) = [(W,, 38) + bbbl
Uj
, 38) +
==1
Ayyoo (xx, x1) + +,
" [(,,, bJl+ cx(x)
V
where * and use Lemma and ** uses that fact that & is orthogonal to all x,.
Using these two lemmas, can now prove Theorem
Proof of Theorem 4 Let S = [x ) be KKT set. Since span(x),.... ..., } Ç Rd there exists a vector & E d that is orthogonal to all x; E S. Since x is directional vector, we can assume without loss of generality that ||*||| = 1. For each x we define two new points X1, Xi ap and X1, = X, 3 for some ay 3 > We need to show that the set = -U" (X,, Xi, } is KKT set. Using Lemma (which
since we know that x and (xi, X1,) have the same
14

---

Published as a conference paper at ICLR 2026
activation pattern from Lemma 13 and also the same classification by assumption), can replace each instance X E S with X X1, iteratively until we get S'. This proves that S' is a KKT set.
Moreover, by Lemma 12 we have that all points in S' are the margin. Lastly, need to prove
that for any distance т > 0 we can chose or, 3r such that d(S, > т. Let us compute the distance
of some X X2 + ai from all points X; as follows
| - x; 11
X1, 2(x1, xs) + ||x:||
X + a 2(x: + a;xx, x;) + ||x;1
X + a - x1) + ||x.11
||x|| + 2ox (x, x) 1/ 2(xx x;) + ||x,||
|x|| + - x²) +|xx;11
|x/ + of x,),,,,) + ||x;
Since x; and x; are constants, then | || 2(x;, and | are also constants. The only parameter can change is au, so we can take aq large enough make sure that d(xi,, x,) > for all E S. The same analysis also shows that this approach can be applied iteratively to all remaining instances. Note that can take or and 31 be large have the same activation pattern as x (for all x E S) by virtue of Lemma T3 We can thus bound the distance between S and S') by
want since the new points X1, and X1, will still
d(S,8')=
min xS
||x
x'I
We now turn to proving Theorem 5
Proof of Theorem 5 Let X S and let x + az and 3r be the new points resulted from splitting x Both points have to remain in the same activation pattern as X;, meaning that for each neuron (x),, we need to make sure that sign(cy(x:) (x:)) sign(c, (x: av)) sign(cy(X; Br)). Namely, that we do not change the ReLU activation of any neuron w] x bj. + au changes an activation when (W,, X + av) bj 0, implying that
(w,, XI av) + bj 0 (Wy, xt) + bj + a(w;, v) = 0
(w), + bj
(W,,
Now we upper bound the magnitude of to see how far away au can be from X Let us denote the signed distance between a point x and the hyperplane (W,, x) + bj = 0 by
Dx(x) (w, We can rewrite [[[: Now let bound [W,,, v)) as follows [w,,0]
ww
(Wy,
X
Asyeo' (X:;;
A Aluill ||(x; vi1
A
i=1
The above implies that
Dx ||,
[x,(x]] ||w]l |1
1
a a>
Σ X;
7
15

---

Published as a conference paper at ICLR 2026
This is sufficient condition all neurons e, to guarantee that we do not deviate from current
activation pattern. This means that any a not exceeding
D
w
min JE
X
a valid choice, where the exact same analysis gives the same bound for 3.
B PROOFS FOR SUBSECTION 3.2
B. PROOFS OF LEMMA 6 AND LEMMA 7
Proof of Lemma 6 Define X == Å + X2 and a == A X1.5 == ox + (1-a)xs It is easy to
see that E (0, 1) and X > 0, which proves that Condition c) holds. Moreover, Condition d) is
also easily satisfied for a sufficiently large 6, and Condition b)
holds since the merged point X1.5 is
convex combination of X1, X2 and thus has the same activation pattern and classification as them. To
show Condition a)
we compute
Σ 1,2
Auy Vo + AIn Voo(0;x)) + A232
A Vo
12
to. +
112
Σ
+u. VoD +11 A2) (
A1 x1 +
t1 + xx
d2
))
11 + 22
=
Σ
+00
X(xxx + o)x2))
112
Σ
Vo. + 4
112
where * from the assumption that yı1 = 1/2 = 9155. and ** follows from Lemma 10 implying that
10
Σ Am Vo(((xx) V1.sA (F 1 oo
1,1
and therefore S' is also an k-KKT set.
BAr It is easy to see that Xa A 0, which proves Proof of Lemma Denote A. ad Ap a+ at
that Condition c) holds. Moreover, Condition ds also easily satisfidd for sufficiently large 6, and Condition b) holds by the assumption that the predictions of & on Zı and 22 change by less than the margin value p. To show Condition(a) observe that since (00; x) is piecewise linear in x, (f;(0)) is affine in x inside each activation pattern, and can be written V (0 matrix A and vector c. This implies that
Ax + c for some
1. AB + Aa
A.0
Aia
\1,
at3
a+3
2. Ax(x1 + av) + Xa - Bv) = =(19 - + d da Bv = \1X1,
a4300
at
which is turn shows that,
A (o;(022)) + Ao (oT(0) = AA + +av) +c)) +A)((AAA( BB) +
-As + Aa)AxI (Ao + Ag)c
=A1(Ax; +c)
(1
16

---

Published as a conference paper at ICLR 2026
The above implies
100
- -
Auy
= 100 [o
Σ Aui \o se,
1#1
hence, S' also an e-KKT set.
B.2 SPLITTING DISTANCE LOWER BOUNDS
In this subsection the appendix, we establish bound on the distance between the original point and the new points obtained through the splitting procedure when using Lemma i the almost-KKT setting. The following lemma will be useful to bound the deviations in the predictions the model that result from splitting a data instance in direction V.
Lemma 14. Given an (e, KKT point O, suppose that v is a directional vector satisfying ||||| = 1 and [x,, 2 S Y for all for some 7 > Then, we have for any neuron that
S 3
A
Proof. Compute
[w,, 1²
[ v)
V/J
Vj
X X3:
Aavio' و. (x, 2)
W 1/j
11
W 13
Xiyio! X
||0|| + 11g
Aeoio''al v3
W Fj
Xiyio Xi
|| + |0g Σ
Xiole 1(xix vi
S Setglag
Σ A
1=1
Equipped with the above lemma, we now turn to prove the theorems.
Proof of Theorem 8 The theorem follows from Lemma as long x taw has the same activation
pattern X,. To this end, need bound (w,,x))+b which is the distance between x; and the
(w...)
hyperplane induced by cx(x), for each neuron ey (x).
Denoting the signed distance between point and the hyperplane (w,, by 0 by Dy(x)
(w,x)+b; we can rewrite
(x(x))l
(xi))
Jaul >
2 + H1; Ei X,
(wj, v)
where the last inequality follows from Lemma 14
Namely,
take lay small
D (xi wwl
min
JEIx
$ + 7/, Σ
The same reasoning yields the same bound for |.t].
17

---

Published as a conference paper at ICLR 2026
Proof of Theorem 9
First, let us bound [( + +av)-
-b(0; x:)) as follows
[0;; x:) - (0;; x + mnDi
=
Σ jEJ
Bj
[w,, xi) + +bil+
Σ
Us [(w,, + tav) + bbg+
JEJ
Σ vj
(((w,, - bb14
+ + b11l+)
SEJ
Σ
|ug1
[ww,, xi) + +by++ - [[w; X: + anw) + bi]+
JEJ
Σ lu au = ai Σ
|ujl(w,,
JEJ
jEJ
2 01
Σ loyl s s A
JEU
where is by the fact that [.). is 1-Lipschitz, and ** is by Lemma 14 If the above quantity does not deviate beyond the value of the margin p. then we can guarantee that Condition bJ still holds and we can use Lemma 7 It now only remains to bound by how much ó deteriorates as follows
A1 (m (00:x + +av) - p) =
N
(yr
(00:x + aav) y1 (00:xi) + y1 (00:xi) p)
ag B1
04 B
A1,31
((,x)) - -p) +
( - (((0x))))
a4 B1
at B1
A1
(
p) +
(y ()( + +av) (0:x))))
0g
B
ai + B1
9
8+
|M||
|nl1((((: x: + aav) - (0)x))))]
aj
+ B1
+ B1
91
8 +
( aav) (()x)))))
aj 31
0i
B1
< 8 lpay
Σ
|uşl
E +
Σ X
JEJ
where the last inequality is due to Equation 9 and the fact that a by summing the above bound with an analogous bound obtained for 3;.
8, < 1. The theorem then follows
C FINDING AN ALMOST ORTHOGONAL SPLITTING DIRECTION USING SVD DECOMPOSITION
Theorems and 9 require finding a direction v such that (p,x)) for all data instances x, to be effective. Conveniently, such upper bound can be derived in terms of the smallest singular value of the data matrix. Given a training set S, we write it as a column matrix S (xi,..., X,). Then we have the following upper bound on the dot products.
Theorem 15. Let S UEVT be the SVD decomposition of the training data matrix S, where U E Rdddd V E Raxx, and Σ E Rdxn is a diagonal matrix with entries .... 2 aa 2 0. Then there exists a unit vector v such that (xx, v31 S aa for all E [n].
Proof. Let us chose We need to show that nd
Ua. ie the d-th column in U. Let us denote X, to be the i-th column. (X, & ad, or equivalently [X] vl & oa. We can write X, as X, = - oj V. T U, where VT,, is the j-th entry of the vector V.. Compute
UEV.
(xi, w>1= =
Σ .. V U, U2
Σ
V. U, U2)
- V id ad.
18

---

Published as a conference paper at ICLR 2026
where is due to the fact that U is orthogonal and ** is due to the fact that V is also orthogonal. and Od 2 0.
We remark that the above immediately implies that S aa for in the statements of Theorems aand Note that the final inequality may be rather loose, since V is a unit vector; equality holds if and only if the entry Vi, equals 1. If, however, the vector Vi, has more equally distributed entries, tighter upper bound with magnitude of roughly Ga/n will hold, further amplifying the efficacy of our theorems.
D SHIFTING THE TRAINING SET POST-TRAINING
In this appendix, we provide additional details on how we obtained the trained network for our CIFAR experiment in Section Recall that our goal to shift the training data by constant bias and then retrain the architecture used by Haim et al. (2022) on the shifted data, to obscure the attacker's prior knowledge. Since shifting the training by a constant bias shifts the gradients of the resulting objective function by the same bias, shifting both the data and the initialization point
leads to convergence the same neural network obtained by shifting the network trained on the original, unshifted data.
In light of this, rather than shifting the data and retraining the network from scratch, we utilized the network trained by Haim et al. (2022) and adjusted its biases in the first hidden layer. As an example, consider a hidden neuron with weights w and bias Given an input x, the pre-activation output of this neuron is (w, x) + b. Suppose we wish to shift each coordinate of the input x by vector u. In this case, the pre-activation output of the shifted input
(w, x + uj + = (w, (w, u. -
Since (w, u) is constant, by modifying the bias term via the transformation b + (w, u), recover the behavior of the original network on the shifted input. Applying this transformation to all biases in the first hidden layer for various values of obtained the networks used in Figure 2b
E ON THE SIZE OF THE TRAINING SET
In this section, we examine how the attacker's knowledge of the number of training points influences our analysis. The size of the training set can act as a prior and can constrain the applicability of Lemma 2 and Lemma possess. Yet, even if the attack does have this number it doesn't 't necessarily mean they can reconstruct training data. First, the lemmas can be used
Moreover, we view this information
strong prior for an attacker to
manner that preserves the total number of samples. For example, if two points are split and then points from different splits are merged, we obtain a different set of the same size. Second, even if our analysis limited, this does not mean there are infinitely many global minima achieved by sets of the same size. We don't 't know how strong this prior is.
F MORE EXPERIMENTS ON SHIFTING THE TRAINING DATA
In this section we conduct more experiments, covering more training sets and more architectures. We experienced consistent results with the experiments in which strengthen claim that prior plays an important role in the attack.
F.1 SPHERE WITH SMALL RADII
We expended the experiment presented Fa include small radii. The network was trained with training points from the unit sphere. Then attacked it with initializations from the spheres of radii 0. 0.5 and 1. As shown in Sggureejiitiiaizzatinn because in high dimension the points the unit sphere very far from each other, so sampling points from a sphere with a smaller radius actually gives us initialization points that are closer to the training points.
with the smaller radii give better results
19

---

Published as a conference paper at ICLR 2026
Bias O, Average distance best reconstructions
Average distance
0.1
0.5
radii
Figure 3: A comparison of reconstruction results based on initializations with small radii, for network that was trained on data sampled from the unit sphere in TR250 and measured by the average Euclidean distance of the 5 best reconstructions. Initializations with the smaller radii give better results because in high dimension the points on the unit sphere are very far from each other, so sampling points from sphere with smaller radius actually gives us initialization points that are closer to the training points.
Bias 0, Average dista nce of 5 best reconstructions
Average distance
20
radii
Figure 4: A comparison of reconstruction results obtained from initializations with varying radii is presented for network trained on data drawn from an annular region with radii the range 1-4. Performance is evaluated using the average Euclidean distance of the 5 best reconstructions. The x-axis indicates the minimum radius of the attack's 's initialization (with the maximum radius fixed at min 3). As the initialization radius deviates further from the training distribution, reconstruction quality consistently degrades.
F.2 ANNULUS EXPERIMENT
We constructed a dataset whose points lie in an annulus: all examples have norms between radii and 4 (chosen arbitrarily) and are drawn from this ring (support). We then ran the reconstruction attack multiple times with different initializations (priors): Initialization with radii between I and 4 (matching the support of the true distribution), Initialization with radii between 2 and 5 (partially overlapping support). Initialization with radii between 6 and 9 (no overlap with the true support). We measured the average distance of the 5 best reconstructions from the original training set. The results of this experiment show that the greater the overlap between the support and the support of the training distribution, the better the attack performed. The results shown in Figure 4
20

---

Published as a conference paper at ICLR 2026
F.3 VARIOUS DATASETS
In this subsection, we tested the training shift procedure on MNIST (Figure 5 and ImageNet (Figure ④
In the MNIST experiment, we trained two fully connected neural networks: one with all images' pixels shifted by 0.1, and one with all images' pixels shifted by 5. The figure shows that while we could reconstruct some images when shifting by 0.1, we could not construct even a single image when shifting by 5.
When training ImageNet, we shifted the pixels by 0.01 and by 0.1. Once again, we see that the quality of the reconstruction depends on the extent which the images are shifted.
We observe consistent results from the experiments in Section 4 the greater the data shift, the less likely the attack is to succeed. This strengthens our claim that shifting the data can mask the prior.
X
8
8
8
3
3
8
8
X
8
6
4
6
8
8
3
8
4
4
4
E
8
5
0
6
0
0
O
4
4
O
O
0
0
A
8
8
O
8
2
O
8
2
0
(a) Reconstructed images (odd columns) and their nearest neighbors from the training (even columns). All pixels were shifted by 0.1.
8
8
8
3
8
8
3
8
8
3
8
8
8
8
8
8
8
8
8
8
3
8
3
3
8
3
3
3
3
3
8
8
8
8
8
8
8
8
8
8
8
8
0
0
3
O
(b) Reconstructed images (odd columns) and their nearest neighbors from the training set (even columns). All pixels were shifted by 5.
Figure 5: Reconstructed MNIST images (odd columns) and their nearest neighbors from the training set (even columns). In the top image, all training data pixels were shifted by 0.1, whereas in the bottom image, they were shifted by We trained a model on the shifted data until it reached an almost-KKT point, without any regularization. The experiment demonstrates that as the data is shifted further, corresponding to a weaker prior available to the attacker, the effectiveness of the attack diminishes rapidly. While the top reconstruction still captures vague characteristics of a small subset of the training set, the bottom reconstruction fails entirely.
F,4 MULTICLASS CLASSIFIER
In this subsection, we check if the shift mechanism holds for multiclass scenario. We trained network on CIFAR1O, where its first layer a convolution layer. We trained once where we moved all the pixels by 0.01 and once where we moved all the pixels by 0.1 ((Figure B) As the figure shows, the weaker the prior the attacker has, the less likely it could reconstruct the training samples.
F.5 EARLY STOP
In this subsection, we repeated the sphere experiment in Figure 2a but this time stopped the attack once reached a KKT loss of 333. Since different attacks yield different KKT losses, the results may be affected directly by this. We wanted to neutralize this effect. As can be seen in Figure 7 the results are similar to those in Figure suggesting that the difference KKT loss between attacks did not affect the quality of reconstruction.
21

---

Published as a conference paper at ICLR 2026
Reconstructed images (odd columns) their nearest neighbors from the training set (even columns). None of the pixels were shifted.
(b) Reconstructed images (odd columns) and their nearest neighbors from the training (even columns). pixels were shifted by
(c) Reconstructed images (odd columns) and their nearest neighbors from the training set (even columns). All pixels were shifted by 5.
Figure Reconstructed ImageNet images (odd columns) and their nearest neighbors from the training set (even columns). In the middle image, all training data pixels were shifted by 0.5, and in the bottom image by 5. We trained model on the shifted data until it reached an almost-KAt point, without any regularization. The experiment demonstrates that the data is shifted further, corresponding to a weaker prior available to the attacker, the effectiveness of the attack diminishes rapidly. While the middle reconstruction still captures vague characteristics of small subset of the training set, the bottom reconstruction fails entirely. The top image is a baseline as there is no baseline in Haim al. (2022) for ImageNet.
G DISCLOSURE OF LLM USAGE
Parts of this manuscript were polished for clarity and style using OpenAl's ChatGPT (GPT-5). The model used solely for language refinement and proofreading; all research ideas, technical content, derivations, and conclusions are the authors' own.
22

---

Published as a conference paper at ICLR 2026
Average distance of 5 best reconstructions KKT loss 323
80
Faarage Dittnce
20-
10
reds
Figure 7: A comparison of reconstruction outcomes obtained using different initialization radii, where each attack is early-stopped once the loss reaches 333. The network was trained on data sampled from the unit sphere, and performance was measured as the average Euclidean distance among the five best reconstructions. The radius increases as the prior knowledge available to the attacker weakens, significantly degrading the quality of the reconstruction.
Reconstructed images (odd columns) and their nearest neighbors from training set (even columns). All pixels were shifted by 0.01.
(b) Reconstructed images (odd columns) and their nearest neighbors from the training (even columns). All pixels were shifted by O.I.
Figure 8: Reconstructed CIFAR10 images (odd columns) and their nearest neighbors from the training set (even columns) in the multiclass scenario. the top image, training data pixels were shifted by 0.01, and in the bottom image by 0.1. The experiment demonstrates that even in the multiclass scenario, as the data is shifted further, corresponding to weaker prior available to the attacker, the effectiveness of the attack diminishes rapidly. This suggests that our analysis for the binary case.
23


Reviews:
[{"review_id": "A7R3679ULT", "reviewer": "Reviewer_73hu", "summary": "The paper analyzes why implicit-bias–based reconstruction attacks on trained neural networks can fail, arguing that without explicit priors about the data domain, the KKT-driven objective used by prior attacks admits many alternative optima far from the true training set. The paper formalizes constructive merge/split operations that generate indistinguishable KKT sets, extends the analysis to approximate KKT points, and presents experiments on synthetic data and CIFAR, indicating that weaker priors degrade reconstructions while better-optimized networks are less susceptible, thereby reframing when leakage is feasible.", "questions": "- Beyond explicitly removing domain priors in the objective, which elements of the training pipeline might still act as priors (data normalization, augmentations, early stopping, weight decay), and how did you control or record them so readers can interpret the reconstruction outcomes?\n- Did you try residual connections to see if the phenomena qualitatively persist?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- This paper offers a clear formalization of the attack objective and isolates the role of priors by analyzing the KKT-only loss.\n- This paper offers practical guidance to related fields by indicating when reconstruction is unlikely without priors, which helps motivate practical countermeasures.\n- The writing is clear.", "weaknesses": "- The theory is developed for binary classification with homogeneous ReLU models, and it's not clear which proof steps would still hold in multiclass or with other losses. A brief *\"what carries over vs. what doesn't\"* discussion, or a tiny multiclass sanity check, would help readers judge how far the results travel.\n\n- Even if the Discussion later clarifies that results pertain to KKT-based reconstruction without priors, the abstract/Section 1 do not foreground this scope where readers first encounter the contributions; briefly surfacing the scope at the outset would align expectations and reduce the risk that practice-facing takeaways are read as general privacy claims, especially given prior work that studies broader regimes (multiclass, other losses) [1, 2].\n\n- The proposed shift/bias trick is intuitive but lacks robustness guarantees and could be learned by an adaptive adversary; scoping the claim (e.g., \"effective for KKT-only objectives under unknown shifts\") would avoid overgeneralization.\n\n- The discussion suggests implicit bias may hinder leakage along the KKT route, but memorization and extraction remain documented in overparameterized and generative settings [3, 4]. The authors should make it clearer: which leakage channels your analysis does and does not cover, so readers don’t overgeneralize the safety implications.\n\n## References\n\n- [1] N. Haim et al. Reconstructing Training Data from Trained Neural Networks. NeurIPS 2022.\n- [2] G. Buzaglo et al. Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Losses. NeurIPS 2023.\n- [3] V. Feldman and C. Zhang. What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation. NeurIPS 2020.\n- [4] N. Carlini et al. Extracting Training Data from Large Language Models. USENIX Security 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FqMtL6j63a", "reviewer": "Reviewer_e1G7", "summary": "This work considers parameter-only reconstruction attacks against shallow homogeneous ReLU neural networks performing binary classification, specifically analyzing the importance of the prior in the reconstruction attack developed in (Haim et al. 2022). The main contributions are the following:\n\nIn the event where a model converges to a KKT point as described by (Lyu and Li (2020), Ji and Telgarsky (2020)), the paper describes a family of candidate training sets indistinguishable to an adversary using the (Haim et al. 2022) attack without prior information through techniques of merging and splitting training data examples. The paper describes how, under an assumption that the training data does not span the whole input space, the adversary can find training sets arbitrarily far from the original training set. The paper also analyzes how far candidate training sets can be from the original training set under a relaxed assumption of closeness of the training data to some linear subspace.\n\nIn the event where a model converges to a $(\\varepsilon, \\delta)$-KKT point (an approximate KKT point), the paper proves the effectiveness of similar merging and splitting techniques, with splitting at the cost of a worse $\\delta$ value. The paper explores how large the family of split training sets can be depending on the adversary's knowledge of $\\delta$, suggesting that models that have converged more completely may be more difficult to attack.\n\nThe paper supports its theoretical arguments with empirical results on 1) synthetic data generated along a hypersphere and 2) CIFAR, showing how a lack of prior on training data magnitude can lead to poor reconstructions, suggesting a possible defense method against such attacks (manually shifting all samples by some magnitude).", "questions": "Do we have any guarantees on the strength of the (Haim et al. 2022) attack when we know a sufficient amount of prior information about the data? If not, does your paper's result move us closer to gaining such guarantees?\n\nReferences in my review above: \n(Haim et al. 2022) - cited in the authors' paper\n(Lyu and Li 2020) - cited in the authors' paper\n(Ji and Telgarsky 2020) - cited in the authors' paper\nChaudhuri, Kamalika, et al. \"Guarantees of confidentiality via Hammersley-Chapman-Robbins bounds.\" arXiv preprint arXiv:2404.02866 (2024).\nStruppek, Lukas, et al. \"Plug & play attacks: Towards robust and flexible model inversion attacks.\" arXiv preprint arXiv:2201.12179 (2022).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-organized and easy to follow.\n\nThe paper adds clear theoretical analysis to the rapidly expanding discussion on the importance of training data priors in reconstruction attacks (see e.g. [Chaudhuri et al. 2025]).\n\nThe paper expands the collective understanding of the conditions necessary for the (Haim et al. 2022) attack to be useful.\n\nThe empirical results clearly support the theoretical analysis, particularly in the 'interpolated' training examples shown in Figure 2b.", "weaknesses": "In the empirical results, the discussion of prior information available to the adversary is limited to data magnitude; while the plausibility of other forms of prior information, such as knowledge of the training data geometry or knowledge of the KKT $\\delta$, are discussed in the theoretical analysis, these are not mentioned in the empirical analysis.\n\nThe discussion could benefit from an expanded set of suggestions for future work in this field, including the potential to expand these results to the larger, higher-capacity models that are often used in empirical analyses of data reconstruction (see e.g. Struppek et al. 2022).\n\nThe empirical results and/or discussion could benefit from an analysis of potential adaptive attacks, in which a defender is aware that the magnitudes of the training data may have been altered.\n\nSpecific areas for clarification / improvement:\n\n- In Equation (6), why are the arguments of $L_{stationary}$ only $(x_1, ..., x_l)$, as opposed to $(x_1, ..., x_l, \\lambda_1, ..., \\lambda_l)?$\n\n- In the synthetic data experiment, why are no radii less than 1 tested? Would such a test be plausible, as a version of the adversary enforcing an overly strong prior on the data?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YSUQtWyokY", "reviewer": "Reviewer_3CEM", "summary": "The paper provides theoretical analysis of the KKT-based data reconstruction objective in the seminal work Haim et. al., and showed that perhaps surprisingly, if one throw away the regularization term based on prior information in the reconstruction objective, then one can construct new data records that are global optima for the reconstruction objective, yet are infinitely far away from the training dataset in Euclidean distance. Their construction is simple yet intuitive, linear combinations of records on the margin that have the same label and same activation pattern would still lie in the margin, and moving training data records along the margin would not break the KKT condition. They then prove that when the training data does not span the full space, one can construct data records with the same label and same activation pattern of a training data that lie arbitrarily far in Euclidean distance. The assumption could be further relaxed to training data that only approximately falls into a low-dimensional space, under which the maximum feasible perturbation scale (without breaking KKT) is inverse proportional to the smallest singular value of the data matrix S. They then extend their analysis to the approximate training case, where the trained model may be $\\delta$ away from the KKT condition, and prove that interestingly, if the adversary does not know $\\delta$, similar splitting constructions still hold; but if the adversary knows $\\delta$, then the feasible perturbation scale of the splitting construction significantly decreases. Experiments for ReLU networks on synthetically constructed classification dataset, as well as shifted CIFAR10 dataset, confirm that the reconstructed image's similarity to true training data degrades with increasing shift (corresponding to weaker prior knowledge).", "questions": "Besides weakness, I have a followup question on prior knowledge. The construction relies on splitting, or merging training data records, while in practice the adversary may be stronger prior knowledge on the number of training data records. In such case, it appears that possibly, a large fraction of the reconstructed data could still be the true training data. This could be another layer of prior knowledge worth discussing, besides the initialization.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-written, and performs thorough theoretical analysis of the reconstruction objective.\n\nThe construction of \"non-training data\" that satisfy KKT condition is intuitive and interesting. The further extended analysis for models that only satisfy approximate KKT condition is non-trivial and aligns well with experiments.", "weaknesses": "The only limitation I see is that the analysis and experiments seem restricted to homogeneous networks, while most recent NN models are non-homogeneous. I understand this limitation is inherent for KKT-based reconstruction objectives, but it would be nice if the authors could discuss or use experiments to understand whether the insights generalize to typical other NN models, such as CNN.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tyOSzztTOm", "reviewer": "Reviewer_2UcW", "summary": "- This paper studies the attack presented in [1], specifically focused on the importance of the prior knowledge loss term in the reconstruction loss\n-This loss encodes information about the pixel min/max values, and constraints the reconstructed images to be within this valid set\n- Theoretically, they show that through two operations: splitting and merging, it is possible to construct sets which satisfy the reconstruction loss exactly (excluding the prior term), but are not the original dataset\n- They extend this analysis to cases where the reconstruction loss is only fit within a error term\n- Empirically they show that when the image initialization is mismatched with the statistics of the actual images, reconstruction quality suffers dramatically\n\n\n\n\n\n[1] Reconstructing training data from trained neural networks", "questions": "- The authors report that all runs achieve a similar KKT loss of 330-332. If I remember correctly, the KKT loss of this attack does not actually decrease significantly over the course of training, so this gap of 330/332 may actual be significant. I would suggest setting up this experiment differently. Instead, would it be possible to terminate all runs when they achieve the same KKT loss exactly? I.e. do early stopping on all runs when they hit loss = 333.0 exactly, for example, then compare quality. This would prevent the issue where the remaining 0-2 loss points carry the majority of image quality.\n\n- Empirically, what KKT loss does the original dataset achieve? I.e. if we initialize the reconstruction images with the original dataset, and compute the loss, what is the result? Is it below 330-332?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Presentation is solid, idea is well motivated\n- Provides some theoretical understanding into these reconstruction losses, which generally speaking, lags far behind the empirical studies of them", "weaknesses": "- Lemma 2 and 3 require that the split/merged datapoints match the activation patterns of the original datapoints. It is unclear how much of a restriction this is. Can the authors empirically demonstrate that such operations are possible (on slightly less toy data than the 2d example presented). While it seems interesting, I am unconvinced that it is possible, for example, to split a datapoint with any reasonably large \\alpha, \\beta splitting coefficients, so that the split datapoints are actually \"far\" from the original. While the authors claim that reconstructed can be \"arbitrarily far\" from the original dataset, it seems this activation pattern constraint could actually restrict is substantially.\n\n- Empirical evidence is somewhat unconvincing. From experience, the attack presented in [1] generally is very brittle and hard to get working. Indeed, this paper aims to study why this might be the case. The authors present empirical evidence on CIFAR-100, where the attack already doesn't work very well. It would be additionally be useful if the authors present results on MNIST (or a simpler dataset), where the attack works better, so that there can be a greater difference between the version of the attack without prior knowledge/with prior knowledge.\n\n\n[1] Reconstructing training data from trained neural networks", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0O9MZdWCBe", "reviewer": "Reviewer_1FR8", "summary": "This paper revisits reconstruction attacks based on implicit bias (e.g., KKT-condition–based attacks) and provides a rigorous theoretical analysis of their limitations. The authors show that without prior knowledge about the data distribution, such reconstruction objectives admit infinitely many global optima that are arbitrarily far from the true training samples. The paper also introduces the notion of a “secret bias,” illustrating that hidden transformations or priors can effectively protect data from being reconstructed.", "questions": "1. Can the theoretical framework extend beyond classification to regression or generative models (e.g., diffusion models)?  \n2. Could the “secret bias” defense be generalized or implemented in practical training pipelines without harming model performance?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Solid theoretical foundations.**  \n  The mathematical analysis is thorough and clearly presented. The authors formalize the implicit assumptions in prior KKT-based attacks and prove that, without priors, the reconstruction loss function has infinitely many degenerate global minima. This provides a precise theoretical characterization of why such attacks cannot succeed “from scratch.”\n\n- **Novel perspective on priors and implicit bias.**  \n  Instead of proposing yet another reconstruction algorithm, the paper takes a principled view, showing that the presence (or absence) of prior knowledge fundamentally determines whether data can be reconstructed. This shifts the discussion of privacy leakage from algorithmic ingenuity to epistemic limitations.", "weaknesses": "1. **Limited empirical validation.**  \n   The experimental results are somewhat narrow, being limited to a shallow 3-layer network on the CIFAR dataset. To strengthen the empirical evidence, it would be helpful to include additional datasets (e.g., MNIST, CelebA, ImageNet) and deeper or modern architectures (e.g., ResNet, ViT) to confirm that the theoretical claims hold in more realistic settings.\n\n2. **Lack of quantitative evaluation of prior strength.**  \n   The paper emphasizes that both the “secret bias” and the attacker’s prior knowledge are key to reconstruction success, but it does not measure how the degree or accuracy of prior information affects attack outcomes.  \n   A systematic experiment varying the strength of prior knowledge (e.g., partial mean shift known, partial pixel-range constraint, etc.) and plotting reconstruction quality or KKT loss would make the argument more convincing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)