ReTrace: Reinforcement Learning-Guided Reconstruction Attacks on Machine Unlearning
Authors: Mengyao Ma
Shuofeng Liu
Jason Xue
Surya Nepal
Guangdong Bai
Venue: ICLR 2026 Poster (2026)
Keywords: Machine Unlearning
Reinforcement Learning
Reconstruction Attack
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Machine unlearning has emerged as an inevitable AI mechanism to support GDPR requirements such as revoking user consent through the "right to be forgotten". 
However, existing approaches often leave residual traces that make them vulnerable to data reconstruction attacks. 
In this work, we propose ReTrace, the first reconstruction attack framework that uniquely formulates unlearned data recovery on large-scale deep architectures as a reinforcement learning (RL) problem. 
By treating residual unlearning traces as reward signals, ReTrace guides a generator to actively explore the input space and converge toward the forgotten data distribution. 
This RL-guided approach enables both instance-level recovery of individual samples and distribution-level reconstruction of unlearned classes. 
We provide a theoretical foundation showing that the RL objective converges to an exponential-tilted distribution that amplifies forgotten regions. 
Empirically, ReTrace achieves up to 73.1\% instance-level recovery and reduces FID and KL scores beyond two state-of-the-art baselines. 
Strikingly, on the challenging task of text unlearning, it improves BLEU scores by nearly 100\% over black-box baselines while preserving distributional fidelity, demonstrating that RL can recover even high-dimensional and structured modalities. Furthermore, ReTrace demonstrates effectiveness across both convolutional (ResNet) and transformer-based models, with Distil-BERT as the largest architecture attacked to date. These results show that current unlearning methods remain vulnerable, highlighting the need for robust and provably private mechanisms.

Published as a conference paper at ICLR 2026
RETRACE: REINFORCEMENT LEARNING-GUIDED RE- CONSTRUCTION ATTACKS ON MACHINE UNLEARNING
Mengyao Ma Shuofeng Liu Minhui Xue Surya Nepal Guangdong Bai*
The University of Queensland "CSIRO's Dataó1 Ressponiible Al Research (RAIR) Centre, Adelaide University *City University of Hong Kong
(mengyao. ma, shuofeng. fiueeuq.- edu. {jason. xue, surya wep.aaaaaaaaaa... csiro.au g. g.bai@cityu.edu.
ABSTRACT
Machine unlearning has emerged as an inevitable Al mechanism to support GDPR requirements such revoking consent through the "right be forgotten". However, existing approaches often leave residual traces that make them vulner- able data reconstruction attacks. In this work, we propose RETRACE, the first reconstruction attack framework that uniquely formulates unlearned data recov- ery on large-scale deep architectures as reinforcement learning (RL) problem. By treating residual unlearning traces as reward signals, RETRACE guides gen- erator to actively explore the input space and converge toward the forgotten data distribution. This RI RL-guided approach enables both instance-level recovery of in- dividual samples and distribution provide thcoretical foundation showing that the RL objective converges exponential- distribution that amplifies forgotten regions. Empirically, RE- TRACE achieves up 73.1% instance-level recovery and reduces FID and KL scores beyond two state-of-the-an baselines. Strikingly, on the challenging task of text unlearning, it improves BLEU scores by nearly 100% over black-box baselines while preserving distributional fidelity, demonstrating that RL can re- cover even -ighenimensio and structured modalities. Furthermore, RETRACE demonstrates effectiveness across both convolutional (ResNet) and transformer- based models, with Distil-BER BERiT the largest architecture attacked to date. These results show that current unlearning methods remain vulnerable, highlighting the need for robust and provably private mechanisms.
reconstruction of unlearned classes. We
INTRODUCTION
Machine unlearning has recently emerged as critical technique to empower users with the ability to remove their data from trained models, aligning with the right to be forgotten in GDPR (European Parliament & Council of the European Union, 2016) and responsible Artificial Intelligence (AI) innovation. To achieve this, early work on exact unlearning enforces deletion by retraining the model from scratch on the dataset with the target samples removed. While this approach provides strong guarantee, it is computationally prohibitive for today's 's large and complex models (Ginart et al., 2019). As result, recent research has shifted toward approximate unlearning (Bourtoule et al., 2021; Li et al., 2024), which directly modifies trained model to erase the influence of unlearned data. These techniques offer more practical trade-offs between efficiency and unlearned strength, making machine unlearning a cornerstone of responsible AI innovation, where compliance with user rights and trustworthy data governance essential (Liu et al., 2024).
Despite its promise, machine unlearning is vulnerable to reconstruction attacks (Zhang et al., 2023a; Bertran et al., 2024), where adversaries attempt to recover the data that was intended to be forgot- ten, unlearning may have the unintended opposite effect: rather than concealing the data, it can inadvertently facilitate the localization of sensitive records in a vast corpus. As a result, it is easier for adversaries search the needle in the haystack. Such attacks pose direct threat to data privacy, as they might effectively reverse the unlearning process and expose sensitive information. The con- sequences be severe in domains such healthcare. For example, in medical setting, even if

---

Published as a conference paper at ICLR 2026
patient requests deletion of their medical images health records, reconstruction attack could still recover identifiable details and compromise confidentiality. Such risk highlights the urgent need to systematically investigate the vulnerability of unlearning mechanisms under reconstruction attacks.
Several recent studies (Bertran et al., 2024; Pang et al., 2025; Hu et al., 2024) have explored recon- struction under unlearning. One line relies on closed-form parameter analysis (Bertran al., 2024), which can exactly recover deleted samples but only applies to simple or linear models. Another line, update-based reconstruction (Hu et al., 2024), requires white-box gradients or parameter up- dates and is restricted to instance-level recovery, failing to generalize when multiple deletions occur. These limitations motivate the need for framework that exploits unlearning traces in deep models, supports diverse access levels, and enables both instance- and distribution-level recovery.
Our work. In this work, we propose RETRACE, the first reconstruction attack framework that exposes privacy vulnerabilities in machine unlearning on both convolutional (ResNet (He et al., 2016)) and transformer-bait based (Distil-BERG (Sanh al., 2019)) models, with Distil-BERY being the largest-scale architecture attacked date. The key insight is that unlearning leaves detectable traces between pre- and post-unlearning models (Chen et al., 2025), which can be exploited with reinforcement learning (RL). RETRACE operates in three steps. The first is trace extraction, which integrates signals such as output shifts, loss differences, and gradient alignments across model access levels, i.c., black-, grey-, and white-box. The second is RL-guided reconstruction, where generator explores the input space and optimizes trace scores as rewards, enabling recovery even from complex models. The third is candidate selection and refinement, which supports both instance-level and distribution-level recovery.
We evaluate RETRACE through both theoretical analysis and empirical validation. On the theoretical side, we first establish that the RL objective converges to an exponential-titted distribution, which provably amplifies regions with stronger unlearning traces. Building on this characterization, we show that RETRACE can successfully recover unlearned data at the instance level with high prob- ability once sufficient number of candidates are generated. Furthermore, using a bias-variance decomposition, we demonstrate that the empirical distribution of reconstructed samples converges toward the deleted-data distribution, with the bias controlled by the separability margin and the variance diminishing with the number of samples.
On the experimental side, we first visualize unlearning traces across different access levels and confirm that unlearned data leave consistent, instance-aligned residual signals. We then evaluate RETRACE on three benchmarking datasets under both exact and approximate unlearning. At the instance level, RETRACE strating its ability to generate faithful reconstructions. At the distribution level, it can reduce the Fréchet Inception Distance (FID) to 99.1 and the Kullback-Leiblrr (KL) divergence 2.53, showing strong alignment with the unlearned data distribution. Compared with baseline methods, RETRACE consistently attains lower MSE, higher success rates, and stronger feature similarity, confirming its superior effectiveness. Ablation study further demonstrates its robustness and generalization.
achieve the best success rate of 73.1% with an MSE of 0.17, demon-
Contributions. Our main contributions summarized as follows.
An impactful vulnerability unveiled on machine unlearning. Most current unlearning techniques do not completely erase the influence of deleted data but instead leave discernible traces in the model. We find that these residual traces can be leveraged as reliable signals to reconstruct the supposedly unlearned data.
A novel reconstruction attack framework. We propose RETRACE, a trace-guided RL- based reconstruction attack that systematically exploits aalearning-induce traces to recover unlearned data at both the instance and distribution levels. Different model access levels further make RETRACE more practical.
A comprehensive study and evaluation. We provide a theoretical foundation for RETRACE in reconstructing the unlearned data at an instance level and distribution level, demonstrating its effectiveness. We also conduct comprehensive experimental evaluations to further show its performance.
2

---

Published as a conference paper at ICLR 2026
2 PROBLEM STATEMENT
2.1 SYSTEM SETTING
Unlearning settings. We consider the standard machine unlearning setting. Let D Dre U Deel denote the training dataset, where Iee represents the samples to be retained and Daet represents the samples requested to be deleted. A model is first trained on the full dataset 'D resulting in the pre-unlearning model It with parameters 0+. Upon receiving deletion request, an unlearning algorithm is applied to remove the influence of Daet. producing the post-umlearning model f with parameters .-. The objective of unlearning to ensure that f behaves as if it were trained only on Dra, thereby eliminating any contribution of Dael to the model.
RETRACE position. We position RETRACE as new form of reconstruction attack against machine unlearning. By leveraging RL, it progressively reconstructs the forgotten data (either at the instance level or distribution level) from residual traces left after unlearning.
2.2 THREAT MODEL
Adversary knowledge. We assume the adversary has access to both the pre-unlearnigg model ft and the post-unleaming model f_, scenario that arises in practice through model versioning, API updates, or common deployment practices (see Appendix A for evidence). Depending on the deployment, we consider three levels of model access. Black-box access: The adversary can query ft and f and obtain their prediction outputs (e.g.. labels and prediction probabilities). Grey-box access: In addition outputs, the adversary can compute the loss with respect the task loss function. White-bor access: The adversary has full knowledge of model parameters 6+ and 0 so that they can also obtain the gradient in training.
The adversary does not have access to the original training dataset D or the forgotten set D Instead, they may rely on an auxiliary public dataset Ppub drawn from similar distribution' which can be used to initialize candidate inputs for trace extraction.
Attack goal. The adversary's goal is reconstruct the data belonging to the forgotten class. We consider two levels of reconstruction: (i) instance-level, where the adversary attempts to recover an individual sample in the unlearned class; and (ii) distribution-level, where the adversary aims to ap- proximate the overall distribution of Dae- Both types of recovery undermine the privacy guarantees of unlearning and expose sensitive information that should have been removed. Formally, the goal can be expressed follows.
Instance-level reconstruction. Let dx : X' X + R>0 be a appropppooeii metric. Given e > 0, say adversary A succeeds at instance-level reconstruction if it outputs 3 E X' such that
3 (I. v) E Daa with dx (f, x) S <.
For a k-set X = (..,..., } success can be measured by the minimum matching distance (MMD) MMDist( & Daei) = min, E (x("), (r))) and threshold e.
Distribution- reconstruction Let Paa denote the (unknown) distribution of inputs in the deleted set Daet and let _ be the distribution induced by the reconstructed samples. We say the adversary achieves distribution-level reconstruction if the reconstructed distribution PP is close to IPae under some statistical distance d(;;)) i.e., d(IP, Paa) e, for small threshold 0.
APPROACH
In this section, we delve into the internals of RETRACE. data by exploiting residual traces left after unlearning. Our insight that when model undergoes unlearning, the discrepancy between the pre-unlearning model f and the post-unlearning model f inevitably reveals subtle unlearning traces (Chen et al., 2025). These traces provide exploitable
It is designed to reconstruct unlearned
Similar distribution means the auxiliary public data can helong to the same semantic class as the forgotten sample, but imply population-level access.
3

---

Published as a conference paper at ICLR 2026
Trace Extraction
RL--Based Generation
Reconstruction
Mapping
Instunce
Candidate
Reconstraztion
Sample
Thace Score
Reward
Generator
5(x) (0,
A
Reluforecmeut Learning
Distribution Level Reconstrction
Traces
Figure I: Overall workflow of RETRACE.
signals regarding the unlearned data. To leverage these signals effectively, we formulate the re- construction problem as a reinforcement learning (RL) task. The learning agent explores the input space, and the feedback it receives, derived from unlearning traces, serves the reward. By opti- mizing toward higher rewards, the agent gradually converges to regions of the input space that better approximate the unlearned data. The overall workflow of RETRACE is illustrated in Figure
3.1 UNLEARNING TRACE EXTRACTION
The first step of RETRACE is extract unlearning traces, which define as residual signals that expose the behavioral differences between the pre-unlearning model f+ and the post-unlearning model f- Since such differences only manifested when the models queried with inputs, probe the models with a candidate input x and extract the trace. Given & drawn from a public dis- tribution pub sampled from the generator Tg. RETRACE quantifies traces three progressively informative levels depending on the degree of model access. We follow (Sarmad et al., 2019) in designing trace extraction strategies tailored different access levels.
Prediction-level traces. With black-box access, we measure the prediction discrepancy between f+ and f check whether unlearning leaves residual influence on the output distribution. The trace is defined as the l distance between their prediction vectors:
(ped(()) = [1f (f (x)]l2-
(2)
Loss-level traces. With grey-box access, we extract traces from the task loss f(-). Using pseudo- label 9 from ft, we compute the absolute difference between the two models' losses:
8 (=) ((( (z). {s (x).
Gradient-level traces. With white-box access, we capture differences in sensitivity to input pertur- bations. The trace is measured by the cosine distance between input gradients:
&gra (x)
cos (V.((f (z), 0), V (x). (x).
(4)
Together, the unlearning trace of an input & under different access levels can be represented as:
(z))
black-box access,
T(x) = (opnaa (z), Sioss (x)))
grey-box access,
(5)
(0)r) Soas (x), Sgma(z)))
white-box access.
T(x) is then used to guide the unlearned data reconstruction process.
3.2 RL-GUIDED GENERATION
Formulation as RL. RETRACE formulates data reconstruction as RL problem in which policy network actively explores the generator's s latent space and is guided toward regions that preserve unlearning traces. The RL iteration begins by sampling latent vector 2o AO(0,) I), representing an uninformed prior over the generator input space. policy network тe. (e.g.. implemented as a multilayer perceptron (MLP)), transforms 2g into new latent vector passed to the fixed generator G' to synthesize an image Φ; = G(z,). Through policy optimization with iterations, Z, will be generated, representing the unlearned image.
no T2(2)), which is then

---

Published as a conference paper at ICLR 2026
The environment consists of the pre- and post-unlearnigg models (ft f). For generated sample Fi, the environment returns a scalar reward s(2;) = (ainmmax(((z)) where
cxpreed (zi)):
black-box,
r(x))
-dprod (Ix)) - B8ross (24),
grey-box,
(6)
adpred (x:)) Bd (21)) (Bgrad((s), white-box,
with y 2 0 controlling the scale, and s(x;) E [0, 1] is the normalized trace score. High trace scores indicate that x; lies closer to the forgotten data manifold, thus providing meaningful learning signals for the RL agent.
Policy optimization via PPO. To optimize the policy network Te, we adopt an actor critic version of Proximal Policy Optimization (PPO) (Schulman et al., 2017). Specifically, in the first training round, for synthesized sample 21 = G(21), its trace score s(x1) first calculated, preparing for the following optimization, which starts from the second round. In subsequent rounds, PPO does not directly update the policy using the absolute reward s(x;); instead, it uses the advantage
A s(x;) Vy(zi),
where Vi is critic network that estimates the expected trace score (reward) at latent state zς. The critic is trained to regress toward s(z;) and therefore provides a learnable baseline that stabilizes the training dynamics by ensuring that updates depend on reward improvements. The sign of A++1 indicate whether the update direction is desirable. A positive ad udvantage implies that the trace score obtained in the previous round exceeds the critic's expectation, meaning that the policy update has improved the generator's output and should continue in similar direction. Conversely, non- positive A++1 indicates that the optimization suboptimal. When |A,+1| is small, the update yields little improvement, whereas large negative advantage suggests that the update may move in the op- posite direction the desired behavior, so the policy should adjust its update direction accordingly.
Through each iteration, the clipped PPO surrogate objective is
(8) where ra(2;) is the importance ratio between the new and old policies, and is small positive clipping parameter (typically 0.1 0.2) that limits how much the new policy is permitted to diverge from the old one, ensuring stable and conservative updates. This objective encourages the actor to adjust Ty such that the generated latent codes 2i+1 (2();) yield larger trace scores than those predicted by the critic baseline. As training progresses, both s(z;) and the critic estimate (*+ in- crease, and the advantage A++1 gradually approaches zero, indicating convergence. At convergence, the policy consistently steers latent vectors toward regions in which G(2+1)) reconstructs samples that align closely with the unlearned data manifold.
(0) ( [min(ro(z:) Art1, clip(ro(zz), e, 1,1+))
+c) A++1)]
By repeated interaction, the policy progressively reshapes the latent codes such that G'(z,) yields images that maximize the discrepancy between f and f- This guided exploration enables RE- TRACE to recover samples aligned with the unlearned data.
3.3 RECONSTRUCTION
After the RL stage, is well-trained. RETRACE generates d reconstructed images by using independent initial latent vectors {=] { J==1 (N, I). Then, for each d, their trace score (d,)
is calculated for further best reconstruction sample selection. Based on RETRACE's RL-guided generation, two levels of reconstruction can be carried out.
Instance level. At the instance level, we identify the top-scoring candidate
à = arg max sd:), d;
(9)
which serves as the best approximation of an unlearned instance.
Distribution level. At the distribution level, goal is to approximate the forgotten data distri- bution. We first rank all generated candidates by their trace scores {s(d,)) and select the top-k elements:
Deorget = (d E Ix),
(10)
where T indexes the l highest-scoring samples.
5

---

Published as a conference paper at ICLR 2026
3.4 DISCUSSION ON EFFICIENCY
The overall complexity of RETRACE is O(I N Cf + N log N), dominated by generator sam- pling and trace evaluation across RL iterations with N candidates per iteration. Here, Cy denotes the cost of a single model forward/backward pass. In practice, the runtime comparable to stan- dard adversarial training or RL-based generation pipelines. Detailed explanation on efficiency is in Appendix B.
4 THEORETICAL ANALYSIS
In this section, we provide a theoretical foundation of RETRACE's attack effectiveness. Our goal to formally establish that, under mild assumptions, RL optimization in RETRACE converges to policies that maximize unlearning traces, hence enabling effective reconstruction of the unlearned data distribution. Although RETRACE implemented using PPO for stability, PPO's clipped surro- gate objective does not correspond to well-defined functional over policy distributions. Therefore, following standard practice RL theory, we analyze tractable KL-regularized surrogate objec- tive that captures two essential behaviors of PPO-reware maximization and conservative policy updates and is used solely as an analytical abstraction (see full proofs in Appendix C).
4.1 DEFINITIONS AND ASSUMPTIONS
Definition 1 (Trace score). Let X 10 denote the (fixed) trace score produced by the detector, quantifying the strength of unlearning traces for any x E X.
Definition (Trace separability). Let Past and Peet denote the input distributions of the deleted and retained daia. Traces are separable if there exists margin A >> O such that
Ez--an [s(x)) 2 BE-FEe [s(x)] A.
(11)
Assumption (Policy expressiveness). The policy class II contains densities absolutely continu- ous w.r.t a public prior po that can approximate Paet in total variation arbitrary precision. In particular, Il includes the exponential-tilinng family
xx(x) po(x) exp(Aa())))
AER.
(12)
Assumption (Abstract RL convergence). The policy distribution x. E II is optimized through the RI procedure described in Section 3, and its evolution can be abstracted by the maximization of strictly concave functional Jr (T) of the form
J+(T) Be [s(x]] (KL(x(1/p))
7 >> 0.
(13)
We assume the optimization dynamics converge almost surely stationary point of Jr within II.
This functional is introduced solely as an analytical surrogate and is not required to coincide exactly with the implementation in Section rather, provides a tractable abstraction that captures the monotonic improvement in expected trace score and policy smoothness.
Theorem 1 (Optimal policy form). Under Assumptions and 2, any stationary point n² of the abstract functional Tr takes the form
म (x) x (2())) exp(s(x)/T).
(14)
and is the unique maximizer within the policy space TT.
Theorem 1 proves the exponential -irtlttti--tteeedd form x* x po e by using a variational (Lagrangian) derivation under the normalization constraint together with the striet concavity of J+: the complete proof is provided in Appendix C.I.
Theorem (Instance-level reconstruction). Let T²* as Theorem and Define
== n T (<sppp((Paa))
(15)
Then for k i.i.d. candidates drawn from n¹ and refined by the local step in Section 3.3,
Pr 3j S k: min dx(0;, x) < 2 (1-11pp))) (1-p.)*. p.)*
(16)
xePaol

---

Published as a conference paper at ICLR 2026
1.
18g
Normaliod trace
0.0
6.5
Normalisn
(a) Original
(b) Black-box
(c) Grey-box
(d) White-box
Figure 2: Traces of unlearned data under different model access levels on CIFAR- in the approx-
imate unlearning scenario. The images in red boxes represent the unlearned data "Apple".
2 8 Normalized trace score
1.
1.0
0.8
Kormall hormarrcc basce
0.6
1
0.2
(a) Original
(b) Black-hox
(c) Grey-box
(d) White-box
Figure 3: The normalized version of Figure 2 show the advancement in Grey- and White-box.
Theorem 2 guarantees instance-level success by using the fact that x assigns positive probability mass to any e-neighborhood of supp(Paaa) and a standard i.i.d. coverage bound 1 - - pa)* the full proof appears in Appendix C.2.
Theorem (Distribution-level reconstruction). Let Ps be the empirical distribution of the recon- structed candidates. Under Definitions 1, 2 and Assumptions 1, 2, there exists a bias term Ci(T, A) such that, for sufficiently large
d d(Pr, Fda) I Ci(T,A) + c(k,6).
(17)
with probability at least 1 - 5. Here d(-,.) )) is a general statistical distance (e.g., Maximun Mean Discrepancy (MMD). Wasserstein), and c(k, 8) is the sampling error vanishing as oc.
Theorem 3 establishes distribution- reconstruction by using biri-variance decomposition d( d(Pk,, Pacl) & d( d(Pk. 7²) + d(x= Pacct) where the bias stems from exponential tilting (controlled by (T, A)) and the sampling term decays with k; the detailed argument is given in Appendix C.3.
5 EVALUATION
5.1 EXPERIMENTAL SETUP FOR MAIN EXPERIMENTS
We evaluate RETRACE on three datasets: CIFAR- 100 (Krizhevsky et al., 2009), Food-101 (Bossard et al., 2014), and PathMNIST (Yang et al., 2023), using ResNet-18 (He et al., 2016) classifiers with paired models (f+,, f-) obtained by both exact and single gradient (approximate) unlearning (Thudi et al., 2022). Unlearning is performed in a class-wir wise manner with class O as the forgotten category. DCGAN-style (Radford et al., 2015) generator used for image tasks, guided by an RL policy to exploit unlearning traces across black-, grey-, and white-box access levels. We assess recon- struction quality at the instance level using mean squared error (MSE), cosine similarity (CS) 2, and success rate (SR), and at the distribution level using Fréchet Inception Distance (FID) and Kull- back-Leibler (KL) divergence. For comparison, we include two state-of-the-ar baselines: Unlearn- ing Inversion Attack (UIA) (Hu et al., 2024) and HRec (Bertran et al., 2024). Detailed experimental settings are in Appendix D.I.
5.2 TRACE VISUALIZATION
To empirically demonstrate the feasibility of using traces as reconstruction signals, we visualize the traces of unlearndd data with heatmaps and further present their distributions, showing that the
We extract features from the pool3 layer of ImageNet-pretraine Inception- V3 network to compute cosine similarity.

---

Published as a conference paper at ICLR 2026
Trace score
lfate score
0.3
0.4
Trace
0.2
20
40
60
80
100
0.0
20
40
60 label
80
100
0.0
20
40
60
80
100
Class
label
Cless
Class
label
(a) Black-box
(b) Grey-box
(c) White-box
Figure 4: Trace distribution under different model access levels on CIFAR-I00 the exact unlearn- ing scenario. Class O represents "Apple".
traces of unlearned data exhibit distinct patterns. Below, we explain the results on CIFAR-100 under approximate unlearning; the full evaluation results are provided in Appendix D.2.
Trace of unlearned data. Figure 2 visualizes unlearning traces on CIFAR- 100 across three access settings. Panel (a) shows 5 5 patch of original images with unlearned samples ("Apple") marked in red; Panels (b)-()) present heatmaps of the normalized trace scores on the same grid. In the black-box setting, unlearned samples already exhibit top-quantile scores, showing that prediction- level discrepancies leave detectable signals. Grey-box access sharpens localization as loss-level differences enhance the contrast between forgotten and retained samples. White-box access fur- ther amplifies this effect: gradient-level information yields the clearest and most stable separation. Overall, unlearned data consistently leaves residual, settance-aligned traces across all access levels.
Additionally, to show the improvement in the white-box setting, Figure 3 provides a normalized version of Figure that can make the contrast between the black-box, grey-box, and white-box settings more visually explicit. We have computed, for each 5 x 5 patch, the average trace value and then get the ratio between the highlighted red-square values and the corresponding patch average. We plot this visualization and show that the normalized ratios further amplify the relative contrast: the white-boo box and grey-box settings shows the largest ratio gap between unlearned and retained samples, while the black-box setting exhibits noticeably weaker ratios. This matches the quantitative behavior observed in our evaluation and reinforces the phenomenon that higher model access level contains more information regarding trace scores.
Trace distribution. We further analyze trace distributions across access levels. Figure shows class-wise mean trace scores CIFAR-100, with class 0 as the unlearned category "Apple". In the black-box setting, class 0 already scores higher than others, indicating measurable separation from prediction-level discrepancies. Grey-box access enlarges this gap by incorporating loss-level information, while white-box access further sharpens the contrast using gradient-level traces. Over- all, the unlearned class consistently exhibits distinctly higher trace values, confirming that residual traces extend beyond instances to distributional patterns.
5.3 EFFECTIVENESS OF RETRACE
We first benchmark the instance-level and distribution-level reconstruction capabilities of RETRACE under three model access settings across three benchmarking datasets, and then compare its performance with two state-of-the-arr reconstruction attacks. Figure 5 provides some generated cases by RETRACE, and more cases are presented in Appendix D.3.
(a) Instance.
(b) Distribation.
Figure 5: Generated unlearned sam- ples ("Applle class") in CIFAR-100 under exact unlearning.
Benchmarking. For instance-level recovery, Ta- ble 1 shows that reconstructed samples achieve MSE values close to the intra-class criteria (e.g., CIFAR-100: 0.20 vs. 0.16), with SR around 50-60% under exact unlearning and further increasing under approximate unlearning (e.g., CIFAR-100: 73.1%) Mcanwhile, CS values remain close to the criteria (e.g.. Food-101: 0.49 vs. 0.54), in- dicating that recovered images preserve semantic similarity to the original unlearned data. These results confirm that RETRACE reliably regenerate unlearned samples the instance level. Since the results presented Table reflect the mean of the runs, Table in Appendix D.5 further shows their standard deviation.
At the distribution level, Figure 6 reports FID and KL divergence. Even under restrictive black- box access, RETRACE achieves reasonable alignment with the deleted distribution (e.g., CIFAR-
8

---

Published as a conference paper at ICLR 2026
Table The effectiveness of RETRACE from instance-level reconstruction
Exacl Unlearning
Approximale Unlearning
Dataset
Black-box
Grey-box
White-box
Black-box
Grey-box
White-box
MSE
SR
CS
MSE
SR
CS
MSE
SR
CS
MSE
SR
CS
MSE
SR
CS
MSE
SR
CS
CIFAR-100
0,23
52.3%
0.43
0.22
58.7%
0,20
62.4%
0.47
0.24
55.9%
0.46
0.21
68.7%
0.49
0.17
73.1%
Food-101
48.9%
0.38
0.23
54.7%
50.4%
0.44
0.20
56.6%
62.5%
0.49
0.16
65.3%
PathMNIST
0.26
49.8%
0.26
0.23
50.6%
0.27
0.22
52.8%
0.31
0.24
51.9%
0.28
0.20
54.6%
0.28
0.19
59.7%
0.33
Criteria: CIFAR-100 100 (MSE = 0.16, = 0.57), Food-101 (MSE 0.13, 0.54); PahhMNIST (MSE 0.13, -S:0.41).
Table 2: Effectiveness comparison of RETRACE and baselines under white-box model access.
Exacl Unlearning
Approximate Unlearning
Dataset
UIA
HRoc
RETRACE (Ours)
UIA
HRec
RETRACE (Ours)
MSE
SR
CS
MSE
SR
CS
MSE
CS
MSS
SR
CS
MSE
SR
CS
MSE
SR
CIFAR-100 100
0.35
46.7%
0.42
0,27
41.9%
0.31
0.20
62.4%
0.47
0.33
59.5%
0.39
0.32
43.0%
0.33
0,17
73.1%
0.50
Food-101 101
0.33
36.2%
0.38
0.46
27.4%
0.31
0.23
50.4%
0.44
0,31
41.4%
0.40
0.45
30.2%
0.33
0.16
65.3%
0.49
PathMNIST
0.43
41.0%
0.26
0.42
33.6%
0.22
0.22
52.8%
0.31
0.39
37.4%
0.24
0.44
37.1%
0.20
0.19
39.7%
0.33
100: FID 142.1, KL 3.62). Adding loss-level traces in the grey-box setting further reduces both FID and KI, and the white-box setting yields the best reconstruction quality (e.g., CIFAR- 100: FID 108.9, KI. 2.49; PathMNIST: FID 96.5, KI. 2.22). Approximate unlearning consistently improves results, with lower FID) and KI. across datasets (e.g.. CIFAR-100: FID) 99.1; PathMNIST: KL 2.00), highlighting that residual traces left by approximate methods are casier to exploit. Together, these findings demonstrate that RETRACE achieves robust recovery of unlearned data across both granular instance-level signals and broader distributional structures. The detailed analyses are presented in Appendix D.3.
Comparison with baseline methods. Table 2 compares RETRACE with UIA (Hu al., 2024) and HRec (Bertran et al., 2024) under the white-box setting, as both baselines are restricted to this access level. the exact unlearning scenario, RETRACE achieves lower MSE (e.g.. CIFAR- 100: 0.20 vs. 0.35 for UIA and 0.27 for HRec), higher SR (e.g.. PathMNIST: 52.8% 41.0% for UIA), and higher CS (e.g., Food-101: 0.44 vs. 0.38 for UIA and 0.31 for HRec), indicating more faithful pixel- and feature-level recovery. Under approximate unlearning, RETRACE further outperforms both baselines across all metrics, with lower MSE (e.g., PathMNIST: 0.19 vs. 0.39 for UIA and 0.44 for HRec), higher SR (e.g., CIFAR-100: 73. % 59.5% and 43.0%), and the highest CS (e.g.. Food-101: 0.49 vs. 0.40 and 0.33). Overall, RETRACE consistently surpasses UIA and HRec, demonstrating superior effectiveness in reconstructing unlearned data.
We also conduct experiments to compare our methods with baselines on distribution-leve recon- struction. The results show that RETRACE still outperforms UIA and HRec in terms of KL and FID, demonstrating its effectiveness. The detailed results are listed in Table in Appendix D.4.
5.4 ABLATION STUDY
We conduct ablation studies to demonstrate the robustness and generalization of RETRACE. We also discuss its potential limitations in Appendix D.11.
Robustness. Table 3. presents the ab- lation results on three unlearned classes from CIFAR-100 (Aquarium fish, Bed, and Bridge) under the white-box setting. Across all three categories, RETRACE de- livers strong performance on all metrics: MSE remains low (0.21-0.23 for exact un- learning and 0.16-0.19 for approximate un- learning), SR consistently stays above 59% and rises around 70% in the approximate setting. and CS values remain stable in the range of 0.41-0.44. These results demonstrate that RETRACE maintains robust reconstruction ability across different unlearned data classes. Some generated cases are listed Figure 18 in Appendix D.7.
Table 3: The effectiveness of RETRACE on different unlearned class reconstruction.
Exact Unlearning
Approximate Unlearning
Class ID
MSE
SR
CS
MSE
SR
CS
Aquarium fish
0.21
59.1%
0.41
0.18
70.2%
0.44
Bed
0.22
60.5%
0.42
0.19
69.2%
0.43
Bridge
0.23
60.8%
0.42
0.16
71.6%
0.42

---

Published as a conference paper at ICLR 2026
CIFRR-00
Ropd 101
PAtHANET
128-0 90.1
221,2125.0
200
90.1 92.4
Eswet Unteaming
2:33 2.59
3.33 3.62
1:28 2.00
2.00
Approximane Unlearning
Blckkbe box
Grey box
White box
Black box
Gray bax
White box
Black-bok
Gray box
White box
Figure 6: FID and KL scores of RETRACE from distribution-level reconstruction
Generalization. For text reconstruc- tion (Ma et al., 2023b), employ GPT- 2-based (Radford et al., 2019) generator, and DistilBERT classifier (Sanh al., 2019) serves as the unlearned model un- der the exact unlearning setting. We eval- uate performance on the AG News (Zhang al., 2015) dataset using two metrics: BLEU (Papineni et al., 2002), which mea-
Table 4:
The effectiveness of RETRACE on text re-
construction
Unlearned Class
Black-box
Gray-box
White-box
BLEU
MMD
BLEU
MMD
BLEU
MMD
Sports World
2.8
0.40
3.6
0.36
4.1
0.31
2.2
0.43
3.9
0.42
4.3
0.37
Business
2.6
0.46
0.46
4.6
0.32
Sci/Tech
2.0
0.41
3.7
0.43
4.0
0.39
sures content similarity between generated and target texts, and MMD (Gretton al., 2012), which assesses the distributional alignment across classes. In Table 4, RETRACE consistently improves with stronger access levels. For example, BLEU increases from 2.8 (Sports, black-box) to 4.6 (Busi- white-box), while MMD decreases across all classes (c.g., 0.46 to 0.32 for Business), indicat- ing progressively better distributional fidelity.
RELATED WORK
Machine unlearning. Machine unlearning (Guo et al., 2020; Bourtoule al., 2021; Chen et al., 2022: Thudi et al., 2022; Liu et al., 2025b) enables models to forget specific training data. Ex- isting methods are either exact or approximate: exact unlearning retrains on the retained dataset with strong guarantees but high cost (Ginart et al., 2019; Guo et al., 2020), while approximate un- learning updates parameters or gradients for efficiency (Bourtoule et al., 2021; Jia et al., 2024a; Li al., 2024). Techniques include partition- based retraining (Bourtoule al., 2021), gradient adjust- ment (Thudi et al., 2022), and adaptive methods (Gupta et al., 2021), with extensions to graphs and LLMs (Chen et al., 2022; Jia al., 2024b; Liu et al., 2025b). Machine unlearning is often evaluated using membership inference attacks (Shokri et al., 2017; Ma et al., 2023a; Zhang et al., 2023b) to measure residual memorization.
Reconstruction attacks. Reconstruction attacks aim to recover sensitive training data, first studied via model inversion and gradient leakage (Fredrikson al., 2015; Zhang et al., 2023a; Bertran al., 2024; Pang et al., 2025; Ma et al., 2024; Feng et al., 2024). In unlearning, this risk is amplified, as pre-post model differences can intensify leakage (Liu al., 2025c). Existing studies either analyze parameter shifts, which succeed in linear models but fail on deep networks, or exploit updates and gradients to optimize synthetic inputs (Ginart et al., 2019; Bourtoule et al., 2021). While recent inversion attacks show that model differentials can expose unlearned data, current approaches remain limited to instance- recovery and do not scale to distribution- scenarios.
CONCLUSION
We propose RETRACE, reconstruction attack framework that systematically exploits residual traces left by machine unlearning. By identifying traces across access levels and leveraging them as rewards, RETRACE enables both instance- and distribution-level reconstruction. edge, this is the first demonstration of reconstruction attacks on large-scale architectures, showing effectiveness on ResNet-18 for vision and Distil-BERT for text. We provide theoretical guarantees of convergence to exponential-tited policy that amplifies high-trace regions, ensuring reliable recovery, and validate the approach with extensive experiments across datasets, unlearned classes, models, and unlearning methods. Our findings expose critical vulnerabilities of current unlearning techniques and highlight the need for robust unlearning mechanisms, positioning RETRACE as both principled framework for analyzing privacy risks and practical benchmark.
our knowl-
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
We thank the anonymous reviewers for their insightful comments to improve this manuscript. Jason Xue and Surya Nepal are supported part by the Responsible Research (RAIR) Centre, a land- mark collaboration between Australia's 's national science agency, CSIRO, and Adelaide University, in partnership with the South Australian Govermment. This work was in part done when Guangdong Bai was with University of Queensland.
ACKNOWLEDGEMENT OF LLM USE
The use of the LLM in this work was strictly confined to linguistic refinement, including grammar and readability. All intellectual and technical contributions are the sole responsibility of the authors.
ETHICS STATEMENT
Our research adheres the ethical principles outlined by the ICLR Code of Ethics. All experiments in this work are conducted on publicly available and fully anonymized datasets, including CIFAR- 100, Food-101, PathMNIST, and AG News. These datasets are widely used in the machine learning community and do not contain personally identifiable information (PII) or sensitive attributes di- rectly linked to real individuals. For text-based tasks, the corpus consists of curated for research purposes, where all identifying information has been removed. For medical data (i.e., PathMNIST), the dataset part of the MedMNIST collection, which is designed to be de-identified and ethically suitable for public use. Therefore, this research does not involve the collection, processing, or exposure of human-subject data, and Institutional Review Board (IRB) approval is not required.
articles already
We further emphasize that our work focuses on methodological contributions to the study machine unlearning and reconstruction attacks. The purpose of this research is analyze vulnerabilities under controlled and responsible experimental settings, and to call for defenses and mitigations in subsequent studies. All experimental results reported for the sole purpose of ad advancing scientific understanding of unlearning privacy issues and informing the development more trustworthy AI systems.
REPRODUCIBILITY STATEMENT
We have taken several steps to ensure the reproducibility of our work. All datasets used in this study are publicly available benchmarks, including CIFAR-100, Food-101, PathMNIST, and AG News. The links for dataset access are provided in our paper. For theoretical analysis, all assump- tions, definitions, and theorems are listed in Section 4. The complete proofs are provided in the Appendix C. The experimental settings, including unlearning techniques, model architectures, RE- TRACE parameter configurations, and training schedules, are fully detailed in Sections 5.1, D.I, and D.10. We report results across multiple datasets and model-access levels (black-, grey-, and white-box), and also provide ablation studies and text generalization experiments to demonstrate robustness. An repository with our method implementation and training scripts is available petps::/githinn co System- Rrace..
REFERENCES
Martin Bertran, Shuai Tang, Michael Kearns, Jamie H Morgenstern, Aaron Roth, and Steven Wu. Reconstruction attacks machine unlearning: Simple models are vulnerable. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), volume 37, pp. 104995- 105016, 2024.
Lukas Bossard, Matthieu Guillaumin, and Lue Van Gool. Food-101 discriminative com- ponents with random forests. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 446-461. Springer, 2014.
11

---

Published as a conference paper at ICLR 2026
Lucas Bourtoule, Varun Chandrasekaran, Christopher A Proqueett-Chhoo Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium Security and Privacy (S&P), pp. 141-159. IEEE, 2021.
Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang- Graph unlearning. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Com- munications Security (CCS), pp. 499-513, 2022.
Yiwei Chen, Soumyadeep Pal, Yimeng Zhang, Qing Qu, and Sijia Liu. Unlearning isn't invisible: Detecting unlearning traces in Ilms from model outputs. arXiv preprint arXiv:2506. 14003, 2025.
European Parliament and Council of the European Union. Regulation (EU) 2016/679 of the Euro- pean Parliament and of the Council, 2016. URL https: htt.ddaaaa...rr.eaeeeeeeee 2016 6////79//
europa eu/eli/ree/ reg/
Xinguo Feng, Zhongkui Ma, Zihan Wang, Eu Joe Chegne, Mengyao Ma, Alsharif Abuadbba, and Guangdong Bai. Uncovering gradient inversion risks in practical language model training. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 3525-3539, 2024.
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confi- dence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Confer- ence on Computer and Communications Security (CCS), pp. 1322-1333, 2015.
Antonio Ginart, Melody Guan, Gregory Valiant, and James Y Zou. Making ai forget you: Data deletion in machine learning. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), volume 32, 2019.
Google Cloud, Safe model rollouts with canary deployments. https:://loodee http:///s/s///oolooogggl..oo// blog, a. Accessed Sept 2025.
Google Cloud. Vision API release notes. https: http::://cloud.. http://w..o.oo....oo/ooeeeeeeeaa release-notes, b. Accessed Sept 2025.
Arthur Gretton, Karsten M Borgwardt, Malte Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The journal of machine learning research, 13(1::733-773, 2012.
Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten. Certified data removal from machine learning models. In Proceedings of the 37th International Conference on Machine Learning (ICML), pp. 3832-3842, 2020.
Varun Gupta, Christopher Jung. Seth Neel, Aaron Roth, Saeed Sharifi- Malvajrdi,,,, and Chris Waites. Adaptive machine unlearning. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), volume 34, pp. 16319-16330, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
Hongsheng Hu, Shuo Wang, Tian Dong, and Minhui Xue. Learn what you want to unlearn: Unlearn- ing inversion attacks against machine unlearning. In Proceedings of the 2024 IEEE Symposium on Security and Privacy (S&P), pp. 3257-3275 IEEE, 2024.
Jinghan Jia, Jiancheng Liu, Yihua Zhang, Parikshit Ram, Nathalie Baracaldo, and Sijia Liu. Wa- gle: Strategic weight attribution for effective and modular unlearning in large language models. Advances in Neural Information Processing Systems, 37:55620-55646, 2024a.
Jinghan Jia, Yihua Zhang, Yimeng Zhang, Jiancheng Liu, Bharat Runwal, James Diffenderfer. Bhavya Kailkhura, and Sijia Liu. Soul: Unlocking the power of second-order optimization for Ilm unlearning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4276 4292, 2024b.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
12

---

Published as a conference paper at ICLR 2026
Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, et al. The wmndp benchmark: Measuring and reducing malicious with unlearning. arXiv preprint S.Kiv:24033032216 2024.
Shuofeng Liu, Zihan Wang, Minhui Xue, Long Wang, Yuanchao Zhang, and Guangdong Bai. Being transparent is merely the beginning: enforcing purpose limitation with polynomial approximation. In Proceedings of the 33rd USENIX Security Symposium (USENIX Security), pp- 6507-6524, 2024.
Shuofeng Liu, Mengyao Ma, Minhui Xue, and Guangdong Bai. Modifier unlocked: Jailbreaking text-to-image models through prompts. In Proceedings of the IEEE Symposium Security and Privacy (S&P), pp. 355-372. IEEE, 2025a.
Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalic Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, et al. Rethinking machine unlearning for large language models. Nature Machine Intelligence, pp. 1-14, 2025b.
Ziyao Liu, Huanyi Ye, Chen Chen, Yongsen Zheng, and Kwok- Yan Lam. Threats, attacks, and defenses in machine unlearning: A survey. IEEE Open Journal of the Computer Sociery, 2025c.
Mengyao Ma, Yanjun Zhang, Pathum Chamikara Mahawaga Arachchige, Leo Yu Zhang, Mo- han Baruwal Chhetri, and Guangdong Bai. Loden: Making every client in federated learning a defender against the poisoning membership inference attacks. In Proceedings of the 2023 ACM Asia Conference on Computer and Communications Security (ASIACCS), pp. 122-135, 2023a.
Mengyao Ma, Shuofeng Liu, MAP Chamikara, Mohan Baruwal Chhetri, and Guangdong Bai. Un- veiling intellectual property vulnerabilities of gan-based distributed machine learning through model extraction attacks. In Proceedings of the 33rd ACM International Conference on Informa- fion and Knowledge Management (CIKM), pp. 1617-1626, 2024.
Zhongkui Ma, Xinguo Feng, Zihan Wang, Shuofeng Liu, Mengyao Ma, Hao Guan, and Mark Hua- song Meng. Formalizing robustness against character-level perturbations for neural network lan- guage models. In Proceedings of the International Conference on Formal Engineering Methods (ICFEM), pp. 100-117. Springer, 2023b.
Meta Engineering. How Facebook ships code: Canary and shadow testing. engineering. com/. Accessed Sept 2025.
https://
Microsoft Azure.
Cognitive services API versioning.
https://
learn. mi mrosoft.. cognitive-
iineeiitiiiiiii vi-e-vieeee servi / Accessed Sept 2025.
NIST. Al risk management framework (AI RMF) pilot: Model state documentation. https:: www. /t -ttp:rrrsssttmameeeeeeetrrraaeee -fr.ewwokraawwkk....mooo 2023. Accessed Sept 2025.
OpenAl. Models documentation: GPT-4 versions. https://platform..orr
com/
docs htt.///oolssss.eeo s hptp//ptt4440 4. Accessed Sept 2025.
Shuchao Pang. Zhigang Lu, Haichen Wang, Peng Fu, Yongbin Zhou, and Minhui Xue. Reconstruc- tion of differentially private text sanitization via large language models. In Proceedings of the 28th International Symposium on Research in Attucks, Intrusions and Defenses (RAID), pp. 1-17. IEFE, 2025.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 311-318, 2002.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv: 15/1.06444. 2015.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models unsupervised multitask learners. OpenAl blog, 188)9, 2019.
13

---

Published as a conference paper at ICLR 2026
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019.
Muhammad Sarmad, Hyunjoo Jenny Lee, and Young Min Kim. Rl-gan-net: A reinforcement learn- ing agent controlled gan network for real-time point cloud shape completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5898-5907, 2019.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 2017.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models. In Proceedings of the IEEE Symposium on Security and Privacy (S&P), pp. 3-18. IEEE, 2017.
Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot. Unrolling sgd: Un- derstanding factors influencing machine unlearning. 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P), pp. 303-319. IEEE, 2022.
Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2-a large-scale lightweight benchmark for and 3d biomedical image classification. Scientific Data, 10(1):41, 2023.
Kaiyue Zhang, Weiqi Wang, Zipei Fan, Xuan Song, and Shui Yu. Conditional matching gan guided reconstruction attack in machine unlearning. In GLOBECOM 2023-2023 IEEE Global Commu- nications Conference, pp. 44-49. IEEE, 2023a.
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text clas- sification. In Proceedings of the Advances in Neural Information Processing Systems (NuerIPS), volume 28, 2015.
Yanjun Zhang, Guangdong Bai, Mahawaga Arachchige Pathum Chamikara, Mengyao Ma, Liyue Shen, Jingwei Wang, Surya Nepal, Minhui Xue, Long Wang, and Joseph Liu. Agrevader: Poi- soning membership inference against byzantine-robus federated learning. In Proceedings of the ACM Web Conference (WWW), pp. 2371-2382, 2023b.
A THREAT MODEL
Adversary knowledge. We assume the adversary has access to both the pre-unleanning model f+ and the post-unlearning model f- While this may appear strong, it is realistic in practice.
API versioning in production. Providers such as OpenAl expose multiple model versions simulta- neously (e.g., 9pt-4-11 3pt-4-1106-previst 06-previeww vs. 9pt-4-01 0015--reviiee where clients can query both before and after updates (OpenAI). Similarly, Google Cloud Vision and Microsoft Azure Cog- nitive Services maintain legacy API versions for backward compatibility (Google Cloud, b; Mi- crosoft Azure). These setups allow adversaries to monitor output differences across versions, rectly corresponding to access to ff*
Deployment practices. Large-scale platforms such as Meta and Google routinely use shadow testing and canary rollouts, where both old and new models serve traffic concurrently for days (Meta Engineering: Google Cloud, a). In regulated industries (e.g., finance, healthcare), rollback readiness mandatory, and older checkpoints are preserved and potentially exposed through misconfiguration or insider threats (NIST, 2023).
Together, these practices show that simultaneous access (f+ is not only plausible but fre- quently realized in real-world Al deployments (Liu et al., 2025a).
B RETRACE'S EFFICIENCY
We mathematically analyze the computational overhead of RETRACE. Its computational complexity arises from three stages. First, trace extraction requires evaluating discrepancies between the pre-
14

---

Published as a conference paper at ICLR 2026
Table 5: Average runtime per optimization component.
Siep
Average Time
PPO training (per optimization step)
0.02 0.005
DCGAN generation
and post-unlearning models. For each input æ; computing the prediction- loss-, and gradient-level traces involves forward and at most one backward propagation through the model, which incurs O(Cr) cost, where C'y denotes the per-sample model evaluation cost. Second, the RL-based generation stage, each policy update requires drawing N candidate samples from the generator and querying the detector for their trace scores, leading to a per-iteration complexity of O(N Cy). With RL iterations, this stage totals O N Cf). Finally, the reconstruction step involves ranking and refining the N candidates, which can be done in O log N) time.
Overall, the end-to-end complexity of RETRACE is
O N Cj + NlogNN
(18)
dominated by the repeated generator sampling and trace evaluation. In practice, since Cj corre- sponds to a single model forward/backward pass, the runtime remains comparable to standard ad- versarial training mennforemeen-leeannng--eee generation pipelines. Table 5 provides the results of runtime for RETRACE per round.
C DETAILED PROOFS
C.I PROOF OF THEOREM 1
Proof. We analyze the abstract optimization problem, which serves the theoretical surrogate in- troduced in Assumption 2.
[+((T) - s(z))(t)dr T/ ((x)) log [(x) dr } s.t. J (x(0) ((())x π
max #Ell
px())
(19)
Introduce Lagrange multiplier À and define
C(T, / [s(x)<()) t ((x) log x(x) po()) dr > (f(tt) de
(20)
For any admissible direction h with f = O, the Gâteaux derivative is
d de c(T eh, A) c- / s(x) T( log
T(z) po(z) + 1) >) h(x) dr.
(21)
Stationarity for all h yields the Fuler- Lagrange condition
s( s(x) T log (x) + 1) A 0 -
log
x (x) s(x) - 1,
(22)
pa(z)
po(x)
hence
= exp A/T - 1) (f e dr)
n² >xxx)= = (ro))))p(()exp((st)/))
(23)
Strict concavity and uniqueness. Consider the feasible set P {TT 0 f 1}, which is convex. The functional
is linear, and the negative KL term is strictly concave:
YT. T2 E P, Va (0.1): -r/ [am + (1- (1 log amT + (1 a) dx
Po
- a Ti log Ti nu dr + (1 - aa) /
log 2a dx
(24)
where the strict Jensen inequality for & Élog applies unless Ha T2 Therefore J, is strictly concave over P, implying the stationary solution nt is the unique global maximizer in II. This completes the proof.
15

---

Published as a conference paper at ICLR 2026
C.2 PROOF OF THEOREM 2
Proof. Fix é > 0 and define the e-neighborhood of the deleted support
A = No. (supp(Paa)) { E X 3mn supp(Pade) s.t. (x 2a) S {e}-
(25)
Because (aa) > 0 for any 0 and Paol < po by Assumption 1, we have po(As) > 0. From
Theorem 1, xxx = 27-po
exp(s(x)/7) with Zr = Pne"/ hence po (xr)
P. := (A,) = JA, po (2( (2)) dx
elutscs, <(0)TT Po(As) 0,
(26)
Ix (o( dx
CauDac s(x)/T
since E [0, 1] and (Ac) > O. Let XI,..... Xx be i.i.d. draws from n². The event (X, Ac, Vj}
has probability (1 p.) Therefore
Pr Bisk: X, E Ao = -1-
(27)
By definition of As. for any such X, there exists x Dael with dx (X,, S E. Let the refine- ment operator be denoted R; assume it is L-Lipschitz and uses stepsize 1) 0 that respects the neighborhood, i.e.,
R((x)) Xyllx Sn and n n E dx Dael).
(28)
Then X, R(X,) E Ag and man dx (X,, x) x) S E. Hence
Pr Pr[jis k 2EDaal
min dx(X) y. x) S é) 1 (1-(1-p.)) p.)
(29)
Finally, to attain target confidence 1 - 6, suffices to choose claim.
2 llo((//))//)) This proves the
C.3 PROOF OF THEOREM 3
Proof. Let n' be the unique maximizer given by Theorem 1. Let Pr. = x Σラ=z 8x, be the empiri- cal distribution of the k reconstructed samples after the local refinement R.
Step 1:
Bias-variance decomposition for a general IPM. Let d(..-) be any integral probability metric (IPM) induced by function class F:
d(P,Q) = sup fEF
Eplf] Eelf]
(30)
By the triangle inequality,
d(Pr, Pdel) I d( d( T²) + (PPet)
(31)
Moreover, if the refinement R is I--Lipschitz and maps pre-selected candidates X, to X, R(X,), then standard stability for IPMs on bounded domains implies (for Lipchitz-bounded function class F)
d(PE,T*)) x*) s Cr. T"),
P
8
xx
(32)
for some constant Cr, 2 1 depending only on L and the diameter of X. Thus suffices to control d( ((Ps, T*) and d(x"" Paei).
Step 2: Sampling (variance) term d(P,,, T)). For common choices of d, we have non-asymptotic concentration with explicit rates:
MMD. F is the unit ball of an RKHS with bounded kernel and sup,, (xxx)) < K, then by standard RKHS concentration (c.g., via McDiarmid), for any d E (0, 1),
K 1og(2/0) k
Pr MMD(P T²) < 2
> 1-6 8.
(33)
16

---

Published as a conference paper at ICLR 2026
W1- If & C R is bounded with diameter D, the Fouinier-Guillin bound gives constants
Ca > 0 such that, for any E (0,1),
(108 ( 1/d
Pr Pr( Ca
2 1 6.
(34)
k
For general IPM with functions bounded by |f I B, symmetrization plus Hoeffding inequality yields the generic bound
2 log (2/0)
Pr d(Pp, xT) S 29 (F) + BV
21-8,
(35)
h
where (J) is the empirical Rademacher complexity (often O(L/vk))
Step 3: Bias term d((=*, Fdel) via exponential tilting. From Theorem a has density
(o( exp(s(x)/T)
xz)
Z, -z po(x) exp(s(x)/7) dr.
(36)
Zr
We first derive an upper bound for drv(#*, Paei) via an upper bound on (I(Pael|=*) and then apply Pinsker. By the chain rule for KL,
(TaePaa (Pac
Braa [s] + log ZT
(37)
Since E [0, 1], Hoeffding's lemma gives (for any base measure)
Epp [s]
1
log Z, log Epo [exp( [)]
(38)
872
Plugging equation 38 into equation 37 yields the explicit upper bound
KL(Paa||>** (LPaall))))) + [Epnlss Spaci [s]
1
+
(39)
87
By Pinsker's inequality,
drv(a*, V2 (PPaelllTTT
(40)
Therefore
Ep [s]
E E
Eraml
(T"T, < (L(Paeillpo)
(41)
T
872
Relating separability to the bias. By Definition 2 (Trace separability). EF..[ss) 2 Ep... [s] + A.
Po is public prior independent of the unlcarning operation, does not increase with the separability margin; hence the difference Ep, [s] E [s] decreases as A grows, which tightens equation 41. Consequently, the total variation bias be controlled by (T, A) and (L(Fdal del (o))
treat Epu [s] constant that
Step 4: From TV to the chosen metric d(:.-). On bounded domain X' with diameter D,
W1(=*, I D drv(r (PTlP
(42)
For MMD with kernel & bounded by K,
MMD(AT Paa)
sup
E. [f] Era 15]
sup
||fllo 2 2drv(T .Peel)
< 2/K K d
(43)
using If S VKIlflm. More generally, for any IPM induced function class F with || S B,
d(r", Paat)
2B drv Pact).
(44)
17

---

Published as a conference paper at ICLR 2026
Combine with equation 41 to obtain an explicit bias bound:
d( Paell Cuet
V [IL(PallPPo)
Byo [s] Braa 8r2
(45)
D
d W1,
where Cmnt
2VK,
d= MMD,
2B,
general d.
Define Ci (T, A) as the right-hand side; decreases as + and the separability margin A increases (since [s] increases with A).
Step 5: Put together. With probability at least we have
d(Pkk x²) S e(ki,) 8),
(46)
where e(k, is the sampling error of Step 2 (e.g.. 2 log(2/6)/k for MMD, or Ca(log(2/8)/k))/) for Wi). Hence, by equation 31 and the stability of R,
d(Pr,Fae)) Pact) I CL (k) + CI(r,),,
(47)
which proves the theorem.
D EVALUATION
D.1 DETAILED EXPERIMENTAL SETUP
Unlearning method. Following existing work (Hu et al., 2024), to obtain the paired models (ft. we adopt two categories of unleariing procedures. Exact unlearning is implemented by removing the forgotten samples and fine-tuning the model on the remaining data for the same number of epochs as the original training. Approximate unlearning implemented using the single gradient unlearning method (Thudi et al., 2022). In both cases, unlearning is performed in class- wise manner, which reflects realistic scenarios where requests often target all samples belonging to specific semantic category. We set class 0 the default target for unlearning.
Datasets. We evaluate on CIFAR- 100 with 50k training images and 10k test images of size 32 x 32 RGB. This dataset contains 100 object categories with relatively low resolution and high intra-class variability, making it a standard benchmark for image classification. Food-101 includes 75k training images and 25k test images of size 224 x 224 RGB. It covers 101 food categories collected from real-world scenarios, characterized by large intra-class diversity, occlusion, and noisy labels. PathM- NIST, subset of the MedMNIST collection, consists of 89,996 training images, 7,180 validation images, and 7,180 test images of size 28 x 28 grayscale. It provides classes derived from col- orectal cancer histology slides, capturing diverse tissue types such adipose, lymphocytes, smooth muscle, and adenocarcinoma epithelium.
Models (ft.f We adopt ResNet-18 18 on CIFAR-100, Food-101, and PathMNIST datasets for image classification tasks. We use cross-entropy loss with the SGD optimizer and cosine learning rate schedule, and maintain a checkpoint pair ( for each dataset.
Generator (RL Policy aa). We use DCGAN-style generator G that maps a latent vector 2 Rd to 32x32 images with tanh outputs in [],, and an RL policy as that produces via diagonal Gaussian whose mean and log-std are predicted by two-layer MLP. each step, the policy samples = (e) ) + exp(log (e))
) @ with ~ (N, 7), generates Ftake G(z), and receives reward built from unlearning traces between f=annd and f- The policy optimized with PPO (clipped objective with entropy bonus), while G can be jointly fine-tuned by surrogate loss that maximizes the trace signal and includes leacher-guided KL distillation from f+ and f- and total-variation regularization. In practice, we initialize G with different pretrained GANs depending on the dataset: an ImageNet- prettraiied GAN for CIFAR-10 100 and Food-101, and a MedMNIST. pretrained GAN for PathMNIST.
RETRACE settings. We set a 0.4, 0.2, and 7 = 0.4 as the relative weights for trace components. For reconstruction, we select samples using top-k with k = 64.
18

---

Published as a conference paper at ICLR 2026
1.0
Normalized trace Score
1.0
Normalize Mommlllzed trace Score
1.
Normalized trace Score
0.8
0.8
8
0
3
2
(a) Original Data
(b) Black-box
(c) Grey-box
(d) White-box
Figure 7: Traces of unlearned data under different model access levels on Food-101 in the approx- imate unlearning scenario. The images in red boxes represent the unlearned data "Apple pie".
1.0
Normalized trace Score
1.0
Normalizedd trace Score
0.8
0.0
0.8
0.6
0.6
0.6
Normalized
0.4
(a) Original Data
(b) Black-box
(c) Grey-box
(d) White-box
Figure 8: The normed version of Figure 7.
Evaluation metrics. For instance-level evaluation, we use Mean Squared Error (MSE) to assess the pixel-level difference between two images; cosine similarity (CS) to evaluate the feature-level difference; success rate (SR) to evaluate the percentage of generated images being unlearned class. For distribution- evaluation, statistical similarity between reconstructed and original distributions, and Kullback-Leibler (KL) divergence assess the alignment of their probability distributions.
Fréchet Inception Distance (FID) evaluate the visual and
Baseline methods. We compare RETRACE with two state-of-thh-art reconstruction attack methods, which are listed as follows.
Unlearning Inversion Attack (UTA) (Hu et al., 2024), which is conducted to recover unlearned data in white-box access.
HRec (Bertran al., 2024), which achieves nearly-perfect attack on linear regression and can be generalized to other model architectures.
D.2 TRACE VISUALIZATION
For both food-101 and PathMNIST, we present the trace distributions (Figures 7, 9, 10, 12).
ff trace heatmaps and the corresponding
D.3 EFFECTIVENESS RETRACE
Instance level. Table presents the instance-level reconstruction results RETRACE across three datasets and three model-access levels. For the criteria of MSE and CS, we compute all pairwise distances within the unlearned class in the original dataset and take the average, which reflects the natural intra-class variability (values are presented in tablenote).
Under the exact unlearning setting, across three datasets, the reconstructed samples achieve MSE values that are only slightly higher than the intra-class criteria (e.g., CIFAR-100: 0.20 vs. 0.16; Food- 101: 0.23 vs. 0.13), showing that the pixel-level differences between reconstructions and unlearned images remain small. The SR values consistently around 50-60%, indicating that method can successfully generate large proportion of samples that are classified as belonging to the unlearned class. Meanwhile, the CS values (e.g., CIFAR- 100: 0.47 0.57) demonstrate that the reconstructed
19

---

Published as a conference paper at ICLR 2026
Tlace youve (mean))
0.20
0.35
0.4
8.05
0.05
0.00
20
40
60
80
100
uw
100
20
80
100
Cass label
Class label
Class label
(a) Black-box
(b) Grey-box
(c) White-box
Figure 9: Trace distribution under different model access levels on Food-101 in the exact unlearning scenario. Class represents "Apple pie".
Normalized trace score
Normalized trace Score
0.6
0.6
Normalizee pell
0.4
0.4
04
0.2
0.2
0.0
(a) Original Data
(b) Trace under Black-box
(c) Trace under Grey-box
(d) Trace under White-box
Figure 10: Traces unlearned data under different model access levels on PathMNIST in
approx- imate unlearning scenario. The images in red boxes represent the unlearned data "Adi- pose".
samples lie close to the original unlearned data in feature space, confirming that the recovered images retain strong semantic similarity.
In the approximate unlearning scenario, the performance further improves across datasets and access levels. MSE decreases noticeably compared exact unlearning (e.g., CIFAR-100 0.17 vs. 0.20), suggesting that approximate unlearning leaves stronger residual signals for pixel-level recovery. SR also increases significantly, with white-box access achieving the highest rates (e.g., CIFAR- 100: 73.1%; PathMNIST: 59.7%), demonstrating that our method can generate even larger number of valid samples for the forgotten class. Finally, CS values are consistently close to the criteria (c.g., Food-101: 0.49 0.54), indicating that reconstructed images not only recover visual details but also align well with the semantic representations of unlearned data.
Distribution level. Figure illustrates the distribution-level reconstruction results of RETRACE in terms of FID and KL. divergence.
Under exact unlearning, across all three datasets, RETRACE achieves meaningful reconstruction performance under all access levels. In the black-box setting, both FID and KL remain at reasonably low values (e.g., CIFAR-100: FID 142.1, KI. 3.62; PathMNIST: FID 125.4, KI. 3.33), showing that the method can approximate the deleted distribution even with limited information. Moving the grey-box setting, the incorporation of loss-level traces consistently reduces both FID and KL (c.g., CIFAR-100: FID drops from 142.1 to 135.0, KL from 3.62 to 2.55), demonstrating improved alignment with the original distribution. The best results are obtained in the white-box setting, where gradient-level traces further enhance reconstruction quality, yielding the lowest FID and KL across datasets (e.g., CIFAR- FID 108.9, KL 2.49; PathMNIST: FID 96.5, KL 2.22). These results highlight that while RETRACE is effective even under restrictive black-box conditions, more informative access significantly boosts distribution-level recovery.
In the approximate unlearning setting, the results further improved. Compared to exact unlearning, FID values are consistently lower (c.g., CIFAR- 100: 99.1 vs. 108.9; PathMNIST: 93.4 vs. 96.5), re- flecting that approximate unlearning leaves stronger distributional traces that RETRACE exploit. KL divergence also decreases slightly in most cases (e.g., PathMNIST: 2.00 vs. 2.22), confirming that the reconstructed samples align closely with the underlying class distribution.
Cases. We present reconstructed samples at the instance level for CIFAR-100, Food-101, and PathMNIST under the white-box setting.
20

---

Published as a conference paper at ICLR 2026
1.0
Normalized trace score
1.0
Norallzedd Selllzedd trace Score
1.
Normalized trace Scora
0.8
0.8
0.8
06
8
2
(a) Original Data
(b) Trace under Black-box
(c) Trace under Grey-box
(d) Trace under White-box
Figure 11: The normed version of Figure 10.
Tace Tace sooeee emeaat
3.70
0.30
84
805-
0.10
0.15
9.2
Class labe
Class label
Cass label
(a) Black-box
(b) Grey-box
(c) White-box
Figure 12: Trace distribution under different model access levels on PathMNIST in the exact un- learning scenario. Class 0 represents "Adipose".
For CIFAR-100, we present reconstructed samples under Exact Unlearning (Figure 13).
For Food-101, we present reconstructed samples under Exact Unlearning (Figure 15) and under Approximate Unlearning (Figure 14).
For PathMNIST, we present reconstructed samples under Exact Unlearning (Figure 17) and under Approximate Unlearning (Figure 16).
D.4 DISTRIBCTION-LEVEL RECONSTRUCTION RESULTS OF BASELINES
To make the results more convincing on evaluating RETRACE's effectiveness, we further plot the distribution-leve reconstruction ability of two baseline methods. Table 6 provides the results regard- ing FID and KL values of them. Compared with their performance, RETRACE performs (Figure 6) much better than both UIA and HRee. Across all three datasets, and under both exact and approx- imate unlcarning. RETRACE consistently achieves the lowest FID and KL divergence, outperform- ing UIA and HRec. While the baselines yield relatively large FID scores (typically above 200 on CIFAR-100 and Food-101, and above 140 on PathMNIST) and KL values around 4-6, RETRACE substantially reduces both metrics across all threat models. shown in Figure 6, our method achieves markedly lower FID (e.g., 99-142 CIFAR-100 and 93-125 on PathMNIST) and KL (approximately 2-4) Black-box Grey-box, and White-box settings. This consistent improvement demonstrates that RETRACE reconstructs samples distribution-level that align much more closely with the deleted data distribution than existing baselines.
D.5 RETRACE MULTI-RUN RESULTS WITH STANDARD DEVIATION
We present the RETRACE of multi-run results with standard deviation in Table 7. Our main eval- uation spans three datasets of different scales and domains, and across all of them, the white-box setting consistently achieves reliably stronger reconstruction performance. This consistency across heterogeneous tasks indicates that the observed improvements are not accidental fluctuations, but reflect the true behavior of the method under increased model access level. In our evaluation, we ran 5 independent experiments, so the results presented in Table 1 reflect the mean of the 5 runs. Thus, we further provide the standard deviation across these runs to confirm that the observed gains not due randomness but remain statistically robust. The results confirm that the improvements in the white-box setting persist under multi-run evaluation. Therefore, the white-box box advantage is statistically stable rather than due fluctuations.
21

---

Published as a conference paper at ICLR 2026
(a) Instance-
level.
(b) Distribution-level.
Figure 13: Generated unlearned samples ("Apple class") in CIFAR-100 under exact unlearning.
(a) Instance-
level.
(b) Distribution-level.
Figure 14: Generated unlearned samples ("Apple pie class") Food-101 under approximate un- learning.
D.6 RETRACE UNDER ADDITIONAL UNLEARNING METHODS
We provide the RETRACE result under additional three unlearning methods in Table 8.
D.7 ABLATION UNLEARNED CLASSES
We present group of examples reconstructed samples for multiple CIFAR-100 classes under Exacl and Approximate unlearning, as shown in Figure 18.
D.8 SCALABILITY ON RESNET-34
We evaluate the scalability of RETRACE by scaling the neural network up to Resnet-34 on CI- FARI00 dataset. The results are presented in Figure 19.
D.9 RETRACE'S PERFORMANCE ON UNSEEN CLASSES
Table 9 lists the performance of RETRACE reconstructing unseen classes.
D.10 GENERALIZATION ON TEXT DATASET
Setup. We employ pretrained GPT-2 (small) decoder as the text generator and fine-tune it with lightweight PPO/REINFORCE objective, while the DistilBERT classifiers (Sanh et al., 2019) serve unlearned model: ft trained on the full AG News corpus and f retrained after Exact Unlearning of the unlearned class. At reconstruction time, we query f- and f on sampled texts to construct trace reward whose components depend the access setting: Black-box uses the prediction di- vergence 6pred (z) = |softmax(f (x)) softmax(f (x)) Gray-box augments this with a loss gap dloss(2) = (e(f+()),))))))))))))()((()))) 0) - ((f((())) (x), ú)] where ÿ = arg max (x); White-box additionally incorporates representation discrepancy drea (x) 1 c)s())(( (a:), h- h=(x)) from hidden states. The total reward is the weighted sum of the trace term, class-prior from f+ (target-class probabil- ity), a discriminative term that increases the target-class logit, a small fluency bonus from the policy log-likelihood, and length penalty. We decode with top-k sampling, repetition penalty, and a min- imum new-token budget to mitigate collapse. We evaluate reconstruction effectiveness with BLEU, computed corpus-level n- n-gram overlap against held-out texts from the forgotten class, and MMD, computed between generated and forgotten-clast distributions in the DistilBERT [CLS] embedding space using multi-bandwidh RBF kernel.
22

---

Published as a conference paper at ICLR 2026
(a) Instance-
level.
(b) Distribution-level.
Figure 15: Generated unleamed samples ("Apple pie class") in Food- 101 under exact unlearning.
(a) Instance-
level.
(b) Distribution-leee.
Figure 16: Generated unlearned samples ("Adipose class") in PathMNIST under approximate un- learning.
Reconstruction examples on the text dataset. The representative reconstructions AG News (forgotten class: Sports) are shown below. These examples illustrate the recovered sports style.
Sample Inter lift Coppa Italia after 1-0 final; Martínez scores decisive header. 7 in Game 5. Sample France edge Spain 1-0 to lift UEFA Nations League crown in Milan.
Sample 1: Celtics survive Heat 104 99; Tatum posts 34 9-7
D.11 DISCUSSION OF POTENTIAL LIMITATIONS
Our method builds upon pretrained generative models, which serve the base model for recon- structing the unlearned content. This design alleviates the challenges of training from scratch, such unstable optimization and mode collapse, and enables more efficient adaptation with RL signals. Nevertheless, the characteristics of the pretrained model itself naturally influence the reconstruc- tion quality. particular, the alignment between the pretraining corpus and the target task domain plays critical role: if the pretraining data diverges significantly from the forgotten distribution, the generated samples may deviate from the intended semantics or result in unsatisfactory outputs.
However, in modern machine learning practice, the use of pretrained models has become standard. Large-scale pretraining not only reduces computational overhead but also greatly improves practi- cality compared to training from scratch. Therefore, this potential limitation, which could otherwise affect RETRACE'S performance, is unlikely to pose a significant concern.
23

---

Published as a conference paper at ICLR 2026
(a) Instance- level.
(b) Distribution-levl..
Figure 17: Generated unlearned samples ("Adipose class") in PathMNIST under exact unlearning.
Table 6: FID and KL divergence comparison under baselines.
Exact Unlearning
Approximate Unlearning
Dataset
Method
FID
KL
FID
KL
UIA
203.6
5.2
189.7
CIFAR-100
Food-101
HRce
5.3
226.6
UTA
156.4
4.1
143.4
PathMNIST
HRec
148.2
4.3
139.4
4.0
Table 7: Mean and standard deviation of 5 independent runs.
Exact Unlearning
Dataset
Black-box
Grey-box
White-box
CS
CIFAR-100
0.23:::.05
52.3%=00.4%
0.43:0004
0.22
50.5%
0.46=0.03
0.20=0.02
62.4%%%0.6%
0.47=0.02
Food-101
0.25
10.5%
0.38.10.04
0.23:10.03
54.7% 10.4%
0.42.10.02
0.202
50.4% 10.6%
0.44100.2
PathMNIST
0.26:1:.04
0.5%
0.26:::003
0.23:::.02
50.6% 20.4%
0.27::0.2
0.22
50.5% 0.31
Approximate Unlearning
Dataset
Black box
Grey hox
White
MSF
SR
CS
MSF
SR
CS
MSF
CIFAR-100
0.24:10.04
55.9%%:0..6%
0.46:10.03
0.21:10.03
68.7%
0.49:10.03
0.17:00.22
10.1%
0.5010.02
Food-101
0.24::0.3
56.6%+0.5%
0.41::0.33
0.18+:002
60.5%
0.49=:0.02
0.16#0.01
66.3%
0.49:001
PathMNIST
0.24:10.03
1..5% 0.28110.22
0.20:10:02
10.5% 0.28:10.02
1..5%
0.33-0001
Table RETRACE under additional unlearning methods.
Black-box
Grey-box
White-hox
Method
MSE
SR
MSE
CS
Sparsity -aased----dddddd Unlearning
0.21
53.1%
0.46
0.20
63.8%
0.47
0.18
70.9%
0.49
Boundary Unlearming
0,20
53.5%
0.47
0.19
65.2%
0.49
0.17
71.3%
0.51
Convolution- conoouttioo-Tanspoeee Unlearning
0.22
53.4%
0.46
0.21
63.2%
0.49
0.18
73.4%
0.50
Exact
Approximate
Exact
Approximate
Exact
Approximate
(a) Class 1: Aquarium fish.
(b) Class S: Bed.
(c) Class 12: Bridge.
Figure 18;
Reconstructed unlearned samples on class 1, and 12 on CIFAR-100.
24

---

Published as a conference paper at ICLR 2026
M45
0.25
0:21
0.19
0.15
5
100
60.2
68.3
74.9
0.5
0.48
0.50
0.52
0.0
Black
Grey bax
Write box
Black- Black-box
Grey box
White
Black
Grey box
White
Figure 19: ResNet-34
The performance of RETRACE CIFAR 100 when the target model is scaled up
Table 9: RETRACE'S performance on unseen class.
Method
BB
0.33
42. 12.1%
0.32
203.4
5.3
Exact
0.30
43.7%
0.36
WB
0.31
45.4%
0.38
183.8
4.7
BB
0.27
$1.3%
0.42
182.1
4.2
Approx.
0.27
58.5%
0.43
169.7
WB
0.24
59.6%
0.45
158.9
25


Reviews:
[{"review_id": "Y8kfUwudoA", "reviewer": "Reviewer_FHN1", "summary": "The paper proposes RETRACE, a reconstruction attack on machine unlearning that frames recovery of deleted data as a reinforcement-learning (RL) problem. The method extracts “unlearning traces” by contrasting a pre-unlearning model  with a post-unlearning model at prediction, loss, and gradient levels, and uses these as rewards to train a generator. The theory claims the RL objective converges to an exponential-tilted policy that emphasizes high-trace regions; experiments report strong instance-level recovery and improved distributional alignment (lower FID/KL) versus UIA and HRec, plus preliminary text results with DistilBERT.", "questions": "1. In the setting of the problem, it is assumed that the adversary has access to the auxiliary public dataset $D_{\\mathrm{pub}}$. However, it is not clear from the paper what the initial $z$ in the experiment is? It has to be clarified what initial $z$ is chosen in the optimization (the initial state). Could the authors please elaborate on the details (with specific examples on the dataset and forgotten sample ideally)?\n2. Have you tested your method on a class that DCGAN has not been trained on? What would happen in that case?\n3. Could the others provide some plots on the number of iterations used in PPO and how the metrics they use (e.g., MSE) changes along this optimization? \n4. Could the authors provide normalized values in figure 2 to show-case the improvement due to more information in the white-box setting. I would suggest computing the average value for all the 25 images in the patch and then reporting the ratio of the values for the red squares over the computed average value for the corresponding patch.\n5. To maximize equation 7, as mentioned in line 214, you need to maximize the reward given in equation 6. For that you need to minimize the differences (due to negative signs). But this would mean samples that the retrained model and original model would act similar on (which would be the retained data) and for example for the retrained data, we would expect equation 2,3, or 4 achieve the smallest values. So why the model should converge to the forget samples. Could the authors please address this confusion I had when reading the approach?\n6. In line 836, you mention exact unlearning method is implemented by “fine-tuning the model on the remaining data for the same number of epochs as the original training”. Why not training the model from scratch instead of fine-tuning the model on the remaining data. In practice the exact unlearning models are derived by fine-tuning the model from scratch because the fine-tuned model still contains information about the forget data and is not equivalent to the retrained model. \n7. Currently the authors only evaluate the effectiveness of their method that rely on SGD update (either gradient descent on remaining samples or gradient ascent on the forget samples). It would be interesting to see the effectiveness of the attack on the two following settings that are shown to be more successful than GA:\n    - Using sparsification methods that only perform SGD updates on a subset of the parameters [1,2].\n    - Unlearning methods that do not rely on SGD on either of the remaining sets and forget sets [3,4].\n\n[1] Jia, J., Liu, J., Ram, P., Yao, Y., Liu, G., Liu, Y., ... & Liu, S. (2023). Model sparsity can simplify machine unlearning. Advances in Neural Information Processing Systems, 36, 51584-51605.\n[2] Fan, C., Liu, J., Zhang, Y., Wong, E., Wei, D., & Liu, S. (2023, October). SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation. In The Twelfth International Conference on Learning Representations.\n[3] Chen, M., Gao, W., Liu, G., Peng, K., & Wang, C. (2023). Boundary unlearning: Rapid forgetting of deep networks via shifting the decision boundary. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7766-7775).\n[4] Ebrahimpour-Boroojeny, A., Sundaram, H., & Chandrasekaran, V. Not All Wrong is Bad: Using Adversarial Examples for Unlearning. In Forty-second International Conference on Machine Learning.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper proposes a new solution based on RL to address a newly proposed threat model against machine unlearning. \n2. The paper considered both instance-level machine unlearning and class-unlearning setting in their approach.\n3. Authors consider three levels of access to the original model and retrained model (black-box, grey-box, and whitebox), and perform most their evaluation on the three settings.", "weaknesses": "1. Some notations are used without proper definitions and clarifications. For example in line 141, $x_{\\pi_{(j)}} is not defined.\n2. Some of the details of the proposed approach are missing, which makes it confusing for the reader. What is the initial $z$ that you use in PPO? The details are missing while I think it might affect the outcome. I think the paper would benefit more from a pseudocode instead of figure 1 that does not provide any useful information.\n3. From assumptions 1 and 2 of the paper, it seems authors assume access to the population data of the forget class, which is a more restrictive assumption of access to the population data of the original training set as in prior work [1].\n4. In general, while this paper builds on an earlier paper for its setting [1], I don’t believe the setting of the attack specifically is specifically relevant to unlearning, but it is more in general about getting two models, one of which is trained on a subset of the training data used for the other, and try to reconstruct that data. So I think it is basically a privacy attack using model differentiation.  That is also why in [1], they only use the retrained model for their evaluations. I believe the assumption about gaining access to the original model in the setting of unlearning is not realistic.\n5. Although the link for the code has been added and reviewers are referred to in the reproducibility statement, the repository was empty and only the score computations method (equation 5) was provided.\n6. Your generative model relies on a DCGAN for generating the data that is missing. However, the pretrained DCGAN data is already trained on the unlearned classes and is capable of generating samples from the unlearned class. Therefore, it wouldn’t be surprising if DCGAN is capable of generating samples whose MSE from the original forget sample fall within the same range as the variance of the samples of that class. Basically, if I want to rephrase what you point out about the MSE value it would be sth like: you have a sample from the ‘Bed’ class that has been removed from the training set. Then you use your method to generate the image of a sample that can be considered a ‘Bed’, but not necessarily the same sample that was unlearned. Given that DCGAN is able to generate images of ‘Bed’ achieving that would not be very surprising. I think some of the confusion about this could be resolved once you respond to weakness 2, specially if based on assumption 1 of the paper, the adversary starts with a prior for the unlearned class.\n7. I think the provided figures are not very informative as they are now. For example, looking at figure 8, I don’t see any significance on the values of the selected squares. Even in figure 2 the advantage of white-box over black-box is not clear from the figure. In white box all the values seem to be larger, not only the forgotten samples.\n8. The results in the table are not accompanied by standard deviations. For example in table 1 the difference in CS score or MSE score for white-box vs grey-box is only at most 0.02, which might simply be smaller than the variance of the data.\n9. Your theoretical results rely on the assumption that the expectation of the score assigned to the samples from the forgotten samples are strictly larger than this expectation for the retained data. However, the score that you defined in equations 2,3, and 4 do not seem to necessarily follow this assumption. For example, if you train a model on the retained data and train a model on the whole data, the loss of these models, should be more similar on the retrained data compared to the forget samples. I think this assumption should be at least accompanied by some empirical observations.\n\n[1] Bertran, M., Tang, S., Kearns, M., Morgenstern, J. H., Roth, A., & Wu, S. Z. (2024). Reconstruction attacks on machine unlearning: Simple models are vulnerable. Advances in Neural Information Processing Systems, 37, 104995-105016.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Wq26ewfDGv", "reviewer": "Reviewer_gE9i", "summary": "The paper presents an attack on machine unlearning using RL-based data reconstruction. The work uses residual learnings as rewards and is comprehensively evaluated on several data samples using blackbox access on standard benchmark datasets. The findings indicate the feasibility of performing such attacks on large scale datasets.", "questions": "The paper presents a good study and evaluation of hidden traces and connections leading to success in attacks against unlearning models. The work is well presented, though some of the popular models and approaches for unlearning have not been explored. I wonder if the comparison can benefit from the larger set of models and methods studied in \"Deep Unlearn: Benchmarking Machine Unlearning for Image Classification\" in EuroS&P'25. \n\nWhat defenses can be used to mitigate the attacks mentioned in the paper?\n\nIt would be great to discuss the complexities of the attack and if it will be realistic to carry out against large datasets and bigger models.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- important problem and timely issue\n- well-carried out attack framework without much prior assumptions\n- good comparison on several benchmarks and good theoretical foundation", "weaknesses": "- comparison is limited to a few models and datasets and can easily be expanded broader\n-  Convolution Transpose,instance-wise unlearning,  or Masked Small Gradients methods could have also been studied\n- would have been great to mention computational costs and complexities", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Kh3cpyeK89", "reviewer": "Reviewer_MuvD", "summary": "This paper presents ReTrace, a new reconstruction attack that reinterprets the problem of recovering unlearned data through the lens of reinforcement learning (RL). Instead of relying on static optimization or inversion techniques, ReTrace uses discrepancies between the original and unlearned models as reward signals to guide a pretrained generator toward regions of the input space that likely correspond to forgotten data. The approach integrates multiple trace signals—changes in predictions, losses, and gradients—across different access levels (black-, grey-, and white-box) and performs reconstruction through RL-guided latent exploration followed by a candidate refinement step. Through both theoretical analysis and experiments on multiple datasets, the paper shows that ReTrace can recover semantically meaningful data at both the instance and distribution levels, highlighting residual vulnerabilities in existing unlearning methods.", "questions": "I would appreciate it if you could also answer my questions: \n\n**Q1**. I might have missed it in the paper, but I don’t fully understand— as also mentioned in my weakness section—whether the optimization in Equation (7) aims to maximize the trace score 𝑠(𝑥) or the reward 𝑟(𝑥) and what the motivation is for defining both and how they are related.\n\n**Q2**. My understanding is that, according to Equation (9), the instance-level reconstruction step selects a single top-scoring candidate via arg max 𝑠(𝑥) and refines that sample. If that is correct, could the authors clarify how the multiple instances shown in Figure 4(a) are reconstructed? A related question: did the refinement step lead to a significant improvement in reconstruction quality?\n\n**Q3**. I noticed that DCGAN is used as the generative model throughout the experiments. Could the authors elaborate on the reasoning behind this choice? Given the many stronger generative models introduced in recent years (e.g., StyleGAN, diffusion models), wouldn’t using a more advanced generator potentially improve reconstruction quality?\n\n**Q4**. In the appendix, it’s mentioned that the DCGAN produces 32 × 32 images. If I understood correctly, some of your evaluation datasets (e.g., Food-101) are higher-resolution. Could you clarify how this resolution mismatch is handled? In particular, how are comparisons with baselines such as UIA and HRec made—do these methods also operate at 32 × 32 resolution, or were their outputs downsampled to ensure fairness?\n\nAs I mentioned before, I like the core idea and find it creative and promising. However, I would need the authors to clarify the points raised in the weakness section—particularly by adding distribution-level comparisons against baselines—and address the questions above before I could confidently give this paper a clear accept.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "- **Novel and creative formulation**. This paper presents a genuinely original framing of reconstruction attacks. It adapts the RL-GAN-Net idea from Sarmad et al. (2019)—originally proposed for conditional image generation—and re-purposes it for unlearning data recovery. Using reinforcement learning to guide a generator’s latent exploration based on unlearning traces is both conceptually interesting and technically innovative.\n\n- **General and modular framework**. The approach unifies different attack settings (black-, grey-, and white-box) under a single reward formulation, where prediction, loss, and gradient discrepancies can be combined or omitted depending on model access. This modularity makes the framework broadly applicable and easy to adapt to new unlearning scenarios.\n\n- **Empirical evidence of residual traces after unlearning**. The experiments clearly demonstrate that models subjected to unlearning still leak identifiable information, confirming the practical relevance of the attack.\n\n- **Largest-scale model targeted to date**. The paper extends reconstruction attacks beyond CNNs to transformer architectures, reporting results on DistilBERT, which the authors state is the largest model attacked to date in the unlearning literature. This demonstrates the framework’s scalability and suggests that the proposed RL-based formulation might have the potential to generalize to high-dimensional transformer settings.\n\n- **Timely and relevant contribution**. The work addresses the emerging area of machine-unlearning security—a topic of growing importance for model safety, privacy, and regulatory compliance—and provides a concrete framework for analyzing these vulnerabilities.\n\n- **Theoretical grounding supporting intuition**. The paper includes a concise theoretical analysis that explains why the proposed RL formulation works, strengthening the intuition behind the method.", "weaknesses": "I like this paper — it’s a creative and well-motivated idea. That said, there are a few points I would like to raise and hopefully discuss with the authors.\n\n- **On the ambiguity in the RL formulation**. Section 3.2 introduces the RL framing with definitions of state, action, and policy, but these descriptions are somewhat abstract and internally inconsistent when mapped to the actual image-generation setup. The text defines the action as “sampling or refining a candidate x from the generator,” implying that the policy acts in the data space, while simultaneously describing the policy πϕ as “outputting candidates from the latent space,” implying it acts over z. This leaves it unclear whether the RL agent’s action space is x or z. Appendix D.1 later reveals that πϕ is in fact a small two-layer MLP producing latent vectors z that are then passed through a pretrained DCGAN G to obtain x = G(z). This architectural detail is crucial for understanding the proposed RL loop but is only specified in the appendix under the Experimental Setup Section. I would strongly recommend that the authors move this clarification into the main text so readers can immediately understand what components are being optimized and how gradients flow.\n\n- **On the mathematical clarity and internal consistency**. While the paper’s theoretical framing is interesting, I found the mathematical presentation scattered and internally inconsistent. Symbols are introduced but never used or formally defined. For example, T(x) is defined once (Eq. 5) and never referenced again. The trace score s(x), which seems to be the central quantity guiding the policy updates, is described conceptually but never expressed mathematically. In addition, a reward r(x) is defined (Eq. 6); if I understood correctly, it corresponds to −s(x), but this relationship is never made explicit. In Eq. (7), it is also unclear what pϕ denotes—whether it is simply the policy distribution πϕ or a learned variant of the prior p0. The term Dpub is briefly defined as “a publicly available dataset with a similar distribution,” but its operational role remains vague. Is Dpub the same dataset used to pretrain the DCGAN generator, or is it only used for the KL regularization term? Clarifying this would help connect the regularized RL objective to the actual implementation.\n\n- **on distribution-level comparison with baselines**. While the paper reports FID and KL metrics for ReTrace across datasets and access levels, it does not provide corresponding values for baseline methods (e.g., UIA, HRec). Since FID and KL are the primary metrics used to evaluate distribution-level reconstruction quality, the lack of direct comparison makes it difficult to assess whether ReTrace actually improves over existing approaches in recovering the overall deleted-data distribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SLnTtsdWby", "reviewer": "Reviewer_tMmg", "summary": "This paper proposes ReTrace, which seeks to reconstruct unlcearned training samples assuming access to the model before and after unlearning. ReTrace uses a reinforcement learning to learn a sample that is likely to minimize the loss functions of the two models before and after unlearning.", "questions": "I found some results confusing: some trace score patterns in Figure 2 and Figure 6 are confusing as the trace score doesn't necessarily correlate with being unldearned or not —for example, (0,0) has a very low trace score despite being unlearned, while other points like (1,0) and (1,1) show high scores even though they are not unlearned. The bottom-right case (4,4) is inconsistent across black-, gray-, and white-box settings -- an explanation would be nice.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Understanding and objectively measuring the effectiveness of unlearning is an important problem. The paper presents a novel and interesting use of RL in this space. Overall, I think as a measurement method, there are some benefits to ReTrace as the results seem to suggest that exact unlearning is more effective than approximate unlearning, which is a nice, though somewhat expected result.", "weaknesses": "The paper claims to evaluate both ResNet and DistilBERT, but the majorit of results are in the image domain. Average attack success rates is only around 50% for exact unlearning and 60% for approximate unlearning, and worse on the text tasks, so the attack is not that  effective. As an actual attack, ReTrace doesn't seem that realistic: as it assumes that an adversary can access both pre- and post-unlearning models while running a costly RL process. The paper would have been stronger if the authors could have further compared the relative strenghts of different unlearning proposals using ReTrace and used their method to explain some of the differences.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)