Learning to Weight Parameters for Training Data Attribution
Authors: Shuangqi Li
Hieu Le
Jingyi Xu
Mathieu Salzmann
Venue: ICLR 2026 Poster (2026)
Keywords: Training Data attribution; Influence function
Primary area: interpretability and explainable AI
TL;DR: Attribution signals are noisy. Our method learns to re-weight layers to amplify the true signal, boosting accuracy and enabling fine-grained (e.g., subject vs. style) attribution.

Abstract: We study gradient-based data attribution, aiming to identify which training examples most influence a given output. Existing methods for this task either treat network parameters uniformly or rely on implicit weighting derived from Hessian approximations, which do not fully model functional heterogeneity of network parameters.
To address this, we propose a method to explicitly learn parameter importance weights directly from data, without requiring annotated labels.
Our approach improves attribution accuracy across diverse tasks, including image classification, language modeling, and diffusion, and enables fine-grained attribution for concepts like subject and style.

Published as a conference paper at ICLR 2026
LEARNING TO WEIGHT PARAMETERS FOR TRAINING DATA ATTRIBUTION
Shuangqi Li'
Hieu Le Jingyi Xu² Mathieu Salzmann'
EPPI., Switzerland 2Stony Brook University, USA 3NNC Charlotte, USA
(shuangqi 1i, mathieu. salzmann @epfl. jingyixu&cs st wtt.obooboooooooo. edu hle40@charlctte.ce
ABSTRACT
We study gradient-based data attribution, aiming to identify which training ex- amples most influence a given output. Existing methods for this task either treat network parameters uniformly rely implicit weighting derived from Hessian approximations, which do not fully model functional heterogeneity of network parameters. To address this, propose method to explicitly learn parameter importance weights directly from data, without requiring annotated labels. Our approach improves attribution accuracy across diverse tasks, including image clas- sification, language modeling, and diffusion, and enables fine-grained attribution for concepts like subject and style.
INTRODUCTION
Tracing how specific training examples influence the outputs of generative models-knen data attribution-is critical for transparency, copyright protection, and data governance. Early gradient- based methods (Yeh al.) 2018 Pruthi et al. (2020) offered practical solution by estimating influence through the direct similarity of parameter gradients. A more theoretically grounded approach was established by Influence Functions (Koh & Liang 2017), seminal technique that leverages the Hessian to better approximate how model parameters would change if training point were up- weighted. While powerful. computing the true Hessian is intractable in practice. This challenge has spurred variety of approximation strategies, including leveraging Arnoldi iteration to estimate Frrerse-Hessian-vedd
products (Schioppa et al. (2026)), using structured approximations of the Hessian such as EK-FAC (Grosse et al. 2023). or building an empirical kernel from projected gradients (Park et al. 2023).
We observe that attribution quality is not uniform across parameters, but instead varies systematically by group. Both theory and practice point to this heterogeneity. From theory, influence functions already suggest that different parameters should matter unequally: the Hessian inverse rescales gradients according to curvature of the loss function, amplifying some directions more than others. In practice, beyond such rescaling, common strategy is to restrict attribution subset of parameters, often to only a classification model's final layer (Koh & Liang, 2017 Pruthi et al. 2020) Yeh et al.) 2022), to the first up-sampling block in UNet (Lin et al. (024). This restriction effectively makes ad-hoc choices about which parameter groups contribute to attribution. In this paper, we will further demonstrate this heterogeneous attribution quality in Section 3| For example, we show that in diffusion models, up-block layers achieve higher Linear Datamodeling Scores (LDS) (Park et al. 2023). and self-attention layers often outperform cross-atee aessetttttttoon layers (Figure I.. These empirical findings are also consistent with common observations that different layers tend to encode different
aspects of the output (Ronneberger al. 2015 Tumanyan et al. 2023 Si et
2024).
At the same time, the exact conditions assumed by influence functions are rarely satisfied in large generative models: parameters are typically not at true optima, the Hessian intractable, and practical methods rely approximations such as EK-FAC combined with random projections. These approximations offer only indirect and potentially noisy estimates of parameter importance, limiting their reliability. Thus, while both theory and practice suggest that attribution quality varies across parameters, current methods only partially capture this effect, leaving room for approaches that address it more directly.

---

Published as a conference paper at ICLR 2026
LDS per Parameter Group
(b) Grouped by Block and Functionality
Figure 1: Attribution Strength Across Parameter Groups measured using the Linear Datamodeling Score (LDS) (Park et al. 2023). (a) The LDS computed using gradients from individual parameter groups shows significant variations in attribution strength. (b) Aggregating the LDS by block depth (e.g., "down_ Noww_blocks.00 representing the first downsampling block the UNet) and functionality within each block (e.g., "attnl" representing the self- self-attention layers, "attn2.to_ representing the query projection layers cross-attention modules) reveals significantly varied attribution strength for different network depths and functional components. Error bars are calculated over 1,000 samples.
To address this, we propose a data-driven method that learns parameter- group weights directly, explicitly aiming to improve attribution quality. Our approach bypasses noisy approximations, adapts across architectures and training regimes, and yields weights that are both effective and interpretable. We cast this into a unified framework that generalizes gradient-based attribution methods, and design self-supervised objective that learns weights without ground-truth attribution labels. By bootstrapping from the influence rankings produced by existing attribution methods, the method adapts to the heterogeneity of different architectures and training regimes. In doing so, it not only improves attribution accuracy across both image and text generation tasks, but also yields interpretable insights into how distinct parameter groups specialize in aspects of generation such as subject, style, or background.
In summary, our contributions are as follows:
We show that attribution quality varies systematically across parameter groups, consistent with both theory and practice, highlighting an opportunity more directly account for this heterogeneity.
We empirically demonstrate that attribution quality varies significantly across parameter groups in diffusion models, motivating the need for adaptive attribution strategies.
We propose unified framework with self-supervised objective that learns parameter-group weights directly from data.
Our approach improves attribution accuracy across vision and language tasks, generalizes across architectures and datasets, and provides interpretable semantic disentanglement (e.g.. subject vs. style).
2 BACKGROUND AND RELATED WORK
Data Attribution. The goal of data attribution is to trace given model output back to its influential training examples. Specifically, data attribution method Tax, D) function 7 X' N - R that, for any sample
X' and training set D of size N, assigns x" E D indicating its contribution to a pre-specified model output function, e.g., the loss function L(x; 0* (D)), where 6* (D) represents the model parameters optimized D.
to each training example
Various attribution methods have been developed. One line research consists of sampling- based methods, including empirical influence functions (Feldman & Zhang 2020), Shapley value estima- tors (Jia et al., 2019: Ghorbani Zou| 20199 Covert Lee| (0021). datamodels (Ilyas et al. (022). and unlearning-bast based methods (Wang al. 2024 Isonuma & Titov 2024). However, such methods
2

---

Published as a conference paper at ICLR 2026
require training numerous models and are often prohibitively costly for practical-sized models. In contrast, gradient- based methods that exploit the information in the gradient with respect to the model parameters computationally more favorable. In this context, TracIn (Charpiat et al. 2019; Pruthi et al. 2020) computes the similarity between the gradient from sample of interest and from the training examples; Influence Functions (Koh & Liang 2017: Schhoopa et al. 2022a MModozeniec et al. 2024) and TRAK (Park et al. 2023) go beyond a simple gradient agreement strategy but intro- duce computation-intensv matrix inversion terms; TRAK (Park et al.) 2023) projects the gradient into a lower- diveesiimensi space to alleviate this issue. In this work, we develop an approach able to improve gradient- braseent-based attribution methods.
Attribution in Diffusion Models and LLMs. Our experiments extend beyond classification to the frontiers of generative AI, where attribution critical. In image generation, the rise of powerful diffusion models (Ramesh et al. 2022 Saharia et al. 2022 Rombach et al.) (022) has created an urgent need for attribution to address ethical and privacy concerns (Carlini et al. 2023 (Somepalii et al. 2023 Birhane et al. 2021). This has led to specialized methods that adapt attribution to the unique properties of diffusion. Journey TRAK (Georgiev et al.] 2023) first extended TRAK to handling the multi-step denoising process of diffusion models. Subsequent work then focused on refining the core gradient signal itself: D-TRAK (Zheng et al., (023) systematically explored different model output functions for computing the loss gradient, while DAS (Lin al.) 2024) derived new output function specifically for the diffusion objective. A parallel challenge exists for Large Language Models (LLMs), whose immense scale demands highly efficient attribution. Grosse itt al. (2023) scaled influence functions to cellion-parrmeter models using an Eigenvalue K-FAC approximation of the Hessian. LoGRA (Choe et al. 2024) further improved efficiency with novel gradient projection strategy. Most recently. TrackStar (Chang et al. (024) integrated existing innovations, including gradient normalization and Hessian approximation, to achieve robust attribution at scale. Nevertheless, none of the above-mentioned methods explicitly model the fact that different network components connect the model's output and the training samples in different ways.
Parameter Importance Heterogeneity. The heterogeneous nature of parameter importance is considered mainly in the field of model pruning (Sun et al. 2023; Ma et al.) 2023 Liu et al. 2023 Jaiswal et al. 2023 Chen et al. 2020),, which aims to identify and remove redundant parameters to improve model inference efficiency. The parameter importance heterogeneity has also been considered in the field of machine unlearning, knowledge editing, and quantization, in the form of knowledge localization (Yu et al., 2023) Wu et al., [023] Wang & Veitch, (025) and mixed-preciste optimization (Kim | Cui Wang 2024).. To the best of our knowledge, our work the first to explicitly address parameter heterogeneity in data attribution.
3 PARAMETER HETEROGENEITY IN DATA ATTRIBUTION
In this section, we review the theoretical basis of gradient- based attribution methods and highlight their implicit assumptions about parameter importance. We then provide empirical evidence that attribution strength is heterogeneous across parameters in diffusion models.
3.1 IMPLICIT PARAMETER HETEROGENEITY PREVIOUS WORK
Gradient- based attribution methods typically compute the influence of training examples via gradient similarity. common simplification of TracIn (Charpiat et al. 2019) Pruthi et al. (0200) computes the dot product between the gradient at query sample (Fquery) and that at a training point (x"):
Tracla (Fquory: x") VeL(z 0)T veL(=";))
where VoL denotes the gradient of the loss function w.r.t. the model parameters O. This formulation inherently treats all parameters uniformly, assuming equal contribution to the attribution score.
However, this uniform weighting assumption overlooks the functional specialization within complex deep models. For example, in the UNet architecture, different components (e.g., attention vs. convolutional layers, shallow vs. deep layers) serve distinct roles. Deeper layers typically capture more holistic semantics, while shallower layers focus on style and texture (Tumanyan et al. 2023) Si ett al. 2024). This structural heterogeneity strongly suggests that gradients originating from different parameter locations carry diverse types and amounts of attribution information, making uniform weighting suboptimal.
3

---

Published as a conference paper at ICLR 2026
SCM
CAT
CHH
2:11
2:C4
1.21
1.43
I-CB
LXT
2C-2
0211
SQI
CAC
Jen
2011
LAM
LHI
6:40
1:4
A11
L
RAD
Kork
FPM
CNT
KTW
X17
(a) Subject
(b) Style
(c) Background
Figure 2: Parameter specialization by semantic element. Heatmaps show average recall @ 10 for attributing (a) subject, (b) style, and (c) background using gradients from different parameter groups (UNet block depth and attention components). Brighter indicates stronger attribution strength for that semantic element by that parameter group.
More sophisticated methods, such as the kerneiized approach TRAK (Park et al. 2023). move beyond simple dot product by incorporating kernel to mediate the similarity score:
FIRAK (Qquery: x²) )- (P Vo VoL( query 0)) IT,
T <(z') P VoL((x)))
P PovoL(o";
(2)
where +- [6(x'))
(3)
Here P is a random matrix that projects the gradients a lower-dimensional feature space, mak- ing the formulation a scalable analogue to Influence Functions (Koh & Liang] 2017). The term
(4 TT + AZ) preconditions gradient similarity using second-order structure estimated from the projected gradients. While theoretically motivated, these kernel-based methods face several practical limitations that affect their precision. First, the kernel is an approximation of the loss curvature. The formal derivation of influence functions assumes access to the true Hessian, which is intractable for deep models. Existing methods therefore rely on proxies such as the damped Gauss-Newton Hessian (as in TRAK) or the Kronecker-Factored approximation (Grosse et al. 2023), which do not perfectly capture the curvature. Second, the theory assumes that the model has fully converged, whereas the gradients are typically computed from checkpoints that are not at true local minimum. Finally, the gradient features suffer from the information loss incurred by random projection.
Consequently, the implicit re- weighting effect is derived from inexact and noisy signal. As such, the resulting implicit weighting may not accurately reflect the true importance of different parameter groups, motivating direct empirical analysis of the attribution signal's 's underlying structure.
3.2 ATTRIBUTION STRENGTH ACROSS DEPTH AND FUNCTIONAL COMPONENTS
To quantify the variation in attribution signal, we measure the attribution strength of different param- eter groups. We define attribution strength as the effectiveness of gradients from a parameter group in identifying influential training examples. We quantify attribution using the Linear Datamodeling Score (L.DS) (Park et al. 2023), which measures the correlation between a model's s sensitivity to training-data subsets and per-sample attribution scores (higher better; see Appendix B.2.I). To assess how attribution strength varies throughout the network, we fine-tuned a Stable Diffusion model on ArtBench-2 (Liao al. 2022) using LoRA (Hu al.) 2022) and performed attribution using D-TRAK (Zheng et al.) 2023). We partitioned the LoRA parameters into 128 consecutive groups by location within the UNet and computed LDS using only gradients from each group.
Figure a shows the LDS for individual parameter groups, revealing substantial variations in attribu- tion performance across the network. Some groups achieve high LDS, while others parttcclaarrlyyyyyy in shallower Mlocks-contribur minimally. Aggregating scores by structural and functional categories further clarifies these trends. Grouping by block depth (Figure [b] top) highlights dramatic differ- ences, with up_ sp__boocs... 1 and up_ blocss.ck.... 2 exhibiting much higher average LDS (5.58) than others. Grouping by functionality within each architecture block (Figure Tb bottom) reveals that attribution strength also varies significantly by component type; for instance, the output projection layers (at tnl. to_out, att to out) consistently show higher average LDS, indicating sig- nificantly greater attribution strength compared other attention components such moss-attention
4

---

Published as a conference paper at ICLR 2026
key/query/valuee projections (attn2. to k, attn2. to__, attn2. to__)..o This confirms that attribution strength varies depending on both parameter location and functional component. Further- more, show via cosine-similarity analysis in Appendix [.T] the pattern of per-group LDS scores and the learned weights is highly consistent across datasets and attribution methods, indicating that this heterogeneity reflects stable, intrinsic property of the model rather than an artifact of any specific setting.
3.3 ATTRIBUTION STRENGTH BY SEMANTIC ELEMENT
Beyond architectural location, we hypothesize that parameter groups also specialize in attributing distinct semantic elements (e.g., subject, style, background). To investigate this, we synthesized dataset of 600 images, evenly divided across three attribute types. For each attribute, we defined 10 categories (e.g., Bulbasaur for subject, blacklight painting for style, beach for background) and collected 20 unique images per category. We then fine-tuned a text-to-image Stable Diffusion model on this dataset and generated new images using prompts that combined these elements (e.g.. "A [style] of a [subject], [background]"). For each generated output, we treated the 20 training samples that matched its constituent subject, style, background the ground-truth contributors for that respective attribute. Accuracy was evaluated using recall @ 10-the fraction of the top-10 attributed training samples that matched the ground-truth set for given semantic element.
Figure 2 visualizes the results, confirming a clear semantic specialization. The heatmaps show distinct patterns of attribution strength for subject (a), style (b), and background (c). Different parameter groups exhibit peak attribution strength for different semantic elements; for example, style attribution tends to peak in earlier layers of the network, whereas background attribution is strongest in specific attention components. This provides strong evidence that different network components are indeed responsible for attributing distinct semantic aspects of generated image.
Taken together, our experiments reveal a clear conclusion: Attribution signal is highly heterogeneous, concentrated in architecturally and semantically distinct components the network. This underlying structure is critical source of information that the uniform or implicit weighting schemes of prior methods fail to exploit. This finding motivates a more direct strategy: Explicitly learning parameter importance from data, approach we introduce below.
4 LEARNING TO WEIGHT PARAMETERS FOR DATA ATTRIBUTION
4.1 SHAMMETERWEEIGHHE DATA ATTRIBUTION: FORMULATION
Let the model parameters 6 be partitioned into M disjoint groups, 0 = . 0br,, }. such as individual tensors or layers. For an imput z, let 9, (z) denote the gradient-derived feature vector for parameter group 0j. Attribution methods typically use the concatenated gradient features g(r) = (91 (z)..... ... 9M (a)) compare query example Equery with training example x..
We introduce a learnable, non-negative weight vector w: = {wi..... wM) ). where each wy scales the contribution of the corresponding parameter group. This approach is motivated by the insight that different parameter groups contribute attribution signals of varying quality. The reweighted query feature is thus
g(z; w) = [w]9 (e.()) WMgM(x)) (x)].
(4)
Let Diag(w) (w) denote the diagonal matrix that scales the coordinates of g(x) by the corresponding group weights, i.e., it repeats wy across all dimensions of yj (z). so that g(x;;) = Diag(w) g(r). Then, the wergheter-weighted attribution score between Eiuerry and xt is
7 (aquery _=; w) = g(Fiquer)) Diag(w) K 9(x"),
(5)
where the matrix K defines a similarity metrie. This formulation subsumes both simple and kernelized methods: for TracIn-style attribution, K is the identity matrix, which yields a weighted dot product (cf. Eq. I for kernelized methods such as TRAK and its variants, K is a kernel matrix, for example
the one built from projected features, { T Φ >/) -1 For the special case K = I, weighting either the query the training feature (or both) is equivalent up to reparameterization of w, since a global scale on one side can be absorbed into the weights. For general kernels, however, the training-side term K g(x²) is precomputed once for all training examples to make attribution scalable. Applying
5

---

Published as a conference paper at ICLR 2026
3.0
0.0
040
402
160
200
Down Blocss
Up Stock
Down Booda
Mc oit
Bidaa
022
0.4
312
0.0
A12
255 22
212
160
2.00
Figure 3: Learned Parameter Importance Weights for overall (top-left), subject (top-right), style (hottom-left), and background (bottom-right) attribution. Each plot shows the learned weights across 256 parameter groups in the UNet, organized by Down, Mid, and Up blocks, with different colors indicating each block type.
weights symmetrically would require recomputing this term every time w is updated, which would make weight learning prohibitively expensive. We therefore apply Diag (w) only to the query features and treat the training-side features K g(=") as fixed. Conceptually, wo specifies how much we trust each parameter group when reading the query's gradient signal.
This unified framework allows any gradient-based attribution method to benefit from learned parame- ter importance weights, improving both attribution performance and interpretability.
4.2 SELE-SUPERVISED WEIGHT LEARNING
Our goal is to learn weighting parameters that maximize data attribution quality. Obtaining ground-truth attribution scores for direct supervision generally infeasible. We therefore propose self-supervised approach that bootstraps from the rankings of an existing attribution method. The core assumption that the top-k scoring training examples, according to a base attribution method, can serve pseudo ground-truth positives. While the initial ranking is imperfect, it provides weak supervisory signal that the optimization can refine.
Formally, for query quuery and training set D, let (Z w) RN be the vector of all parameter- weighted attribution scores. Let Inop-K (w) C {I...., NN denote the set of indices corresponding to the top & scores in this vector. We optimize the weights by maximizing the average score of these sseudo-positiver normalized by the overall score magnitude. This is expressed as the loss function:
Lssl(w; Equary, D,k) (Iquery, Di Diw) w))l
1
Σ (EZepa(w)
F(Equoyy x' w)
(6)
During optimization, the set of top-k indices, Trop* (w), is re-evaluated with the updated weights at each step, allowing the model to bootstrap a progressively better signal.
As formally derive in Appendix A minimizing this loss serves principled proxy for maximiz- ing the SNR of the attribution score. Specifically, our weighted formulation (Eq. 5 is designed such that the numerator acts as estimate of the signal strength, while the denominator, the Eg norm, estimates the total noise level.
To learn single, generalizable weight vector w, minimize the expectation of this loss over distribution of query samples Q, adding a regularization term to prevent overfitting. This yields:
arg min Ex Csst (w; Equery, D, k:] A||w1/2-
w 220
In practice, we enforce weight non non-negatinn nongaviatyvy via a softmax parameterization. While performance depends on the hyperparameter k, find it to be stable across broad range of values (see

---

Published as a conference paper at ICLR 2026
Table LDS (%) on ImageNet classification. Our method significantly improves attribution for both CNNs and Trans- formers. We report LDS + 95% CI half-width.
Table 2: LDS (% on 1ikFText-103 with GPT-2-small.
Method
w/o
R8
ViT-B/16
TracIn
6.31 1:0.88
9.23 1:096
Method
w/o
w/o
TRAK
12.69 1:100
14.63 1:004
TracIn
11.39 1100
23.92
1126
9.67
2004
17.63
11.08
LoGRA
11.42 1:100
12.86
1:004
TRAK
16.86 11003
23.30
#118
14.77
00.97
16.74
11.03
EKFAC
15.14 1:104
18.33 1:11
Table 3: AUC for Mislabeled Data Detection on ImageNet. We treat the manually misla- beled samples as the positive class and correctly labeled points as negative. Higher is better.
Table 4: Tail-pateh score (%) on WikiText-103. denotes the average change per query with 95% CI half-width, Higher better.
Method
w/o w
w
(A)
ResNet-18
ViT-B/16
Random
0.39
Method
w/o w
w
w/o
w
TracIn
4.66
5.60
0.94
#0.17
TracIn
54.40
61.46
71.27
83.58
TRAK
7.77
7.88
0.10
10005
TRAK
59.66
67.72
80.08
83.48
LoGRA
6.34
6.54
0.20
1:0009
EKFAC
5.54
6.09
0.55
=0.11
Appendix C.2 for a sensitivity analysis). We have also experimented with other loss functions (see Appendix such as a supervised loss using pseudo labels resulting from data augmentation, but found them inferior compared to our self-supervised one (Eq. 6). The weight learning process is highly efficient, typically converging in under minute. This efficiency stems from two factors: First, we only learn a small number weights (one per parameter group. e.g., one per layer); second, formulation allows to precompute attribution contributions from each parameter group, and then apply weights to the group-wise attribution scores during optimization, avoiding recomputing attribution scores with weighted gradient features each iteration.
Fine-Grained Attribution. Beyond general attribution, we can aim to isolate the influence of training data on specific aspects of the generated content, such as style or subject. Our data-driven approach naturally extends to such fine-grained attribution tasks by learning specialized weight sets for different semantic aspects (e.g., Wayle. Wsubject Wouckgrood)- The key insight is to construct query sets Q that emphasize the target attribute while leaving other aspects unspecified. For instance, to learn style-specific weights, we generate query images by specifying different styles in the prompts while not specifying subjects or backgrounds. This causes style-relevant training samples rank higher in the attribution scores, encouraging the optimization to up- up-weight parameter groups that that these specialized weights are more
consistently contribute to style semantics. focused than general-purpor ones, with distinct patterns for different semantic elements.
5 EXPERIMENTS
In this section, we demonstrate the effectiveness of our proposed framework. We first show that learning parameter importance consistently improves standard data attribution across diverse domains, including image classification, language modeling, and image generation. We then show that method can be extended to perform fine-grained, semantic-specific attribution. Further details on all models, datasets, baseline methods, and evaluations are provided in Appendices B. B.3 and B.2
5.1 STANDARD DATA ATTRIBUTION
We apply self- self-super method to learn parameter weights for suite of prominent gradient-
based attribution methods, including TracIn (Pruthi et al.) 2020). TRAK (Park et al., 2023), EK-
FAC (Grosse et al. 2023), JourneyTRAK (Georgiev al. (002)), D-TRAK (Zheng et al. 2023 and DAS (Lin et al.) 2024). Across all experiments, we enforce three disjoint roles for data: (i) model training data for learning parameters 6, (ii) beaanin-e data for optimizing parameter- geruu weights w. and (iii evaluation data for LDS, tail- mull-p and fine- ine-ggrined Recall @ 10. The precise
We use the EKFAC implementation in the LoGRA eodebase.
7

---

Published as a conference paper at ICLR 2026
Table 5: LDS (%) comparison on diffusion models. Our method improves performance across multiple datasets and attribution baselines. We report LDS + 95% CI half-width.
AntBench-2
Naruto
SB-Pokemon
CIFAR-2
Method
w/o
w/o
w/o
w/o
TracIn
17.63
22.02:+1.05
10.54_0.2
13.59=081
9.34
11.79_001
1.39
8.48#2.77
TRAK
18.39
22.15
14.61 =0:30
17.02:tox5
10.68:00.2
12.24
8.51
10.59:0075
JTRKK
11,56
16.00
13.41
=0:30
14.56
7.97=
9.24+10.4
6.03+a079
8.39++079
DTRAK
22.72
25.15
16.75
17.85 =0.8
33.88-0091
35.05
55.05
10.17 70.78
12.18 00.77
DAS
30.47=1.15
+1.58
18.72=002
20.44=0092
33.55
36, 36.12=009
12.66:nn80
13.79=0.81
splits for each setting are described below and summarized in Table Appendix [.1
We assess
the impact of our method across multiple evaluation metrics, detailed in the following subsections.
5.1.1 IMAGE CLASSIFICATION
We first evaluate our method on image classification by training a ResNet-18 (He al., 2016) and
a ViT B/16 (Dosovitskiy et al. 2020) model on a 50,000 subset of the ImageNet (Deng al. 2009) training split. Weights w are learned using 1,000 additional images sampled from the ImageNet training split, disjoint from this 50,000-sample subset. LDS (Park et al. 2023) is evaluated on 1,000 images sampled from the ImageNet validation split, disjoint from both the model-training and weight-learning data. As shown in Table I our method brings substantial improvements to both TracIn and TRAK across both architectures.
To assess performance
practical downstream task, we evaluate on mislabeled data detection. The key insight, following prior work (Koh & Liang 2017 Pruthi et al. 2020). is that mislabeled examples, being outliers, tend to exhibit high self-influence scores. We therefore rank all training data by normalized self-influence. By treating mislabeled points as the positive class and correctly labeled points as the negative class, can measure the ability separate them by computing the Area Under the ROC Curve (AUC). For this evaluation, we corrupted 10% of the training labels, and compute self-influence on a model trained on the corrupted data. As shown in Table 3] our method consistently improves AUC, indicating that our weighted approach provides a more effective ranking for prioritizing noisy samples for human inspection.
5.1.2 LANGUAGE MODELING
We next test our method on language modeling. We train a GPT-2-small (Radford et al. 2019) model from scratch on the WikiText- 103 (Merity et | (016) training split, containing 232,585 sequences of length 512. Weights are learned using 523 sequences from the WikiText-103 test split, and LDS is evaluated on 487 sequences from the validation split. Table shows consistent LDS improvements for TracIn, TRAK, LoGRA (Choe al. 2024). and EKFAC this held-out validation set.
In addition, we evaluate using the tail-patch score (Chang et al. 2024), a metric designed to measure the direct, additive contribution of helpful training examples. For each query, the procedure is as follows: we identify the top-128 training proponents, treat them as a single batch, and perform one incremental training step (a "tail-patch") on the final model checkpoint using this batch. The score is the resulting increase in the query's 's target sequence probability, averaged over all queries in the test set. A higher score indicates a better ability to retrieve genuinely useful training data. As shown in T our weighted approach more effectively identifies these helpful training examples, leading to larger performance gains.
5.1.3 IMAGE GENERATION
Finally, we evaluate on diffusion models with attribution methods such as JourneyTRAA (Georgiev ctt al. 2023), D-TRAK (Zheng et al.) 2023), and DAS (Lin et al., 2024). in addition to TracIn and TRAK. We use four datasets: CIFAR-2, subset of CIFAR- 10 (Krizhevsky
al. 2009);
ArtBench-2, a subset of ArtBench (Liao et al. 2002);; Naruto, a curated dataset of the naruto anime images (Cervenkal 2022); and our synthetic SB-Pokemon dataset. We train DDPM (Ho et al. 2020) for CIFAR-2 and fine-tune Stable Diffusion (Rombach et al., 2022) with LoRA (Hu et al. 2022 for the others using the respective training sets. For each dataset, we learn weights w from
8

---

Published as a conference paper at ICLR 2026
Top-5 Positive Samples
Top-5 Negative Samples
Subject
Style
Background
Figure 4: Fine-grained attribution example. The query image (top-left) generated with the prompt "A watercolor illustration of Eevee, in a bamboo forest". The first row shows the top-5 positive and negative training samples identified by standard D-TRAK without weights. Each subsequent row (Subject, Style, Background) displays training samples retrieved using specialized weights learned for that semantic element. Images marked 'S', 'P", or 'B' are ground-truth contributors for style, subject, background, respectively.
Table 6: Fine-grained attribution on SB-Pokemon using D-TRAK. Each row corresponds using weights learned for specific semantic target. Columns show Recall 10 (%) for attributing Subject, Style. Background elements, and their summation (All), on the train and validation prompt splits. "Rep. Dist." represents a simple baseline that ranks training images by f2 distance to the query in the feature space of ResNet-50 model pre-trained on ImageNet using MoCo (He al. 2019).
Train
Validation
Weight target
Subj.
Style
Backg.
All
Subj.
Style
Backg.
All
None (w/o w)
22.1
64.9
9.1
96.0
19.4
61.8
17.5
98.7
Rep. Dist. (Singla al. 2023
22.0
10.5
5.7
38.2
13.4
13.4
16.3
43.1
Subject
51.6
35.5
10.9
98.0
50.6
30.6
18.4
99.6
Style
17.0
82.1
0.6
99.8
15.2
73.0
10.9
99.0
Background
30.0
48.5
12.9
91.4
29.8
41.4
26.6
97.7
separate set model-generated query images, and evaluate LDS on another disjoint set of generated queries, following the prompting strategies described in Appendix B.2 and summarized in Table 7 Table that our method significantly improves LDS scores across all methods and datasets.
Across all three domains, method consistently improves the performance of all data attribution techniques. Furthermore, as shown in Appendix C.5 the learned weights exhibit strong robustness, generalizing well across datasets and attribution methods, suggesting they capture the model's intrinsic characteristics instead of overfitting to specific queries or methods.
5.2 FINE-GRAINED DATA ATTRIBUTION EVALUATION
In addition to improving general attribution, our framework enables fine-grained autribution-alllwn us to find the influence of training data specific aspects of an image, such as style, subject, or background. This done using the same self-supervised method, but with query sets curated focus on each target attribute (see Section 4.2)
We use our synthetic SB-Pokemon dataset for this experiment. For each semantic element, we learn a separate set of weights: Watyle. Wsahjeat, and Whackgroumd The query set Q for learning each specialized weight set is constructed from images generated using prompts that focus on variations of that specific element while keeping others general. For instance, to learn Weackgrond: Q consists of 200 images generated with prompts emphasizing different backgrounds (c.g.. "in misty marsh", "on a rocky beach"), without specifying subjects styles. To assess generalization, split the 10 categories

---

Published as a conference paper at ICLR 2026
for each semantic element into train split (5 categories) and a validation split (the remaining categories). The prompts used to learn specialized weights contain only concepts from the train split.
For evaluation, measure Recall @ 10 (the proportion of the top- 10 attributed samples that were known ground-truth contributors) for each semantic element separately. We generate 500 test images for both the train and validation splits. These images are created using prompts that combine elements from the respective splits, following the pattern "A [style] of [subject], [background]". (5 styles x subjects x 5 backgrounds 4 repeats 500 images).
As shown in Table applying specialized weights improves attribution for the targeted element. The validation results show strong generalization unseen styles, subjects, and backgrounds. Figure provides qualitative examples supporting the effectiveness of this approach: while some retrieved training images still reflect other concepts, target-concept contributors become much more prominent than under the unweighted baseline, which is expected since reweighting can suppress but not completely remove aff-target signal.
CONCLUSION
We have shown that explicitly learning parameter importance consistently improves mudientt-based data attribution. Our self-supervised, SNR-ggoudded framework boosts attribution accuracy across image classification, language modeling, and diffusion models, and enables fine- ine-grained semantic analysis.
Despite these advantages, our approach has two main limitations. First, the weights are layer-wiss in practice, which is computationally efficient but coarse; learning more granular, parameter-wis weights is natural next step, but may face stronger overfitting. Second, because the self-supervised objective bootstraps from existing attribution methods (for example, TRAK or D-TRAK) and a constructed query dataset, the learned weights inevitably inherit part of their inductive biases. We view this inherited bias as an acceptable trade-off for improving practical attribution quality without requiring ground- -ouuh influence labels.
Overall, our findings demonstrate that parameter heterogeneity can be modeled and learned from data, enabling more accurate, controllable, and interpretable training data attribution.
REFERENCES
Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. Multimodal datasets: misogyny, poroggaapyy, and malignant stereotypes. arXiv preprint arXiv:2110.01963, 2021.
Nicolas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Security 23), pp. 5253-5270, 2023.
Fole Cervenka. Naruto blip captions.
https://hugginge htt.a//ffaa.ccccaaaattttttteeet. 2022.
lambda wwtps/aaatttoop---eaeeee
Tyler A Chang, Dheeraj Rajagopal, Tolga Bolukbasi, Lucas Dixon, and Ian Tenney. Scalable influence and fact tracing for large language model pretraining. arXiv preprint arXiv:2410., 2224400.74415 17413, 2024.
Guillaume Charpiat, Nicolas Girard, Loris Felardos, and Yuliya Tarabalka. Input similarity from the neural network perspective. Advances in Neural Information Processing Systems, 32, 2019.
Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Zhangyang Wang, and Michael Carbin. The lottery ticket hypothesis for pre-trained bert networks. Advances in neural information processing systems, 33:15584415846, 2020.
Sang Keun Choe, Hwijeen Ahn, Juhan Bac, Kewen Zhao, Minsoo Kang, Youngseog Chung, Adithya Pratapa, Willie Neiswanger, Emma Strubell, Teruko Mitamura, et al. What is your data worth to gpt? lmm-scale data valuation with influence functions. arXiv preprint arXiv:2405. /3954, 2024.
Ian Covert and Su-In Lee. Improving kemnlssha: Practical shapley value estimation using linear regression. In International Conference on Artificial Intelligence and Statistics, pp. 3457-3465. PMLR, 2021.
10

---

Published as a conference paper at ICLR 2026
Wanyun Cui and Qianle Wang. Cherry top: Parameter heterogeneity and quantization in large language models. arXiv preprint arXiv:2404.02837, 2024.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly. et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010.11929, 2020.
Vitaly Feldman and Chiyuan Zhang- What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems, 33:2881-2891, 2020.
Kristian Georgiev, Joshua Vendrow, Hadi Salman, Sung Min Park, and Aleksander Madry. The journey, not the destination: How data guides diffusion models. arXiv preprint arXiv:23/2.06205, 2023.
Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In International conference on machine learning, pp. 2242-2251. PMLR, 2019.
Aaron Gokaslan, Vanya Cohen, Ellie Pavlick, and Stefanic Tellex. Openwebtext corpus, 2019.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Ilama herd of models. arXiv preprint arXiv:2407.2177 2024.
Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage. Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, et al. Studying large language model generalization with influence functions. arXiv preprint arXiv:2308.033: 03299, 2023.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. arXiv preprint arX1v:1911.05722, 2019.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.1259. 2022.
Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851. 2020.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Yuanzhi Li, Shean Wang. Lu Wang,
Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21226. 2024.
Andrew Ilyas, Sung Min Park, Logan Engstrom, Guillaume Leclere, and Aleksander Madry. Data- models: Predicting predictions from training data. arXiv preprint arXiv:2202.0062 00662, 2022.
Masaru Isonuma and Ivan Titov. Unlearning reveals the influential training data of language models. arXiv e-prints, pp. arXiv-2401, 2024.
Ajay Jaiswal, Zhe Gan, Xianzhi Du, Bowen Zhang, Zhangyang Wang, and Yinfei Yang. Compressing Ims:: The truth is rarely pure and never simple. arXiv preprint arXiv:2310.013322 2023.
11

---

Published as a conference paper at ICLR 2026
Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes. Nezihe Merve Gürel, Bo Li, Ce Zhang, Dawn Song, and Costas J Spanos. Towards efficient data valuation based on the shapley value. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1167-1176. PMLR, 2019.
Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael Mahoney, and Kurt Keutzer. Squeezellm: Dense-and-spars quantization. arXiv, 2023.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pp. 1885- 1894. PMLR, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language- -mgggg pre- training for unified vision-language understanding and generation. In International conference on machine learning, pp. 12888-12900. PMLR, 2022.
Peiyuan Liao, Xiuyu Li, Xihui Liu, and Kurt Keutzer. The artbench dataset: Benchmarking generative models with artworks. arXiv preprint 1rXiv:2206.11444 2022.
Jinxu Lin, Linwei Tao, Minjing Dong, and Chang Xu. Diffussin attribution score: Evaluating training data influence in diffusion model, 2024.
18639
Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Zhang, Yuandong Tian, Christopher Re, et al. Deja vu: Contextual sparsity for efficient Ilms at inference time. In International Conference Machine Learning, pp. 22137-22176. PMLR, 2023.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 17X05:1711.05001, 2017.
Xinyin Ma, Gongfan Fang, and Xinchao Wang. LIm-pruner: On the structural pruning of large language models. Advances in neural information processing systems, 36:21702-21720, 2023.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models, 2016.
Bruno Mlodozeniee, Runa Eschenhagen, Juhan Bae, Alexander Immer, David Krueger, and Richard Turner. Influence functions for scalable data attribution in diffusion models. arXiv preprint arXiv:2410. 13850, 2024.
Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclerc, and Aleksander Madry. Trak: Attributing model behavior scale. In International Conference on Machine Learning, pp. 27074-27113, PMLR, 2023.
Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. Advances in Neural Information Processing Systems, 33: 19920-19930, 2020.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog. 1(8):9, 2019.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer- assisted intervention-MICC 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pp. 241. Springer, 2015.
12

---

Published as a conference paper at ICLR 2026
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479-36494, 2022.
Andrea Schioppa, Polina Zablotskaia, David Vilar, and Artem Sokolov. Scaling up influence functions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 8179-8186.
2022a.
Andrea Schioppa, Polina Zablotskaia, David Vilar, and Artem Sokolov. Scaling up influence functions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 8179-8186, 2022b.
Chenyang Si, Ziqi Huang, Yuming Jiang, and Ziwei Liu. Freeu: Free lunch in diffusion u-net. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4733-4743, 2024.
Vasu Singla, Pedro Sandoval-Sepe Segura, Micah Goldblum, Jonas Geiping, and Tom Goldstein. A simple and efficient baseline for data attribution on images. arXiv preprint arXiv:2311.0338. 2023.
Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? investigating data replication diffusion models. Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 6048-6058, 2023.
Jaaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv
preprint arXiv:2010.02502, 2020.
Charles Spearman. The proof and measurement of association between two things. 1961.
Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306. 1695, 2023.
Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven mmage-to-image translation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 1921-1930, 2023.
Sheng-Yo Yu Wang, Aaron Hertzmann, Alexei Efros, Jun-Yan Zhu, and Richard Zhang. Data attribution for text-to-image models by unlearning synthesized images. Advances Neural Information Processing Systems, 37:4235542666 4266, 2024.
Zihao Wang and Victor Veitch. Does editing provide evidence for localization? arXiv preprint arXiv:2502.114477 2025.
Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. Depn: Detecting and editing privacy neurons in pretrained language models. arXiv preprini arXiv:2310.20138, 2023.
Chih-Kuse Kuan Yeh, Joon Kim, Ian En-Hsu Yen, and Pradeep K Ravikumar. Representer point selection for explaining deep neural networks. Advances in neural information processing systems, 31, 2018
Chih- Kuan Yeh, Ankur Taly, Mukund Sundararajan, Frederick Liu, and Pradeep Ravikumar, First better than last for language data influence. Advances in Neural Information Processing Systems, 35:32285-32293 2022.
Charles Yu, Sullam Jeoung, Anish Kasi, Pengfei Yu, and Heng Ji. Unlearning bias in language models by partitioning gradients. In Findings of the Association for Computational Linguistics: ACL 2023, pp. 6032-6002000 6048, 2023.
Xiaosen Zheng, Tianyu Pang, Chao Du, Jing Jiang, and Min Lin. Intriguing properties of data attribu- tion on diffusion models. The Twelfth International Conference on Learning Representations, 2023.
13

---

Published as a conference paper at ICLR 2026
A SNR-BASED DERIVATION OF THE
LOSS
In this section, provide theoretical justification for our proposed parameter weighting framework and self self-supervised loss, grounding it in Signal-to-Noise Ratio (SNR) maximization principles.
A.1 A SIGNAL-PLLUS--OOTE MODEL. FOR ATTRIBUTION
A conventional gradient-ban based attribution score, such as TracIn, can be decomposed into sum of scores from M disjoint parameter groups:
M T(Zg kn) = VoL. VoLn = Va, C Vo, Ln
M Σ ==1
(5
This formulation implicitly assigns a uniform weight of to each per-group score aj. Our method generalizes this weighted sum by introducing learnable, non-negative weights wj:
M Σ (Zq j=1
F(rgs Fnj w)
To formalize the rationale for this, we model the per-group score as a noisy linear measurement of true, unknown influence signal (1g Tn) IR:
(g Fn) aj - [x In) + t
where aj 2 0 is the group's signal sensitivity and is zero-mean random noise with variance of.
assumed be independent across groups. The weighted attribution score for a single pair is then:
F(IggZni w)
Σ Wjjy
(I)
+ Σ WjEj.
j=1
The quality of this aggregated score can be measured by its Signal-to-Noiss Ratio (SNR), defined as the ratio of the variance of the signal component the variance of the noise component:
Var(( Wyaz) Var V,r wyez]
C (ara)) Σ, w
SNR,
E wya,)²² Σ, uz0z By the Cauchy-Schwar Schwarz inequality, this expression is maximized when the weights are proportional the group's ' intrinsic signal-to-note noinoise characteristic: W x az/o3. This shows that an optimal non-uniform weighting exists and provides a clear theoretical motivation for learning weights.
Since Var(T) is a constant, maximizing the SNR equivalent to maximizing
A.2 OUR SSLEVIEDE Loss AS A PROXY FOR SNR MAXIMIZATION
Having established a theoretical goal (maximize SNR), we now show that our practical suff-supee objective is principled proxy for this goal. Let 7 (w) R be the full vector of attribution scores for a given query, and let (w)) be the set indices the top & scores that vector. Our objective is maximize:
#1 EiEZIp((w)) F(Eg, x²: w)
L(w)
[[
We now demonstrate that maximizing this objective approximates maximizing VSNR
14

---

Published as a conference paper at ICLR 2026
Numerator Maximizes Signal Sensitivity. examples. Let Iupp=((0)) = [Tg Øn)) In E Tropxx be the average true influence for this group. The expected value of the numerator is:
The numerator is the average score of the top-k training
E
Σ
Flaq 2';
BfF x ww)
k
(EIupa(w)
iEZups (w)
M
Σ wjaj
[2 20))
iETupa(mn)
M
[wp=((w).
Wjaj
During optimization, both terms on the right-hand side are affected by w. As w improves, the ranking becomes more accurate, likely causing [wp++(w) increase. However, the primary driving force of the optimization is the direct, linear dependence on the effective signal sensitivity term, E; Wj Therefore, maximizing the numerator is primarily driven by finding weights that maximize this term.
Denominator Estimates Noise Level. The denominator is the l2 norm of the score vector. Its squared value [=))l = >' F(Igg x"; w) We assume the vast majority of training examples have negligible true influence (1), x*) x 0). For these examples, the score is dominated by noise: F( xx; w) xe E3 wzej. The expected squared norm is therefore dominated by the noise variance:
N B mg x": :
Σ wyes )
E B
E
N Σ
0
Thus, the denominator [ serves as data-driven estimator proportional to the square root of the total noise power, v 2, mz0
Conclusion. Combining these insights, practical objective is direct proxy for the ideal one:
Σ3 Wjas WE,
((w) x
x V SNR...
This proves that optimizing our proposed loss is a principled method for maximizing the SNR of the attribution score under our segnal-plus-nois model. The iterative learning process allows the system to bootstrap its way to progressively better estimate of the optimal weights wi x az/oj.
Our SNR-based objective operates only on the raw attribution scores F(zg. Zni w): it increases a top-k average while controlling the overall l scale, and LDS values (a Spearman correlation over retrained models) never enter the optimization. Furthermore, the same learned weights improve several evaluations that are not defined terms of L.DS (mislabeled- data AUC, tail-patch, and qualitative analyses), and ablations such as removing the normalization term or replacing the Top-k term by a Top-k minus Bottom-k variant either collapse performance or fail to match our main loss. Together, these observations are hard to reconcile with superficial metric hacking and are instead consistent with the intended SNR interpretation, where the loss encourages the attribution mechanism place more weight parameter groups whose scores carry stable influence signal and to down-weight groups whose scores behave like noise, rather than manipulating evaluation metries directly.
Remark on "metric hacking" LDS.
15

---

Published as a conference paper at ICLR 2026
Table 7: Data usage across experiments. For each setting we distinguish three disjoint roles for data: model training (for learning 0), weight learning (for optimizing w), and evaluation (for LDS, tail-patee tail-patch, and/or fine-grained Recall 10).
Setting
Model training data
Weight-learning data
Evaluation data
Image clas-
50,000 images from the ImageNet training split (10% label corruption applied only for the
1,000 additional images from the ImageNet train- ing split, disjoint from the 50,000-sample sub-
1,000 images from the
sification
ImageNet validation split for LDS evaluation; 5,000 training images with corrupted labels for mislabeled data detection experiments.
(ImageNet;
ResNet- 18
ViT-B/16)
mislabeled-de experi- ments)
set
Language modeling (WikiText-
Texts from the training split concatenated and segmented into 232,585 sequences of length 512
523 sequences from the
487 sequences from the validation split (used for LDS and as tail-patch queries)
test split
103; GPT-2-
small)
Diffusion models
Dataset-specific training sets described below
1,000 model-generated query images per dataset for learning (for fine- grained SB-Pokemon weights, 200 generated queries containing only train-split concepts)
1,000 model-generated query images per dataset with disjoint random seeds for LDS; for fine- grained Recall @ 10 SB-Pokemon, 500 gen- erated queries per split (train / validation)
(CIFAR-2,
ArtBench-2,,
Naruto, SB-
Pokemon)
B TECHNICAL DETAILS
B. DATASET AND MODEL DETAILS
CIFAR-2 (32 X 32) The CIFAR-10 (Krizhevsky al. 2009) dataset consists of 50,000 training and 10,000 test images size 32 x 32 across 10 classes. CIFAR-2 is built by randomly selecting 5,000 samples from the "automobile" and "horse" classes for training, each with 2,500 images.
For the CIFAR-2 dataset, our unconditional DDPM (Ho et al. 2020) comprises 35.7M parameters. The diffusion process operates with maximum timestep of T000, utilizing linear variance schedule where /3 ranges from 31 = 10-1 to 37 = 0.02. Training incorporates a dropout rate of 0.0 and random horizontal flip data augmentation. Optimization is performed using AdamW (Loshchiloy) & Hutter 2017) with a weight decay of 10-6, The model is trained over 200 epochs using batch size of 128. he learning rate follows a cosine annealing schedule, initiated at 10 after 0.1 fraction warmup period. For inference, we employ the 50-step DDIM solver (Song et al. 2020).
ArtBench-2 (256 256) ArtBench (Liao et 2022 is a dataset of artwork images consisting of 10 distinct artistic styles, each consisting of 5,000 images in the training set. We follow the setup in (Zheng al. 2023) to build ArtBench-2, subset of ArtBench, containing 5,000 randomly selected samples from the "post-impressionism" and "ukiyo-e" styles, each with 2,500 training images.
On ArtBench-22 2, we fine-tune a text-to-image Stable Diffusion model (Rombach et al.) 2022) using LoRA (Hu et al. 2022) with the rank set to 128, leading to 25.5M learnable parameters. To accommodate ArtBench's 256 resolution, we utilize Stable Diffusion checkpoint pre-adapted for this size (from an original 512x512 resolution). The model is trained caads--ondiiiollly.. with textual prompts formatted as "a class painting" (c.g.. "a post- pest-ippressionism painting"), Training incorporates dropout rate of 0.0, AdamW optimization (Loshchilov & Hutter 2017) with 10" weight decay, and random horizontal flip augmentation. We train for 100 epochs with a batch size of 64. The learning rate, initially x 10-4 follows cosine annealing schedule with 0.1 fraction
16

---

Published as a conference paper at ICLR 2026
Cubone, black background
Bulbosour, gray background
An acrylic pour pointing
blocklight painting
cup, misty
guitor, near smoking valcono
Illustrative examples from the SB-Pokemon dataset, showcasing samples of Pokemon sub- jects, artistic styles, and environmental backgrounds. These images were generated using MidJourney V6.1 from the categories listed in Table 8
Figure 5:
Table 8: The categories used for generating SB-Pokemon dataset, divided into Pokemon subjects, artistic styles, and environmental backgrounds. For each column, the first five categories form the training split, and the subsequent five form the validation split, separated by a dashed line.
Subjects
Styles
Backgrounds
Cubone
An acrylic pour painting
a misty marsh rocky beach in a sunny meadow
Jigglypuff'
A blacklight painting
Mewtwo
A risograph print
Bulbasaur
A cyanotype style photo
sunflower field
Lickitung
A pointillism painting
near smoking volcano
Rhydon
A watercolor illustration
by a calm lake
Gengar
A Gongbi style painting
under aurora borealis
Snorlax
A sepia tone photo
frozen tundra
Arcanine
A red wash painting
a bamboo forest
Fevee
A charcoal pencil sketch
beneath waterfall
warmup. For inference, images are generated using 50-step DDIM solver (Song et al. 2020) and guidance scale 7.5 (Ho Salimans 2022).
SB-Pokemon (256 256) The SB- SB-Pokemon dataset was synthetically created using MidJoumny V6.1 to evaluate attribution for distinct semantic elements: subject, style, and background. consists of 600 images, derived from 10 categories each for Pokemon subjects (e.g.. Bulbasaur), artistic styles (e.g., blacklight painting), and environmental backgrounds (e.g.. beach), with 20 unique images per category. This structured design provides unambiguous ground-truth contributors for fine-grained attribution. The complete list of 30 categories are provided in Table g with illustrative examples shown in Figure
We use the same fine-tuning setup as for ArtBench-2.
Naruto (256 x 256) We curated the original Naruto dataset (Cervenkal 2022 by replacing the BLJP-generated (Li et al. 2022) captions with more accurate ones generated by GPT-40 (Hurst et al. 2024). For examples, the original incorrect caption "a with red hair and black shirt" was replaced with "a ninja wearing a unique mask with a playful design and spiky brown hair stands confidently.". It contains 1,221 images from the Naruto series and corresponding descriptive captions.
We use the same fine-tuning setup as for ArtBench-2.
B.2 METRICS
B.2.1
LINEAR DATAMODELING SCORE (LDS)
The Linear Datamodeling Score (LDS) PPar al. 2023) evaluates data attribution method, by assessing its ability to predict how model outputs change when the model trained on different subsets of data.
17

---

Published as a conference paper at ICLR 2026
Consider full training dataset S of size N, model output function f(;; 8) (e.g., the model's loss or logits for am input æ given parameters 0), and the data attribution method 7 under evaluation. The LDS calculation proceeds as follows:
1. Generate Subsets: Create M random subsets of the training data, denoted as {8th J m==1- Each subset 5th S typically has a fixed size, often a fraction a . N the total dataset size (c.g., 0,5 for 50% subsets).
2. For each query sample Equery and each subset S
(a) Calculate Actual Model Output (u Retrain a new model (or fine-tune the existing model architecture) using only the training samples in the current subset S Let the parameters of this retrained model be 0m. The actual model output for the query Zquery using this subset-trained model then ym اسtuall (Iqqury 0m).
(b) Calculate cirribution-baser Predicted Output (ypredicted)) This prediction is derived using the attribution method т. The attribution scores, (IIquery S) represent the influence of each training sample (+) from the original full dataset S on the query Equery (as determined by using the original model trained on S). The predicted output for the subset Sth is the sum of these attribution scores for all samples (4) that are part of Sm
predicted Vm
9r ququayy STh ST,SS
Σ
(FEquery 8)(1)
3. Compute LDS:
The LDS for the attribution method t with respect to the query Equery the Spearman rank correlation p (Spearman, 1961) between the list of M actual model outputs and the list of M attribution- based predicted outputs:
LDS(T, Zquery) P (
[M]) (uquey 0m) E [M]}, (gr (Fquery; Sth ,S) m E [M]]) (9)
[M],, (yeredieedd
A higher LDS value (closer to 1) indicates that the attribution scores provided by are more consistent with the true impact of training data subsets on the model's behavior for I'uuery suggesting a more accurate attribution method. Conversely, an LDS closer to O suggests poor correlation.
Implementation Details. To conduct the LDS evaluation, we set a = 0.5, and sample M 100 (M = 64 for diffusion) different random subsets of the training set D, and train models from scratch on each one of these subsets. For diffusion models, we let the model output function f( 0) be the simple loss Simple (z; 0) E,,, [||e - eo(vā + vT ā,é []] from DDPM Ho al. 2020). To approximate the expectation, we select 50 timesteps evenly spaced within the interval 1, = 1000]; and at each selected timestep, we sample one Gaussian noise e (O I). Finally, average the LDS scores over all the samples of interest.
For each LDS evaluation for diffusion, we utilize 500 query samples generated by the original model using distinct random seeds, and the average is reported. The query images are generated using the following dataset-specifie prompting strategies:
CIFAR-2: Query samples generated unconditionally, as the DDPM for this dataset is trained without text prompts.
ArtBench-2: 500 query images are generated with the prompt "a post-mmpressionism painting" and another 500 with "a ukiyo-e painting".
SB-Pokemon:
1000 query images are generated using combinations of subject, style, and background categories from Table 8 (e.g., "A risograph print of Eevee, in misty marsh"). The train/validation split of these categories is not considered for the LDS query set generation.
Naruto: 1000 query images are generated with 1000 different prompts randomly sampled from the dataset.
For image classification task, we randomly selected 1,000 samples from the training set (that non-overlapped with the 50,000 samples used for pre-training for learning weights, and 1,000 samples from the validation set queries for LDS evaluation.
18

---

Published as a conference paper at ICLR 2026
For language modeling task. used the training split of WikiText-103 (Merity et al. 2016) for pre-training, which contains 232,585 sequences length 512. We used the 487 text sequences from the validation split queries for LDS evaluation and the 523 sample from the test split for learning weights.
B.2.2 MISLABELED DATA DETECTION VIA SELF-INFLUENCE
Data attribution can be used for the practical task of identifying noisy or corrupted labels within training set, helping experts prioritize their attention (Koh & Liang 2017 Pruthi et al. 2020).
The core hypothesis is that an incorrectly labeled example, being an outlier with respect to its assigned class, will exert strong, corrective influence the model. This is measured by its self-influence. i.e., the influence of a training point on its own loss, calculated as 7 (T; 2:). A high self-influence score suggests that the model finds the example surprising difficult fit, which is characteristic of a mislabeled sample.
The evaluation procedure is follows:
1. Simulate Noisy Labels: experiments) intentionally mislabeled. model then trained on this corrupted dataset.
To create ground truth, fraction of the training data (10% in our
2. Rank by Self-Influence: The self- influence score is computed for a 5,0000-sample subset in the training set. All samples are then ranked in descending order based on these scores. Each self-influence score is normalized by dividing the sum of the scores of the top-ranked training samples. We found using the top-10 10 samples for normalization works the best for the mislabeled data detection task on ResNet-18 (top-100 for ViT).
3. Compute AUC: To quantify the quality of the ranking, we treat the known mislabeled samples as the "positive" class and the correctly labeled samples as the "negative" class. The Area Under the ROC Curve (AUC) is then computed. higher AUC indicates that the self-influence scores more effective at separating the mislabeled samples from the correctly labeled ones, with an ideal method ranking all mislabeled samples at the top. An AUC of 0.5 corresponds to random ranking.
B.2.3 TAIL-PATCH SCORE
The Tail-Patch Score (Chang et al. 2024) is an influence metric designed to measure the direct, causal impact of influential examples, rather than just correlational similarity. quantifies the additive contribution of top-ranked proponents by measuring how much they improve the model's performance on a specific query through incremental training.
The procedure for given query sample is as follows:
1. Identify Proponents: Use the data attribution method to identify the top-k most influential training samples (proponents) for the query. our experiments, we use 128 because the batch size we used for pretraining GPT-2 is 128.
2. Form a Batch: These l proponents are collected and treated as a single training mini-batch.
3. Perform Incremental Training: Starting from the final, converged model checkpoint, perform single, additional training step (a "tail-patch" step) using this mini-batch of proponents, maintaining all original training hyperparameters.
4. Measure Probability Increase: Measure the change in the model's log-probability of predicting the correct target sequence for the query before and after the tail-patel step.
The final Tail-Patch Score is this change in prgobabbbbllty, averaged over all queries in the evaluation set. A higher score indicates that the attribution method is more effective identifying training examples that are genuinely helpful for improving the model's performance on specific examples.
B.3 SELECTED ATTRIBUTION METHODS
Our experimental setup for the baseline methods aligns with (hhng et al. (023) and (Park et al.) 2023 SSpecifically, for all considered attribution methods, incorporate TRAK's scalability
19

---

Published as a conference paper at ICLR 2026
optimization (Pak et al., 2023). which utilizes random projection of gradient features enhance computational efficiency and reduce storage demands. Furthermore, also consistent with (Zheng the final checkpoint of the trained model for all evaluations, omitting intermediate checkpoints to manage computational costs.
et al. ) we exclusively
Next, we provide the detailed definitions of the baseline methods that consider.
TracIn. We utilize the TracIn estimator as described by Pruthi et al. (2020). The attribution score for training sample x" E 'D with respect to query sample Equery is implemented as:
t Σ
(P. Ve (Xquery; 0*)) T (P. V Simple (a"; 0*))
T TTquary x²))
(10)
where the number of model checkpoints & selected evenly from the training trajectory, Pe is the projection matrix, and Simple (1:; 09) is the simple loss function from DDPM (Ho et al. 2020). Note that the TracIn method we report in our main paper is specific instance of TracInCP where
C== 1, meaning only the final model checkpoint is used. This aligns with our general approach for other baseline methods where we exclusively use the final checkpoint for computational efficiency, as detailed previously. For TracIn and other methods except Journey TRAK, cach gradient is the average of gradients over all timesteps. We use the same projection dimension & = 25472 for all methods throughout
diffusion experiments. For image classification, k = 11264 for ResNet- 18 and l 12800 for ViT. For GPT-2 on language modeling tasks, TracIn and TRAK set k 196608 and LoGRA uses ls 442, 368 with -9.
TRAK. We employ the TRAK estimator formulated by Park et al. (023). For its application in our diffusion model setting, the model output function is instantiated as the simple loss in DDPM (Ho et al. 2020), Simple. Consistent with our approach for other baselines, we use only the final model checkpoint, denoted as A. The TRAK attribution score for a training sample x. with respect to query sample Jquery is then calculated as:
TIRAK TIRAK(FQuary x²) (P V Vimpii quaayy 8))
T + XI) P P Vo Simple 0),
(11)
where + ()(x (N) with (xx()) = PT Vo Simple (x (n)));))
(12)
In these equations, P represents the random projection matrix. The matrix c is constructed from the projected gradients of all N training samples æ00 in the dataset D, and À is regularization parameter for the kernel inversion for numerical stability. For each method except TracIn, we sweep the regularization parameter over the values [5 10 3.33 5 10 2, 5 10-1,5, 5,55
D-TRAK. The implementation of D-TRAK is very similar to TRAK. The only difference is that the model output function is replaced by Lsquare (x; 0) E.,,, [Ile(l/ā, vI a,,, +)]]]]
TD-TRAK (Fquery, ) (PT Vo LSquare (Equery; f)) T o क + AT)
P V L Squuie (I";0),
(13)
where 4 (0(x ..... ((........((())))))). ((x with ((x(1))) PT V LQ 00)
(14)
Journey TRAK. Journey TRAK (Georgiev et al., 2023) focuses on attributing influence to noisy images I, at a specific timestep throughout the generative process. contrast, our approach aims to attribute the final generated image. Journey TRAK is adapted to our setting by averaging the attributions over the denoising timesteps as follows:
T V
1
(F-uRAK(Qquory..o x²)
Σ
PTRAK + A1) PT V Cimplee (2 (x" 0).
(15)
Here, Tquery is the final query image, and x" is the training sample from dataset D. T' denotes the total number of inference steps (set to 50 in our experiments), and 2; is the noisy version of Equery inference step along the sampling trajectory. The matrix TTRAK is constructed identically to & as defined in Equation equation 12 (i.e., using the average of projected gradients of Liiple for all training samples with the final model parameters 0). The term Simpte (z;; 6) and Sample (I²; 8) are both evaluated for the noisy images at inference timestep
20

---

Published as a conference paper at ICLR 2026
DAS. Diffusion Attribution Score (DAS) is recently improved method for attributing influence to training samples in diffusion models (Lin et al. 2024). To attribute the influence of a training sample
r" E D on generated sample Equery throughout the entire generation process, the attribution score
TDAS (Tquery, x²) is calculated as follows:
(PT VOLDAS(Eques) 0))
④ T + +1) PT V CD 0))rr) rm
TAS (Uquery, x²)
( AT) PT V Lonss( 11?
(16)
1 - (PT VO (oDDAs(=";0)
where 4 [(((x ................................ (1 with ((zا)) P VeCoosS(
(17)
1 T
and p" = T Σ |co(0) t) - cel1
(18)
Here, LDAS(E;;) 0) = E.,e [calv + vI- vi-āaee t)] and r" is the average of residual noise over all timesteps for the training sample x².
LoGRA. LoGRA (Low-Rank Gradient Approximation) is a method that significantly improves the efficiency of gradient projection for large models (Choe et al., (024)). Its key insight stems from the structure of backpropagation. For a layer with weights W, the gradient VW can be expressed as a sum of Kronecker products between the forward activations (layer inputs x;) and backward activations (output gradients Dr.,):
vee vec(VW)=
Σ Zitt & DEo,t. t=1
(19)
Instead of first computing the full, high-dimensional gradient VW and then projecting it, LoGRA applies separate, smaller projection matrices, P; and Po, directly the activations:
gw gw(x) = (Paadd @ (P.Dzo.))
(20)
where (w) the projected gradient for the layer's weights. The final feature vector for the entire model, ÓLOGRA (r), is concatenation of these efficiently computed projected gradients from all targeted layers. This "project-then teett-thh---obbbeee strategy avoids ever materializing the full model gradients, leading to substantial memory and computational savings. The final attribution score follows the TRAK formulation (Eq. TT) using these features.
EKFAC. The Figenvalue-correeted K-FAC (EKFAC) method scales influence functions by construct- ing a high-quality, tractable approximation of the inverse Hessian matrix (Grosse et al. 2023). For our EKFAC baseline, we adopt a hybrid approach: compute gradient features using the efficient LoGRA projection (Eq. 20), but replace the standard empirical kernel with the EKFAC Hessian approximation.
The EKFAC kernel, HERFAC- is built by approximating the full Hessian H. as a block-diagonal matrix, with one block H; for each model layer I. Each block is then approximated by the Kronecker product of two smaller matrices:
= diag((H)..... Hz). where H2 = A & Gt.
(21)
Here, A is the covariance of layer inputs and G2 is the covariance of layer output gradients. The inverse is then computed efficiently with cigenvalue corrections for stability, as detailed in the original work. The final attribution score is calculated as:
TEKFAC(Aqqury x²) OLoGRA (Tquer) HEREAC ØLoGRA (x").
(22)
This hybrid approach leverages LoGRA's efficiency for feature generation and EKFAC's sophisticated Hessian approximation for the kernel.
B.4 WEIGHT LEARNING
The self-supervised weight learning procedure employed in our experiments is detailed in Algorithm I| For general-purpose weight learning, the query consists of N, = 500 samples, generated using
21

---

Published as a conference paper at ICLR 2026
Algorithm Parameter Group Weight Learning
Require: Set of N₂ query samples {x/² /
Require: For each query ziq ( precomputed group-wise contributions ((+), an x M matrix where [n,j) is the j-th group's attribution score of the n-th training sample query xig (N: number of training samples, M: number of parameter groups).
Require: Hyperparameters: learning 1/, number of epochs max the number of top-ranked samples to be considered k, weight regularization X. R. 1: Initialize raw weights Wnaw MR with random values.
Ensure: Learned parameter group weights
2: Initialize optimizer O (e.g.. AdamW) for Wraw with learning rate 1y and weight decay X
3: for epoch = 1 to Emax do
4: for each query Zyg do
5: Let Cy + ((+) (the N x M matrix for current query 2n
6: w Softmax (Wrw))
Derive actual weights w, from Wrawjj
Calculate attribution scores S E R.V for query 24q (t) M ンラ= Sn + wy [C for I.
7:
8:
9:
5 S/||S||2
Normalize scores
10:
Stankd & reordered scores from sorting S in descending order. LSSl +-- - Avg (aakkal .k)
Self-supervised loss (Eq.
12:
O.zero goad() loss. boss.backward() O.step()
13:
14:
Update Wraw
15:
end for
end for
17:
Obtain final weights w Softmax(wm.))
return
random seeds and diverse prompts from the respective dataset. For fine-grained weight learning (e.g.. to discern influences of specific semeantic elements like backgrounds), a more targeted set of Ng 200 samples is utilized, generated with prompts that emphasize particular semantic element (c.g.. for background, "in a misty marsh" "on rocky beach", while omitting explicit subject or style specifications). Standard training settings include Fmax 10 epochs and learning rate r) 0.01, managed by an Adam W optimizer with cosine annealing schedule. For each weight learning task, we conduct hyperparameter search, varying keop (the number of top- and bottom-ranked samples for the contrastive loss) across the values [1, 5, 10, 20, 50, 100, 200, 500, 1000, 5000]. and weight regularization parameter X' over [0.0, 0.02, 0.1,0.1.000. 0.3, 0.5, 0.8. 1.0. 1.5). The primary computational cost lies in generating the query samples and pre-computing their corresponding group-wise contributions C(,), which scales with Ny, the cost of the generative model, and the chosen attribution method. Once these prerequisites are precomputed, the weight optimization process itself efficient, typically concluding in a few seconds.
C ADDITIONAL RESULTS
C.1 CONSISTENCY OF PER- GER-GROUP ATTRIBUTION SIGNALS
In Section we demonstrated that attribution strength is highly heterogeneous across model's parameters. critical question is whether this pattern heterogeneity is stable, intrinsic property of the model, artifact of specific dataset or attribution method. To validate the former, conduct a similarity analysis.
For given model, can characterize its pattern of heterogeneity by a vector containing the per-group LDS scores. We compute these vectors for multiple settings, varying both the dataset (ArtBench-2, Naruto, SB-Pokemon) and the base attribution method (D-TRÁK, TracIn), and then measure the cosine similarity between them. A high similarity indicates that the underlying pattern of which parameter groups most important is consistent across different settings.
22

---

Published as a conference paper at ICLR 2026
-000
5:00
619
-800
6m
180
5:10
+80
6012
LG3
6280
+800
6217
1:0
642
LG3
6131
100
1:00
1:0
180
Megren Made
amter poase
eabut pisse
(a) Similarity of Per-Group LDS Scores
(b) Similarity of Learned Weights (w)
Figure 6: Consistency of Attribution Signals and Learned Weights. Heatmaps show the cosine similarity between (a) vectors of raw per-group LDS scores and (b) our learned weight vectors, across different datasets and methods. Brighter colors indicate higher similarity. Our learned weights show significantly more stable and robust pattern.
Figure 6a visualizes the similarity matrix for these per-group LDS scores. The heatmap shows consistent, moderately strong correlation across all pairs (average off-diagonal similarity of 0.689). Given that individual per-group LDS measurements are inherently noisy, this level of similarity provides strong evidence that stable, underlying pattern of parameter importance exists within the model architecture.
Furthermore, we hypothesized that if our method is effective, it should be able to filter the noise in these raw scores and distill a more robust representation this underlying pattern. test this, we performed the same similarity analysis on the learned parameter weights (w) from each setting. The result, shown in Figure (66, confirms this hypothesis. The learned weights are significantly more consistent, with much higher average off-diagonal similarity of 0.845. This is powerful validation of our approach: it demonstrates that our self-supervised learning algorithm successfully identifies and amplifies the stable, intrinsie signal of layer-wise importance while filtering out the noise present in the raw ger-group attribution scores.
C.2 SENSITIVITY ANALYSIS FOR HYPERPARAMETER A:
Our self-supervised learning objective (Eq- 6 depends on the hyperparameter which defines the number of top-scoring training examples to use as -ossiiipsssivvs. To assess the stability and sensitivity our method to this choice, we performed sensitivity analysis where we varied & across range of values: (1 10, 20, 50, 100).
Table 2 presents the results, showing the improvement in LDS (ALDS) achieved by our method over the unweighted baseline for different values of k. The analysis was conducted for both TracIn and TRAK on the ArtBench-2 and SB-Pokemon datasets.
The results reveal two key findings. First, on datasets like ArtBench-2, the performance is remarkably stable across a broad range of k, indicating that the learning process is robust and not dependent on fine-grained tuning.
Second, on datasets like SB-Pokemon, performance is more sensitive and can degrade for larger values of k. This is expected behavior: as l increases, the set of pseudo-positives is more likely to include noisy, non- non-influential examples, which can corrupt the supervisory signal for weight learning.
Despite this sensitivity, the method remains highly practical for reasons. First, the weight-learning process is extremely efficient, typically converging in under one minute, which makes tuning & for a given dataset or task entirely feasible. Second, show in Appendix C.5 C.5
the learned weights often
23

---

Published as a conference paper at ICLR 2026
Table 9: Sensitivity to k. The table shows the final LDS (%) for different values of the number of pseudo-positives. The "w/o w" column is the unweighted baseline, while the "w" column shows the result using the best-tuned as reported in the main text. Performance is stable for range of smaller k but can degrade if & becomes too large.
Method
Dataset
w/o w
k=
k=5
k=10
k=20
k=50
k=100
TracIn
ArtBench-2
17.63
21.07
22.01
22.02
21.97
21.91
21.92
TRAK
ArtBench-2
18.39
21.75
21.92
22.15
22.07
21.97
21.89
TracIn
SB-Pokemon
9.34
11.63
11.79
11.73
7.46
8.78
6.23
TRAK
SB-Pokemon
10.68
11.83
12.24
11.97
11.00
8.98
9.00
VIT-8/16 on ImageNet
Stable Diffusion on wrtBench-2
18
16
22
Tracin (Bazeline)
TRAK (Wwightee)
(Welghtod)
(56)
3.8
Journey TRAK (Weightdd
TRAAK (Baseline)
(Weighoo))
TRAK (Baselin))
JouneyTRAK (Baselino)
16-
10
12
10
10
10P
10a 102
6
10a4
18P
10:
10F
Notse
Figure Robustness Noise during Weight Learning. LDS obtained after learning weights from attribution scores perturbed with Gaussian noise of standard deviation The weighted methods maintain their advantage over a broad range of noise levels, degrading only when the injected noise completely dominates the original signal.
demonstrate strong transferability across different settings, reducing the need for repeated tuning for the same model.
C.3 SENSITIVITY ANALYSIS FOR NOISE IN ATTRIBUTION SCORES
To investigate if our weight-learning algorithm is robust noise in the attribution scores, perturb the layer-wise attribution scores during weight learning. Concretely, we first measure the standard deviation oy of the attribution scores for each parameter group across all query-training pairs. Then, we add i.i.d. Gaussian noise EÚ A/(0, (s a²) to each contribution ay (2 In.) before computing the top-k objective, where is the noise scale multiplier. Finally, we evaluate the learned weights on clean validation queries. Figure reports the resulting LDS function of the noise scale multiplier for both ViT-B/16 on ImageNet (TracIn and TRAK) and Stable Diffusion on ArtBench-2 (D-TRAK and JournyyTRAS TRAK).
Across all settings, the weighted methods retain sizeable lead over their unweighted baselines even when one to two orders of magnitude larger than the typical score magnitude. For ViT, the TracIn-based weights remain virtually unchanged up to a 10, and only begin to converge towards the unweighted model beyond a = 30. The TRAK variant exhibits the same trend, degrading smoothly rather than collapsing. The diffusion experiments mirror this behavior: D-TRAK maintains its LDS improvement for small and moderate noise, while Journey TRAK stays above its baseline up to = 10. These results corroborate the intuition discussed in the rebuttal that our objective only requires the top-k set to be enriched with true influences on average; sporadic corruptions behave like zero-mean noise that is averaged out during weight learning.
24

---

Published as a conference paper at ICLR 2026
Table 10: LDS (%) evaluation of alternative loss functions on ArtBench-2. Our proposed self-supervised top-k loss consistently achieves the best or near-best improvements. We bold the bighest-performing loss for each attribution method. The 'Top-k' column corresponds our main proposed loss (Eq. 6
Method
w/o
Top-k (Eq. 6
Supervised
Bot-k
Top-k
Bot-k
No Norm
TracIn
17.63
10.96
22.02
11.05
22.31
1:099
21.28
1.06
21.66
6.89 10.1
TRAK
18.39
1:996
22.15
10,99
21.83
4:099
20.95
11000
21.83 +1000
9.80 20085
JTRAK
11.56
#0089
16.00
11000
13.87
40.44
13.96
+:096
15.50 11005
1.62 #0.80
DTRAK
22.72
11.02
25.15
1.00
24.50
11.06
20.37
1:094
24.00 11.05
14.69 10.85
DAS
30.47 11:15
31.58
11.15
30.90
11:15
31.03
1:116
31.58
11.14
7.72 1:44
C.4 ABLATION STUDY THE LOsS FUNCTION
In the main text, we propose a self-supervised loss function (Eg. 6 to learn the parameter weights w. To validate this choice, we experimented with several alternative formulations. For each alternative, we performed hyperparameter sweep to ensure fair comparison. The results, evaluated on the ArtBench-2 dataset, are presented in Table TO Below, we detail each of the alternative objectives.
Supervised Loss with Data Augmentation. A natural alternative is supervised approach. We generate a query sample x squery by applying data augmentations (random horizontal flip. resized crop, color jitter) to training sample x". We then treat x" the ground-true ttttttuth sreudd-positive for the query query- The loss function formulated to maximize the score of this single pseudo-positive, while keeping the normalization term:
7 [ query x= w)
SSupervised (w)
|| [=('query D; w)) ||2
As shown in Table TO this approach is competitive but generally outperformed by self-supervised loss. While avoids the hyperparameter k; it introduces a new dependency on the choice of data augmentations, which can be highly Nomain-speciff and may not always preserve the core semantic content needed for attribution.
Bottom-k Loss. Another potential source of signal lies in the least influential (most negative) examples. We test loss function that minimizes the average score of the bottom-k examples:
(Bo Ejetam Tj
[FF(2)
In theory, if negative influence signals are strong, minimizing them should improve attribution. However, as shown in Table TO this objective yields mixed results: while often improves over the unweighted baseline, it consistently underperforms the Top-k objective and sometimes even degrades performance (D-TRAK). We reasonably hypothesize that truly harmful training examples are rare, and the bottom tail of the attribution score distribution is largely dominated by noise rather than meaningful negative signal; neural networks tend to "memorize' specific examples but does not actively "forget" the exact opposite of point.
Top-k minus Bottom-k Loss. A common strategy in contrastive learning is to maximize the gap between positive and negative examples. We implemented this by subtracting the average score of the bottom-k examples from the numerator:
N EieT 7,
EiE 7
GGap (w)
[F(w)l12
Intuitively, this combines the Top-k signal with the Bottom-k minimization. However, we found that this did not provide a consistent benefit and sometimes slightly underperformed our main loss. This further suggests that the bottom-k scores significantly dominated by noise (Eş), so including them adds variance while providing little useful signal. Thus, focusing on the top-k positive signal (as in main objective) remains the most robust strategy.
25

---

Published as a conference paper at ICLR 2026
Table 11: Generalization of Learned Weights across Datasets. We report LDS (%) for: the standard approach ("w/o w"); with parameter weights learned on the respective dataset ("w"): ) and with parameter weights learned on the other dataset shown ("Woter" e.g.. for ArtBench-2, Wother uses weights from Naruto, and vice-versa). Higher LDS values indicate better performance. TRAK's random projection was applied to all methods and datasets for fair comparison.
ArtBench-2
Naruto
Method
w/o w
W
Wother
w/o w
W
Wother
TracIn
17.63
22.02
20.19
10.54
13.59
13.48
TRAK
18.39
22.15
19.35
14.61
17.02
17.19
JTRAK
11.56
16.00
12.55
13.41
14.56
14.36
DTRAK
22.72
25.15
23.61
16.75
17.85
18.36
DAS
30.47
31.58
30.74
18.72
20.44
20.45
No Normalization. To validate the role of the denominator in our loss, we performed an ablation where removed the &a norm, leaving only the numerator be maximized. As the "No norm" column in Table (Ooswss, this completely destabilizes the learning process, leading to a drastic drop in performance. This result strongly confirms the importance of the normalization term, which, as argued in our SNR derivation, acts as a crucial regularizer that estimates the noise level. Without it, the optimization can trivially maximize the numerator by simply scaling up the weights w, failing to learn meaningful signal of parameter importance.
These experiments validate that our proposed sup--sueee loss provides the most robust and effective performance across a range of attribution methods, offering well bell-balanced objective that is both theoretically grounded and empirically superior to these alternatives.
C.5 GENERALIZATION OF LEARNED WEIGHTS
Our experiments demonstrate that the learned parameter importance weights exhibit notable general- ization capabilities, not only across different datasets but also across various attribution methods. This suggests a degree of robustness and potential for transferring learned weights, potentially mitigating the need for re-learning them in every new scenario. In most cases, applying weights learned from different context (either another dataset or another attribution method) still results in an improvement in the Linear Datamodeling Score (L.SS) compared to not using weights.
Generalization Across Datasets. As shown in Table IT weights learned on one dataset often yield performance improvements when applied to a different dataset. For instance, when evaluating on Naruto, applying weights learned from ArtBench-2 consistently leads to higher LDS scores across all attribution methods compared to using weights learned directly on Naruto. Conversely, when evaluating on ArtBench-2, weights learned on ArtBench-2 itself generally provide the best results. This observation suggests that the characteristics of the dataset used for weight learning can influence the quality and transferability of the learned weights. It is plausible that larger or more diverse dataset, such ArtBench-2 in our experiments, might facilitate the learning of more universally effective weights. However, the precise factors contributing to this phenomenon remain an avenue for future investigation.
Generalization Across Attribution Methods. Table [2] details the performance when weights learned using one attribution method are applied to another, evaluated on the ArtBench-2 dataset. key observation is that for most methods (TracIn, TRAK, D-TRAK, DAS), the highest LDS is achieved when using weights learned for the method itself (i.c., the values in the "w" column). Nevertheless, applying weights learned by other methods often leads to LDS improvements over the baseline "w/o w" scenario.
Overall, these findings underscore the adaptability of the learned weights. While dataset-specific and method-specin speccffi weights often yield optimal results, the ability of these weights to generalize offers practical advantages, particularly in scenarios where re-computing weights for every specific mathset-method combination might be computationally prohibitive.
26

---

Published as a conference paper at ICLR 2026
Table 12: Generalization of Learned Weights across Methods. We report LDS (%) for each method on ArtBench-2: without any learned weights ("w/o w"); with weights learned by the method itself ("w"); and with weights learned by other attribution methods (e.g., "W'rnaci" signifies that weights learned by TracIn were applied to the method in the current row). Higher LDS values indicate better performance. TRAK's random projection was applied to all methods.
ArtBench-2
Method
w/o w
W
WTaclnn
WTRA
WITRAK
UDTRAK
WDAS
TracIn
17.63
22.02
21.02
20.28
19.96
19.97
TRAK
18.39
22.15
21.80
20.63
19.68
20.82
Journey TRAK
11.56
16.00
15.64
15.58
12.16
14.05
D-TRÁK
22.72
25.15
23.60
23,45
22.07
23.91
DAS
30.47
31.58
30.14
30.71
28.69
29.68
Table 13: Tail-patch score (%) OpenWeTText with Llama3-8B-Instruct A denotes the average
change per query with 95% CI half-width. Higher is better.
Method
w/o
(A)
Random
-0.07
TracIn
0.04
0.65
0.61
10.11
LoGRA
1.78
1.92
0.15
=0.05
C.6 SCALABILITY TO LARGER MODELS
To address the concern regarding the scalability of our method, we conduct additional experiment using -B-nat--B-Instruet (Grattafiori et al. 2024). Following the protocol in Choe et al. (2024), we perform data attribution the Open WebText (OWT) dataset (Gokaslan et al. 2019), assuming that the model was pre-trained on this corpus, without conducting further training ourselves. We extract the first 1 million entries from the dataset and process them into 99,642 sequences of fixed length 512 tokens to form the training set. We then generate 500 queries, each consisting of 512 tokens, by prompting the model continue text from 500 snippets randomly sampled from the training set. We employ the tail-patch evaluation: for each query, we take the top-1 attributed training example, perform one additional fine-tuning step of the model on that example, and measure the resulting change in the query's 's log-likelihood. The results, reported in Table [3] show that the weighted attribution consistently outperforms the unweighted baselines, mirroring the patterns observed for smaller models in the main text.
C.7 QUALITATIVE ANALYSIS FOR LANGUAGE MODELING
To provide more insights into the qualitative effectiveness of our method, we provide examples of data attribution results, comparing the top-1 ranked training example retrieved by the baseline attribution method and the weighted counterpart. The results are shown in Figures 9 for GPT-2-small and in Figure 10 for LJama3-8B-Instruee Instruct.
D USAGE OF LARGE LANGUAGE MODELS
Following ICLR 2026 policy, hereby acknowledge that used LLMMs for proofreading and polishing the main text of this paper, and structuring our draft into formal sections B.2.2 B.2.3 and C.4 the Appendix.
27

---

Published as a conference paper at ICLR 2026
Query
August 31, the same night that the NK I Corps offensive rolled across the river.
Near the cnd of the month two recoanaisiance patrols from the 9th Infantry had crossed to the west side of the Naktong and observed North Korean tank and troup activity 2 miles (3.2 km) west of the river. Information obtained later indicated was in fact the command past of the NK 9th Division. On August 25, 9th Infantry commander Colomel John G. Hill outlined projected Operation Manchu, which was be company-size sized combat palrol cross the river, advance to the suspected North Korean command post and communications center, destroy it, cagture prisoners, and collect intelligence. The Infantry Regiment had planned Task Force Manchu on orders from the 2nd Division commander Major General Laurence B. Koiser, which in turn had received instructions from Eighth United Stares Army commander Lieutenant General Walton Walker for aggressive patrolling. Keiser decided the patrol should cross the river at the Packchin ferry. The 9th Infantry reserve, E Company, reinforced with section of light machine guns from II Company, Battalion, was to Imansprrt it across the river in assault boats the night of August 31, Two heavy weapons companies, and H. were each fornish section of heavy machine guns, section of 81-mm. mortars, and one section 75-mm. recoilless rifles for supporting fires. A platoon of 4,2 4.2-inch mortars was also to give support.
to be the allack force. The 1st Platoon, 2nd Engineer Combat
After dark on August 31, First Licutenant Charles I. Caldwell of D Company and First Licutenant Edward Schmitt of H Company, 9th Infantry, moved their men and weapons to the base of Hill 209, which was within B Company defense sector and overlooked the Puekchin lerry crossing of the Nakiong River. The raiding force. Company, was still its regimental reserve position about 2 miles (3.2 km) west Yomgsan, getting ready with the engineer platoon to move to the crossing site. Colonel Hill went forward the evening with the 4, 2-2-ncch mortar platoon to its position at the base of Hill 209 where the mortarmen prepared to set up their wespons. By 21:00, closest front line unit was B Company top of
Top-1 Retrieved by Baseline EKFAC
Composition =
Glen Boyd of Blogeritics said that the album has Deep-uss bass lines, old school funk samples, and plenty of street smart ghetlo albitude are what powers this record. Jerry Heller wrote that Eary raps more up frant on the alhum than he does on Siriggh Outta Compton, and insists that the album 's lyrics contain
sexual humor than gangsta vibe.
The album title track and lead single" Eazy-Duz-It written by MC Ren. opens with a woman acclaiming Eazy-E 's style. Eazy then outta here. This followed by a bass provided by Dr. Dre. Soon, Eazy
interrupts saying Bitch shut fuck up, get begins to rap about himself and things that be does. The song declares that Eazy is a hardcore villain who collects money from his prostitutes, and feels great when his pockets are lat. Thhe chorus, repeated three times, states that be "is a gangsta having (un " The piece is laden with the aural mainstays of gangsta rap, including gunshots, and references several drugs.
Boyz n the Hood was written hy Tee Cuhe, with some contribution by Fary- The song about growing up in Compton, California, and describes the gangster lifestyle. conceives the" ghetto landscape generalized abstract construct [ and | also introduces localized muance that conveys certain proximity. effectively capturing narrowed sense of place through which young thugs and their potential crime victims move tandem. as put by cultural historian Murray Forman.
" Noo More ? a is similar to Boyz the Hood in its theme. The piece begins with interview between Eazy and a femalo journalist. who asks about his childhooo. Easy explains (in verse) that be was ruthless, in gang. " "specialized in gankin, (loosely, to steal from) and had respect for roles. He then asked he has been an armed robhery. He responds, You mean 211 ? "The following verses tell of Eary 's exploits as thief and thug.
Reception
=== = Commercial perforaace===
Top-1 Retrieved by Weighted EKFAC (Ours)
1935 Fichmann married Veronika (Vera) Licbl (1909 93). The couple had four Klaus (b. 1936 in Berlin), Horst Adolf (b. 1940 in Vicnna). Dicter Helmut (b. 1942 in Prague) and Ricardo Francisco (b. 1955 in Bucnos Aires). Eichmann was promoted to SS- SS-Haupsschar/lheer (head squad leader) 1936 was commissioned as an SS- Untersturmfüürrr (sccond lieutenant) following year.
Nazi Germany used violence and economic pressure to encourge Jews leave Germany of their own volition; around 250.000 the country 437,000 Jews emigrated between 1933 and 1939, Eichnunn travelled to British Mandulory Pulestine with his superior Herbert Hagen in 1937 to assess the possibility of Germany 's Jews voluntarily emigrating to that country, disemharking with forged press credentials at Haifa, whenee they travelled Cairo Egypt. There they Feival Polkes, an agent of the Haganah, with whom they were unable to strike a deal. Polkes suggested that more Jews should be allowed to leave under the temms of the Haavara Agreement. bui Hagen refused, surmisiig that a strong Jewish presence Palestine might lead their founding an independent state, which would nun contrary to Reich policy. Fichmann and Hagen attempted in return to Palestine few days later, but were denied entry after the British authorities refused them the required visas. They prepared a report on their visit, which was published in 1982.
In 1938, Fichmarn was posted Vienna help organise Jewish emigration from Austria, which had just been integrated into the Reich through the Anschluss. Jewish community organisations were placed under supervision of the SD and tasked with encournging and facilitating Jewish emigration. Funding came from moncy seized from other Jewish people and organisations, well donations from overseus, which were placed under SD control. Eichmann was promoted to SS OS-OObesttrmffhher (fisst lieutenant) in July 1938, and appointed to the Central Agency for Jewish Emigration Vienna, created in August. By time 100,000 Jews had left Austria legally, and many more had heen smuggled nut to Palestine and elsewhere. = Second World War
Viennu May 1939, nearly
Figure 8: Qualitative Analysis for GPT-2-small. Comparison of the most influential training example retrieved by the baseline EKFAC and our weighted EKFAC. The weighted method retrieves a training example that is semantically more relevant to the query.
28

---

Published as a conference paper at ICLR 2026
Query
in the castern provinces of Rome and with the Parthians as well. The peace between Parthia and Rome lasted 50 years until Emperor
Trajan of Rome invaded Armenia in 114.
= Other mujor power struggles and rebellions
The war with Parthia was not Nero 's only major war but be was hoth criticized and praised for an aversion to battle. Like many emperors, Nero faced number of rebellions and power struggles within empire.
British Revolt of 60 61 (Boudica 's Uprising)
province of Britannia. While governor Gaius Suctonius Puulinus his troops were busy capturing the island of Mona (Anglesey) from the druids, the tribes of the southeast staged revolt led by queen Boudica of the Iceni. Boudica and her troops destroyed three cities before the amm of Pauliaus could return. receive reinforcemeats, and quell the rebellion in 61. Fearing Paulinus himself would provoke further rebellion. Nero replaced him with the passive Publius Petronius Turpilianus. The Pisomian Conspiracy of 65
a major rehellion broke out
65, Gajus Calpumius Piso, Roman statesman, organized a conspiracy against Nero with the help of Subrius Flavus and Sulpicius Asper, trihune and centurion of the Pmettriann Guard. According to Tacitus, many comspirators wished to rescue the state" from the emperor and restore the Repablic. The freedman Milichus discovered conspiracy and reported Nero secretary, Epaphroditos. As a result, the conspiracy failed and its members were exccuted including Lacan, the poet. Nero 's previous advisor, Senoca was ordered to commit suicide after admitting he discussed the plot with the comspirators.
The First Jewish War of 66 70
66, there was a Jewish revolt in Judea stemming from Greek and Jewish religious tension. In 67, Nero dispatched Vespasian to restore order. This revolt
eventually put down 70, after Nero death. This revolt famous for Romans breaching the walls of Jerusalem and destroying the Second Temple of Jerisalem.
=== = The revolt of Vindex and Galba and the death of Nero===
In March 68, Gaius
Top-1 Retrieved by Baseline EKFAC
Lestie Fuller. Nikki Webber, and Terrence McDonnell. For its first seasons the syndicated version had Deirdre Cossman for its manuging producer, then Dennis F. McMahon became producer for the next two seasons (joined by Dominique Bruballa as his line after which Jennifer Weeks produced the next four seasons of syndicated Millionaire shows, initially accompanied by Amanda Zucker as her line producer. but later joined for the 2008- - 09 scason by Tommy Cody (who became sole producer in the 2009 - 10 season). The lirst 65 shullle format episodes were produced by McPaul Smith, and of 2011, Lasseter. The network version had Ann Millr and Tiftany Trigg for its supervising producers; they were joined by Wendy Roch in the first two seasons, and by Michael Binkow in the third and linal season. After Rubino 's promotion to co-executive producer. the syndiculed version later supervising producers included Sirop (2004 09), Geena Ginizig (2009 10), Brent Burnette (2010- 12), Geoff Rosen (2012 14), and Liz Harris (2014 16).
producer is beld by Bryan
The original network version of Millionaire
syndicated version consulting producer for its first two seasons, and then as the director of Duel, which ran on ABC from Devember 2007 to July 2008. The syudicated version was directed by Mathhew Coben from 2002 2010, by Rob George from 2010 2013, and by Briun McAloon 2013 season. Former Price is Right director Rich DiPiroo became Millionaire 's directer 2014.
directed by Mark Geatile, who loter served
== Production
The U.S. version of Millionaire is co- production 2waytraffic, division of Sony Pictures Entertainment, and Valleyerest Productions, a division of The Walt Disney Company. 2waytraffic purchased Millionaire 's original production company Celador in 2008, while Valleyerest has produced the series since its boginning. and holds copyright on all U.S. Millionaire episedes date. The show distributed by Valleycrest 's corporate sibling Disney- - ABC Domestic Television (previously known as Buena Vista Television). The U.S. Millionaire was taped at ABC Television Center East studio on the Upper West Side Manhattan
Top-1 Retrieved by Weighted EKFAC (Ours)
house arrest, he signed a plea bargain for minor offenses, connected to hringing in foreign currency without a permit, and was sentenced community service and indicial supervision three years. In view the Abergil brothers extradition to the US, he and his wife were arrested for violating immigration laws. is believed that American authorities tried to apply pressure to Ben Harosh to incriminate the Abergil brothers failed.
Hai Vaknin, called an Abergil henchman " hy the Ismell duily Hauretz, was arrested in the USA in 2006. In January 2011, he signed plea bargain, confessing to money laundering and receiving 57-mooth jail sentence, which had already served, and was ordered to three years under supervised releuse. Ilis description of loans and extortion practices expected to help convict Itchak and Meir Abergil.
early August 2008, Itzhak and Meir Abergil were arrested on suspicion of involvement in the murder of Margarita Lautio who had died after being mistakenly sbot during a failed assassnaation attempt by members of the Abergil mob. On August 2008 lizhak Meir Abergil along Moshe Malul
Israel Ozifa were brought before a Jerusalem Magistrate Court judge for their alleged role in the killing of Israeli drug dealer Sami Atias in Eacino in August 2003, as revenge for his allegedly having stolen money from them. They were remanded in custody together with Sason Barashi as result of a request by law enforeement the United States for their extradition. The indictment includes four different crimes that are attributed Yitzhak Abergil: involvement togother with Malul the murder of Atias in California in 2003, trade in Eestasy, extortion and violence against businessmen, and money laundering and fraud. The inquiry into case bad lasted more than ten countries in America, Europe, Asia and the Middle East.
years, involving FBI, tax authoritics and law enforcement officials
July 2009, Jerusalem District Court approved the state' request to extradite Ahergils, Sasson Barnshy, Moshe Malul and Israel Ozifa to the US. In December 2010, the Supreme Court rejected an appeal by the three associates of the Abergil brothers.
On January 12, 2011 ltzhak and Meir Abergil, together with
Figure 9: Qualitative Analysis for GPT-2-small. Comparison of the most influential training example retrieved by the baseline EKFAC and our weighted EKFAC. The weighted method retrieves a training example that is semantically more relevant to the query.
29

---

Published as a conference paper at ICLR 2026
Query
is often more sophisticated than that used by other coo-extremist groups In summary. while the ALF has history of violent direct action. they have also employed tacties like sabotage. property damage, and animal liberation achieve their goals. The group's modus operandi includes recomnsisaanee, surveillance, and careful planning before carrying out an operation. Their actions are typically targeted at specific industries or individuals perceived as responsible for harm to animals
eavironment. The ALF's willingnasss use violence. combined with their resourcefulness and ability adapt, makes them a significani threal to law enforcement and private security agencies. #e# Earth Liheration Front (ELFF
The ELF is another cco-extremist group known for its radical anti-cnvironmectal activities. Founded in the 1990s, the ELF is believed to be a spin-ofI of the ALF. but some experts argue that they may be sepurate entities. Like the ALF, the ELF advocates for the protection of the envirumment and wildlife through direct action, incloting arson sabolage, and property destruction.
The EI. has heen linked to numerous incidents across North America, Earope, and Australia, including arsons, hombings, and vandalism targeting industrics such as logsing. mining, and biotcchnology. Some notable examples include:
1. August 2001: An ELF cell claimed responsibility for setting fire to a University of Washington research center, which housed genetic engincering lah.
2. October 2002: suspected ELF cell set ablaze timber company's 's office bailding in Oregon, causing $10 million in damages.
3. June 2015: An ELF cell was accused of torching a fossil luel pipeline construction site in Pennsylvamia.
Like the ALF, the ELF uses various tacties evade detection, including using coded language, encrypting messages, and employing secure communncatton chamnels. They often claim responsibility for their actions through anonymous communiques sent to media outlets and online forums.
Key characteristicss of Fl F operations include:
Taaggetdd industry disruption: The ELF focuses on discupting spocific industries doemed harmful the environment. such as logging.
mining. and fossil fuels.
Use of violence and intimidation: The ELF employs violent meuns, such us arson and bombing, to convey their mewsage and
Top-1 Retrieved by Baseline LoGRA
a bad left her crutches at home that ovening, because her friends were ablc to help her walk. Her logs have bocn partially paralyzed since Jan, 25, 2014, the third anniversary of the revolution, when security forces shot her on that evening, along with her two friends. When they left the restaurant, several police officers dressed in civilian clothing pulled them into a minivan. The family had no idea what had happened, and there was no news from her for two weeks. Esraa's 's lather went on television to usk the public for help. They contacted an atlorney. who filed complaint with the attorey general. "Tell me where she and we will investigate the case," the general prosecutor's is office replied, suggesting that perhaps she had run off with a lover. was only by accident that the family learned what had happened Esraa. young woman who had seen her prison contacted the family via Facebook.Perfidious Perfidious PacticsDusa Taweel says she talks in her sleep at night, constantly repeating the number of her sister's 's missing person report: 1991. The day before our interview, she saw her sister again for the first time since her disappearance, but only from distance. beyond barrier in front of the building that houses the prosecutor general's office. Esraa's deteation was extended by 14 days. but the same thing has been happening every two weeks. She is churged with being a member of the Muslim Brotherhood, and of having disseminated false reports and provided information other countries. "Those the top three charges if you to arrest someroe these days,' says her attorney, Halem Henish. El-Taweel denies them all. Henish describes the enforced disappearances as a perfidious tactic, because "it allows the government hide people from the law." person who officially arrested cannot interrogated without an attorney present. The person's 's case would have to be presented to the public prosecutor within 24 hours, and the government would have to release him initially taken a building owned the state security service. "Confessions are forcibly extracted from them there." " sometimes through torture, Henish explains. Then an indictment is prepared. According to ber allomey, Esram El Taweel was interrogated for 18 hours, and
back during a demonstration She disappeared
if there were indictment, says Henish.
"disappeared" are stuck in a legal vacuum sorts. They are
Top-1 Retrieved by Weighted LoGRA (Ours)
in the wake of past attacks and many ALF operatives have diverted their efforts toward the homes of exocutives and researchers (like the UC-Santa Cruz researcers)) and other soll lurgets. Gravilating loward soller lurgets makes less likely operatives will caught in the act. Additionally, the surveillance tradecrafi utilized by the ALF and its operatives and the operational security they practice is usually better than that demonstrated by jihadist lone wolves. Organizations such as the Ruckus Society conduct detailed courses on preoperational surveillance, which called "scouting" in their parlance. Alsa, since ELF/AL FLFivLFactivisss tend to he young Caucasians, they are generally not viewed as potential threat, even they are spotted conducting surveillance. Moreover, since these activists have focused
attacks that cause property damage. law enforeement has understandably not placed the same priority catching ELFALF activists as it has other actors such as jihadists, who intentionally target people. In Bond's case, be might have had some difliculty not drawing atlention himsell as cased leather stores fuie gras restaurants because with the word "vegan" tattoned across his throat in large block letters flanked on either side by crossed wrenches. "Monkey wrenching" is a lerm widely used by activists associated with ELFALF and amarchist destrction such
talloos covering half
to refer to direci-action allacks that involve property Anyone involved in animal research or selling animal products who is observant enough would surely look suspiciously upon person with such distinctive markings. When all of these factors combine, it is usually very difficult to solve an ELF/ALF arson bombing case uniess mistake is made, or confidential informant comes forward. Most successful prosecutions such cases have come as a result of informants, and because of this we have wimessse cat-and mouse game between activists and the government regarding informants, government giving them permission to do so. A Although the CI the Bond case was just an acquaintance of Bond who was concerned about his arson activities and not a person specifically dispatched to peneirate the movement, without the help of the CI. the government probably had very little chance of identifying Bond. Animal rights blogs and websites have already begun dissecting the Bond case and providing lessons learned to current (and aspiring) animal rights activists. Many of these sites have focused on Bond's mistake in confiding
activist groups pressing informants to commit illegal activities hefore heing accepted and
and have indicated that they believe the informant
Figure 10:
Qualitative Analysis for (lama3-8B-Instruee Comparison of the most influential training example retrieved by the baseline LoGRA A and our weighted LoGRA. The weighted method retrieves a training example that is semantically more relevant to the query.
30

---

Published as a conference paper at ICLR 2026
Query
boping to improve their chances winning a World Series titl. The Rod Sox did just that before the 2013 scason, adding Shane Victorino, Mike Napoli, and Jonay Gomes
stready strong core. They won the division and eventually the World Series. But other leams made similar moves and didn't it have the same success. The Yankees signed Kevin Youkilis, Travis Hufner, and Vernon Wells, but missed the playoffs entirely. The Dodgers truded for Hanley Ramirez and Adrian Gonzalez, but they eliminated in the National League Championship Scries. There's Spore'soe no guarantce that making big moves will lead to a successful season.
Thee importance starting pitching can't be overstated Surting pitchers are often most critical componens team's rotation. Red Sox had strong group last year, with Clay Buchholz, Jun Lester, and John Lackey all having solid seasons. The Tigers' Justin Verlander and Max Scherzer formed one of the best 1-2 punches bascball, while the Cardinals' Adam Wainwright and Lance Lynn were equally impressive. strong starting rolation carry leam through lough stretches and give them chance to win the postseuson. Rallpens are hecomnng increasingly important. In recent years, hullpens have become more specialized and crucial to a team's is success. strength throughout the season, with closers Koji Uchara and Craig Breslow heing key contributors. Other
The Red Sox's 's bullpen tcams like the Pirates and Athletics also relied heavily on their pen to get outs and secure victorics. As the game becomes more focused on matchups and leverage situations, the
the relief pitcher only going to continue grow.
4. Defense matlers, especially in the postseason. The Cardinals" defense was major factor in their World Series victory, they turned double plays and made crucial plays in the field to snuff out rallies. The Tigers" defense was similarly stingy, with Miguel Cabrera and Prince Fielder forming formidable 3-4 combination the plate. In the posisenoo, every play counts, and team's ability to delend its territory can he the difference between advancing or going home
Top-1 Retrieved by Baseline LoGRA
of how machine learning algorithms work can help provide understanding as to what surt of questions machine learning can help to answer, and sort of questions are problematic. Secondly, I hope this series encourages some you dig deeper, to learn more about this topic. Machine learning rapidly growing field that is expanding to every aspect of life. This includes, recommendation engines on websites, astronomy where it helps identify stars and planets, the phurmaceutical industry where is being used to prediet which molecular structures that are likely produce aseful drigs, and mayhe most famously, trinning self-driving cars drive in the real world. Whatever your primary interest, there is likely to be some machine learning applications being developed or being used already.11| |I| There are range of metrics that can be to this. available metrics in the Seskit Learn package, see Full scriptcimport pandas as pd import numpy as import sghoost as xgb from skleam import cross, valossaaiiation decompusition, grid search from sklearn prepmeessing import LabelFacoder #han www.e ww wwwp://wwww.....................
Functions
Remove outliers def remove outliersddf, column, min_ min_val, vall: col values dffccolumn] di[column] up. whwrotnp. logicallooooooo logical_ortcol, values<<mmin val. col_ valuuuuuussaaa val), np.NaN, col values) return # Home made One Hot Encoder del convert_ to_ binary(df. column converty; categories list to_ po__ooveet).drpp duplicates()) cat name = str(category) (efccategrry...ppppeeeeee replace(
oog_oaeee..gggdddiiicccee))))))e for calegory in categories: "repplace("" ((", "")xreplacee("" ) "'Jreplace(""", "_ ").replace(""" "'.lwwer() col, name col- umn_ to cc_ovverr::]] + cat_ name[:10] dffeol_name]] = dfflee[(fffcoommm to _ccoveetet m= category), col, __nmm]] return Count occurrences of value in columa def convert ca___oo__cuus(ddd id_col, column convert): id Jist dffid_ooll) dupllicdde)) of counts doofo[[[id col, column tn convent]] df counts| count') ] 1 df counts = df counts grouphytby-iidd coll, column to convert). convert, values= 'count') of
index-Faaas., sort=False) sum) new_df = df counts cou.tts.piottiinee pivotindex=id col. columns-column new di dillaa))) # Rename Columns categories = list(d]] column_ to_ converttt drop duplliiccaatss() for category in categories: cat name stricategory) (eagggorrryrrppeeeeeee. )replace("" ".
Top-1 Retrieved by Weighted LoGRA (Ours)
When it's all over, we media know-it-alls tally everything up. then declare with breathtaking confidence who won and who lost, The problem is, we'ne doing wrong. Signing the highest priced free agent great, but it might get you far you neglect other roster weaknesses to get The 2013 Angels were a perfect example of this phenomenon. When the Halos speat $125 million on Josh Hamilton, was hard look at that team see any thing other
Mike Trout, Albert Pojols, Hamilton laying waste to American League pitching. Problem was, their own pitching stunk. The top of the rotation was functional, if nothing special, with C.J. Wilson, 24 starts from Jered Weaver, Joe Blanton werc a nuclear disaster, and the bullpen was the sixth-worst in baseball by anding-independent measures. This isn't 't one-year trend, eitber. One year carlier. the Angels (along with the Marlins) landed some of the biggest names on the market (Pujols. Wilson, Juse Reyes, Mark Buchrle) only disappoint the following season.2. 2. Bargain huming can and does often pay Francisocs Liriano. Russell Martin. Bartolo Colon. James Loney. Koji Lecaar.. Marlon Byrd The list of low-priced free agents who delivered big returns for their teams in 2013 a long one. Each of these players was acquired cheaply because he hud a penceived defect. The Pirates got Martin cheap hecause he hit just 211 in 2012 even though he hopped 21 homers, played his usual strong defense, and was just 29 years old. Loney a catastrophic 299//293/3336
24 more from Jason Vargus.
the 58 starts made by Jerome Williams, Tommy Hanson, and especially
2012, but he was just 28 years old, played excellent defense, and had pat up some docent offensive scasons, albeit without the power you'd 'd hope for from a slugging-heaacy position like first base. Liriano and Uehara had injury histories, though both had shown flashes of
Figure 11:
Qualitative Analysis for Llama3-8B, ---nna---- Comparison of the most influential training example retrieved by the baseline LoGRA and our weighted LoGRA. The weighted method retrieves a training example that is semantically more relevant to the query.
31


Reviews:
[{"review_id": "l0zbI84hXE", "reviewer": "Reviewer_6Ser", "summary": "This paper studies the task of training data attribution using influence functions. Specifically, they analyze the varying effects of attribution across the parameter groups in a neural network. The first part of the paper shows this variance empirically, and the second part focuses on learning weights for individual parameter groups. Authors propose a self-supervised methodology to learn these weights. They use a baseline attribution method to provide pseudo ground-truth training examples.\n\nExperiments span ImageNet classification, image generation and language modeling. Results show that the proposed weighting scheme helps improve attribution across a range of attribution methods (TracIn, TRAK, LoGRA, EKFAC). Additionally, the authors also curate a synthetic Pokemon dataset with images by varying subject, style and background. Results show strong results with attributing style, with moderate (/poor) results with subject and background.\n\nOverall, the idea that different parameter groups have different effects on attribution is intuitive. Prior work makes ad-hoc assumptions, and this paper studies this behavior in a more principled fashion. The main concerns are that the paper mostly explored small-sized models and doesn't include any qualitative analysis on where they find gains over the baseline. For instance, does weighting parameter groups instead of pre-selection help improve attribution in LM generated text?", "questions": "A few additional questions and comments,\n\n- I would update the lines 218-219 to briefly mention the analysis from C.1. This could strengthen your argument in the main text.\n- Why does the proposed method show better gains with some attribution methods? For instance, the gains are higher with TRAK and EKFAC and smaller with LoGRA in Table 2. Do you have a hypothesis for these differences?\n- Any reason for not including TrackStar (line 127) in your experiments?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- This work presents a methodology to automatically learn weight model parameter groups on their attribution abilities. Prior work in attribution using influence functions focuses on a predefined set of parameter groups. This work presents a more principled solution for this.\n- Experiments include comparison against multiple strong attribution baselines.\n- The analysis of involving subject, style and background variations in synthetic dataset shows strong results.", "weaknesses": "- The language modeling experiments use GPT-2 small and its unclear to me if the gains will necessarily transfer to larger LMs. As the authors highlight in section 2, recent methods (LoGRA, TrackStar) have improved efficiency. Any reason for not experimenting with larger models in this work?\n- The paper could benefit from analyzing its relative performance gains across different baseline attribution methods. For instance, they show strong results with TracIn but moderate results with recent methods.\n- The paper could also benefit from qualitative analysis of their method for language modeling experiments; this would be an extension to their analysis on the synthetic Pokemon dataset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jqR0WPFZj2", "reviewer": "Reviewer_uSNX", "summary": "This paper addresses the problem of data attribution, which aims to identify the training samples most responsible for a model’s prediction. While existing gradient-based methods (e.g., TracIn, TRAK, Influence Functions) treat all parameters equally, this paper observes that attribution quality varies significantly across different parameter groups.   To tackle this, the authors propose a learnable parameter-weighted attribution framework, introducing explicit, non-negative group weights to scale gradient contributions. Extensive experiments on image classification (ResNet-18, ViT-B/16), language modeling (GPT-2), and diffusion models (Stable Diffusion, D-TRAK, DAS) show consistent improvements across metrics like LDS, mislabeled-data AUC, and tail-patch score.", "questions": "1. How sensitive is the learned weighting to the quality of the initial pseudo-ranking? Would a noisy baseline still lead to meaningful weights?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Identifying parameter heterogeneity as a fundamental but overlooked factor in data attribution.\n2. Generalizes across TracIn, TRAK, DAS, and others with a single weighting formulation.\n3. Strong, consistent results across image, text.\n4. Learned weights provide semantic insights into layer-level specialization", "weaknesses": "1. Technically, TRAK and similar Hessian-based methods already introduce an implicit parameter weighting through the approximation of $H^{-1}$, which scales gradients by local curvature. The proposed explicit weighting can thus be viewed as learning functional heterogeneity on top of curvature-based scaling.\n\n2. The self-supervised loss bootstraps from existing methods (e.g., TRAK), so its ultimate accuracy may inherit their biases.\n\n3.  Equation (6) defines the weighted attribution as  \n  $\\tilde{\\tau}(x_q, x_i; w) = g(x_q)^\\top \\mathrm{Diag}(w)K g(x_i),$ but shouldn't it be $\\tilde{\\tau}(x_q, x_i; w) = g(x_q)^\\top \\mathrm{Diag}(w^2) Kg(x_i),$? which corresponds to reweighting both query and training gradients equally.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "twsOvHd9rO", "reviewer": "Reviewer_tnbV", "summary": "The paper is motivated by the observation that network parameters are not uniformly contribute to the model behavior, and thus the data attribution process. The paper propose a novel way to choose the weight of each parameter. The new method shows good effectiveness on traditional evaluation methods as well as provide more fine-grained attribution of concepts.", "questions": "- A core question could be: what is the essential reason that makes the hessian based parameter reweighing does not work well?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The paper propose an interesting question (and opportunity) to improve the effectiveness of data attribution. This direction is discussed but not checked in depth in previous literature.\n- The self-supervised mechanism to learn parameter weight is practical and easy to use.\n- The improvement over standard data attribution methods are obvious.", "weaknesses": "- The main weakness lies in the self-supervised weight learning loss design. The analysis use a signal-to-noise ratio model to see the attribution score and try to optimize the parameter weight to get highest signal-to-noise.\n  - Problem is that the definition of signal is related to the top-k attribution score. The decision is not justified well in Section 4.2 as well as in Appendix A.\n  - Intuitively, top-k attribution score is very important (to the counterfactual prediction), while the least-k attribution score is also important and affect the counterfactual prediction. Why only choose top-k could be stated more carefully in the paper.\n\n- The fine-grained attribution of concepts is somehow weak in both motivation, method design and experiment results\n  - In Section 3.3, the motivation of revising parameter weight for concept attribution use similarity as the groundtruth, which is not a good practice for data attribution. Since how does the model learn a data may be different from the intuition that \"similar data helps most\".\n  - The method design relies on the query data's distribution, could this also work for uniform/hessian based parameter reweighing?\n  - Figure 4 shows some \"imperfect\" attribution results: e.g., style No.5 (not in the watercolor style), background No.2&5 (not with the same background)...", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0WPrvj2TVK", "reviewer": "Reviewer_4e32", "summary": "Many methods for data attribution consider model parameters uniformly, as if all parameters contribute equally. While influence functions do consider how some directions in parameter-space matter more than others, the true Hessian is intractable in practice and only subsets of parameters are used. The authors show that attribution quality is not uniform across parameters, and propose a method that learns parameter group weights from data.", "questions": "Is maximizing the SNR of an attribution score a form of \"metric hacking\" of the LDS score? Why or why not? As a simple example, suppose we were measuring RMSE for some task. If one wanted to get a lower one could optimize to reduce variance and this would produce lower RMSE but not necessarily be better at the task (this is what I mean by metric hacking)", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "To my knowledge, this work's exploration of how attribution is impacted by the heterogeneous nature of parameters (that some parameters are more important than others for particular tasks) is the first of its kind. It brings up some new findings, which concur with pruning work:\n1. up blocks exhibit higher LDS attribution scores than others. In pruning, deeper computation is also more high-level/hierarchical\n2. there is significant variation in terms of how much each parameter (and even each parameter group) affect attribution. Without variation in parameter importance in the context of pruning, we wouldn't be able to prune large fractions without affecting accuracy proportionally.\n\nSeparately, using weights from section 4.2, their method improves attribution (LDS score) of TracIn and TRAK for CNN and ViT. On text data (WikiText-103 w/ GPT-2-small) LDS score also improves for all methods tested (TracIn, TRAK, LoGRA, EKFAC). \n\nAppreciate the extra detail in appendix motivating Eq. 6, which describes how minimizing their loss equates to maximizing the SNR of the attribution score of the existing base attribution method.", "weaknesses": "Main issue: The authors do not specify (or at least do not make clear) what data is used to learn the weights from section 4.2. In section 5.1 they state \"weights are learned using samples that are not in the training or test data\", but what are that data? In the first imagenet section 5.1.1, there is no specification as to the data that is used, and that continues for the following sections on other data modalities. More specifically, in section 5.1.1, lines 372-376 are on how they train the models and how they evaluate LDS. Lines 388-396 are about evaluating mislabeled data detection. This section misses the most important part: what data is used for $w$ (the column that brings the improvement)? Without information on the above, the improvement could be due to a bit of metric hacking of LDS score. \n\nSecond issue: The section on Finegrained data attribution focuses solely on Recall@10. Under this task, I feel simple baselines like using feature embeddings from pretrained models [1] should be tried. Prior work like Datamodels also do this and call it \"representation distance\" (See Section F.3.1 Estimation using baselines of [2])\n\n[1] Singla et al., 2023. \"A Simple and Efficient Baseline for Data Attribution on Images\" (https://arxiv.org/abs/2311.03386)\n\n[2] Ilyas et al., 2022. \"Datamodels: Predicting Predictions from Training Data\" (https://arxiv.org/abs/2202.00622)\n\nWilling to raise score if weaknesses are corrected and questions addressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)