Fast Data Attribution for Text-to-Image Models
Authors: Sheng-Yu Wang
Aaron Hertzmann
Alexei A Efros
Richard Zhang
Jun-Yan Zhu
Venue: NeurIPS 2025 poster ()
Keywords: Data Attribution
Influence Estimation
Text-to-Image models
Image Retrieval
Primary area: deep_learning
TL;DR: We distill a slow, unlearning-based data attribution method to a feature embedding space for efficient retrieval of highly influential training images.

Abstract: Data attribution for text-to-image models aims to identify the training images that most significantly influenced a generated output. Existing attribution methods involve considerable computational resources for each query, making them impractical for real-world applications.  We propose a novel approach for scalable and efficient data attribution. Our key idea is to distill a slow, unlearning-based attribution method to a feature embedding space for efficient retrieval of highly influential training images. During deployment, combined with efficient indexing and search methods, our method successfully finds highly influential images without running expensive attribution algorithms. 
We show extensive results on both medium-scale models trained on MSCOCO and large-scale Stable Diffusion models trained on LAION, demonstrating that our method can achieve better or competitive performance in a few seconds, faster than existing methods by 2,500x - 400,000x.  Our work represents a meaningful step towards the large-scale application of data attribution methods on real-world models such as Stable Diffusion.

Fast Data Attribution for Text-to-lmage Models
Sheng-Yo Yu Wang' Aaron Hertzmann
Alexei A. Efros
Richard Zhang²
Jun-Yan Zhu
CCrneeiie Mellon University
Adobe Research
UC Berkeley
Abstract
Data attribution for text-to-image models aims to identify the training images that most significantly influenced generated output. Existing attribution methods involve considerable computational resources for each query, making them imprac- tical for real-world applications. We propose novel approach for scalable and efficient data attribution. Our key idea to distill slow, unlearning-bant tion method to feature embedding space for efficient retrieval of highly influential training images. During deployment, combined with efficient indexing and search methods, our method successfully finds highly influential images without running expensive attribution algorithms. We show extensive results on both medium-scadd models trained on MSCOCO and large-scale Stable Diffusion models trained on LAION, demonstrating that our method can achieve better or competitive perfor- mance in a few seconds, faster than existing methods by 2,500 400,000x. Our work represents meaningful step towards the large-scale application of data attribution methods real-world models such as Stable Diffusion.
attribu-
Introduction
457 457.000x speedup
Data attribution for text-to-in text-to-image models 3 5
4
~2,500x speedup
10 aims to identify training im- ages that most significantly influence generated images. For given output, influence is de- fined counterfactually: if influential images were removed from the training dataset, the model would lose its ability to generate the output. However, directly identifying influential images by retraining models with all possible training subsets computationally infeasible.
6
g
6.0
ADÚ
Generation speed
(1.988
Attribution (ALoss)
5.5
(439MB)
O-TRAK (30GB)
Durs (354MB)
5.0
(57MMB)
(354MB)
(232MB]e
(232MB)*
10a
To make attribution more tractable, researchers have proposed several approaches to approxi- mate this process. One common strategy is to estimate individual influence scores by ex- amining the effect of removing single training images. tradient-based methods, such as in- fluence functions 121 [33, approximate this removal process by calculating the inner product of the training and test images' model gradients, reweighted by the inverse Hessian. Unfortu- nately, computing and storing these gradients per training image is costly. Projecting gradients into lower dimensions reduces accuracy further [1], while still being computationally prohibitive for mod- els with billions of parameters [15] Another approach approximates influence by directly "unlearning" individual images, which improves accuracy but significantly increases runtime [6. 17). None of the above methods is practical for applications requiring fast test time performance. Furthermore.
Throughput (queries/sec)
Figure I: Attribution performance vs. through- put. Previous methods (AbU [ D-TRAK (2]) offer high attribution performance but computa- tionally expensive for deployment. Fast image sim- ilarity using off-the-shelf features (DINO) lacks attribution accuracy. We distill slower attribution methods into feature space that retains attribution performance while enabling fast deployment.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

while text-to- to--mage platforms typically charge users 5-10 cents per image [8 19 existing attribution methods could cost way more per query due to heavier computation needs. This significant cost disparity, costing orders of magnitude more than generation revenue, prevents real-world deployment.
In this work, propose new method to overcome the tradeoff between computational complexity and attribution performance. Our key idea is to distill a slow, unlearning-basee attribution method feature embedding space that enables efficient retrieval of highly influential training images. More concretely, leveraging off-the-shelf text and image encoders, we learn our embedding model's weights via learning-to-11 to-rank, supervised by the unlearning- based attribution method [ During deployment, our method scales to hundreds of millions of training images at a low cost.
However, this approach introduces several technical challenges. First, data curation is computationally intensive - collecting influence scores across large datasets (e.g.. LAION-400M (20) consumes significant GPU hours. To address this, we adopt a two-stage retrieval strategy: rapidly indexing [21 to select top candidates and then reranking them with more precise attribution methods. We show that our embedding function can be effectively learned from these reranked subsets. Second, designing objective functions to accurately predict dense rankings is non-trivial, as most learning-to-rant literature focuses on ranking fewer items. After evaluating several learning-to-rank objectives, we develop a simplified yet effective loss function.
We validate method through extensive experiments. First, benchmark approach on MSCOCO using counterfactual evaluation, showing that our learned feature embeddings achieve strong attribution performance. We further compare runtime and storage costs against recent methods, as shown in Figure | performance. Finally, we apply our approach to Stable Diffusion, achieving superior predictive performance on held-out test data. Our code, models, and datasets are at: htttp::/pptterwang51/ github. io/ iaabbbtGD// FastGDA
demonstrating the best tradeoff between computational complexity and
In summary, our contributions are:
A new scalable approach to data attribution that employs a leaming-to-rank method learn features related to attribution tasks.
A systematic study of key components that ensure efficient and effective learning-to-rane including tailored objective functions and a two-stage data curation strategy.
Extensive benchmarking against existing methods, demonstrating superior performance in both computational efficiency and attribution accuracy.
The first successful application and evaluation of an attribution method on large-scale model such as Stable Diffusion trained on LAION.
2 Related Works
Data Attribution.
The interplay between training data and models has been extensively studied
across various domains, including data selection 22 23
25
26
27
28
29
30
31
32), dataset
mixing 33 34 33|
36| 37 (8), and data valuation 39
46
47 In contrast, we
focus identifying training images influential to given generated output in text-to-image models. For classification tasks, prior methods typically average contributions over models retrained on random subsets 48
39
40
49 46), inspired by Shapley values (50), or estimate gradient similarity
across training checkpoints
52
12 are also widely used, as they require retraining or intermediate Influence functions
checkpoints. The main idea is to approximate changes in the model output loss after perturbing the training datapoint. This requires estimating gradients and inverse Hessian matrix of the model parameters. To make evaluating the Hessian tractable, previous methods explore inverse Hessian- vector products [12), Arnoldi iteration [3] and Gauss-Newton approximation 44 53 which is most commonly adopted for text-to-inn eext-to-image models for its efficiency.
15 55),
Despite these advances, challenges persist from high storage costs for model gradients across the dataset. To address this, researchers proposed several solutions, including estimating influence at test time [54] 1 55
16 17 without storing gradients at cost of increased runtime, and gradient or low-rank adaptors [5]. These methods introduce tradcoff between computational resources (storage, runtime) and attribution
dimensionality reduction through random projections
[44 53 2

---

performance, heavier approximations worsen attribution 22 J. To address this, we propose
learning feature space specifically trained predict attribution results obtained by computationally expensive but accurate methods.
Learning to Rank (LTR) aims to predict item rankings [57 and broadly categorized into pointwise, pairwise, and listwise approaches. Pointwise methods independently score each item, applying methods such multiclass classification [58), ordinal loss [59]. or regression [60]. Pairwise
methods are trained to predict relative orderings between pairs, leveraging boosting [6.. SVM [62]
and neural networks with cross-entropy 663 or rank-based losses [4] Listwise methods optimize
the entire list directly 665 66 67 68 aiming to improve ranking metrics such as NDCG [69], mAP, Precision @ k, and Recall @k 70). LTR benchmarks typically involve ranking fewer than 1,000 items [71 72.. In contrast, our task involves ranking significantly larger lists (e.g., 10,000 items). For simplicity, we focus on a pointwise method and propose a simple, effective variant that outperforms strong baselines in our application.
often trans- late across tasks [76 177 (8). For example, a representation learned from an image classification
Representation Learning. The representations learned in deep networks 773
75.
dataset [79] can be efficiently repurposed for tasks, from detecting objects
76
modeling human
perception [80] 81 Unsupervised [82 [33 or self-supervised learning [84 85
86
87
88) methods
aim learn image representations without text labels. A particularly effective training objective is
(93, where co-occuring data or two views of the same datapoint are associated together. in contrast to other unrelated data. 955 evaluate and tune representation towards attribution in the customization setting. In our work, we leverage counterfactually predictive unlearning methods and learn representations for general-purpose attribution.
the contrastive loss 89 90 91 92
(such as text and image CLIP
Previous work by Wang et al.
Method
Our goal is learn an attribution-speciiic feature using a collection of attribution examples, generated from an accurate but slow attribution method. Section 3.1 overviews the attribution algorithm we adopt and explains data collection process. In Section 3.2 we introduce our learning rank objective for training serriuttionspeccinn features.
3.1 Collecting Attribution Data
Following the convention from Wang et al. []] a text-to-image model 0g = A(D) is trained with learning algorithm A on dataset D = (xi, c;) where x and denote an image and its conditioning text, respectively. The model 0o generates a synthesized image & conditioned on caption c. To simplify notation, we denote synthesized pair as 2 (x, c) and training pair z, (x,, C.).
Data attribution aims attribute the generation attribution algorithm т, which assigns an attribution score 7(2, z;) to each training datapoint z,, with higher score indicating higher influence. We assume that r has access to all training data, parameters, and the loss function, but omit them for notational brevity.
its influential training data. We define data
Attribution by Unlearning. We collect attribution examples using Attribution by Unlearning (AbU) [] one of the leading attribution methods for text-to-image models, shown in the top of Figure 2 The method runs machine unlearning (96] 97 98 99] on the synthesized image and evaluates which of the training images are forgotten by proxy. The authors observe that the forgotten training images are more likely to be influential. Intuitively, this reverses the question: which training points must the model forget to unlearn the synthesized image?
Formally, to unlearn the synthesized image and evaluate which training images "forgotten", start with the pre-trained model lg. and apply certified unlearning [96] on the synthesized sample à:
V ()2 N
a
this creates unlearned model 0_ where ar is the step size, N is the size of the training set, and F is the Fisher information matrix. the training loss of text-to-image diffusion model, as defined in Appendix A.I This constitutes a one-step Newton update to maximize the loss of the synthesized
3

---

Attribution by Unlearning
(slow toacher)
Sort and compute ranks
Distill by Learning to rank
Unleam
Evaluate CN
Synthesized
Training
Croes entropy loss
Encoder
Learned
Embed
MLP
Fast Data Attribution (Ours)
$2
Figure 2: Our method. Given synthesized sample, data attribution aims find which elements in the training set are more influential. (Top) Attribution by Unlearning (AbU) is a slow but accurate method. It works by unlearning a synthesized example and evaluating the change in reconstruction loss on each training image, where each evaluation takes many forward passes. We generate AbU scores, using them train an cerrbbutoo-focusser embedding (bottom), so that attribution can be performed by fast similarity search, while retaining the accuracy of the slower AbU method.
sample through gradient ascent, regularized by elastic weight consolidation (EWC) loss 100 to retain other knowledge.
After unlcarning, the attribution score is assigned to each training sample by evaluating the training loss change, before and after unlearning:
T(2, 2) = -(),00))-
The loss change in r(2, z) measures how much the unlearned model loses its capability represent each training image. The method is effective but computationally expensive, as it requires evaluating unlearned model on every training image at runtime.
Attribution by Unlearning+. The Fisher information matrix of a deep network is difficult to compute and store, its size grows quadratically with the number of model parameters. In AbU, Wang al. [I] simply approximate this using diagonal approximation. We find that replacing the diagonal approximation with the Eigenvalue-correcte Kronecker Factorization (EK-FAC) approximtiion [10]] yields better performance and refer to this as AbU+.
While this method is accurate, it takes significant amount of runtime. For example, it takes 2 hours to process a single query for training dataset with 100K images. We aim to distill this slow but accurate method into a fast embedding next.
Two-stage data collection. Recall that our goal is collect dataset of attribution scores between synthesized and training examples. However, the expensive runtime prohibits collecting data across all training samples. We resolve this using a two-stage data collection process to improve efficiency. We note that most training samples do not mcaningfully contribute synthesized example, so evaluating all training samples D for a given query is unnecessary. first obtain the K nearest neighbors of the synthesized sample using off-the-shed features D3 D. We then collect attribution scores for them S₂ = (r(2, 2;) J2. As long as the subset contains the most influential images, the data is suitable for learning to identify them. Guo al. [102] also explored a similar two-stage approach, but to reduce runtime cost for attribution, whereas we apply speed up data collection.
narrow the search space, we
Our final dataset contains query images, corresponding training subset, and collected attribution scores: D3, Sg.

---

3.2 Learning to Rank From Attribution Data
Given query and attribution scores Sy in dataset Da, we obtain normalized ranked scores ni. [ :: :..,.. 1] for each training sample 2ú, where the most influential sample is assigned 1 and the least assigned 1.
Predicting attribution rank. We then learn a function Th z, to efficiently predict the true rank oं of the sample, shown in the bottom branch of Figure 2 We parameterize re "2 z,) cos(Sx(2) (2): Fu (z:)) to compare cosine similarity across feature embeddings. This enables one to store embeddings ((z) and perform simple feature similarity search at attribution time.
We parameterize feature embedding f = By o Φ where & is pretrained network and Fv is learned MLP that maps pretrained features towards embedding focused on attribution.
Learning to rank by cross-entropy. We adopt the pointwise learning-to- to-rankk approach 58 59 60
and apply cross-entropy loss to optimize each rank prediction individually. We then learn r through binary cross-entropy loss LACE:
[(v), ar B)
E 22
LBCE 0a (r zi))
where randomly sample synthesized images from collection 2 and training images D2, and function (I)) I+e-(oa+m) is a logit with learned affine scaling. We also explored other performs poorly, whereas Ordinal regression loss from Crammer and Singer [59] delivers competitive ranking performance. However, implementing the loss requires changes in our rank function designs, such that it no longer supports fast feature similarity search inference time, A full derivation of the ordinal loss, along with more details of the alternative ranking losses, could be found in the supplementary material.
alternatives. Direct regression 660 103
Sampling strategies. To further improve accuracy and reduce attribution computation costs, modify our sampling of training examples in two ways. First, sampling only from top- K neighbors focuses the model on fine- -naaiid distinctions but hurts its ability to recognize unrelated images. To counteract this, with probability 0, we draw a random example z outside the neighbor set and assign it the worst rank: oे = 1. This encourages the model to rank non-neighbor images last.
Second, computing attribution scores on all K neighbors at every step still incurs ((K) cost. We therefore subsample M < K candidates uniformly at each iteration, compute true ranks only that subset, and train using those A examples. practice, we find that choosing . 0.1K (or even smaller) preserves ranking performance, while greatly reducing per-query curation time. We provide a detailed study on how each strategy affects accuracy and efficiency in Section 4.1
4 Experiments
We address two questions in this section: (Q1) Rank prediction: Can our tuned feature improve attribution-rank accuracy? (Q2) Counterfactual forgetting: Does better rank prediction translate into stronger counterfactual prediction, the "gold standard" to evaluate attribution? This counterfactual prediction verifies whether the model forgets about the synthesized sample when its influential data are removed. We evaluate QI on both MSCOCO and Stable Diffusion models, and Q2 on MSCOCO only, since repeated retraining on Stable Diffusion is computationally prohibitive.
4.1 MS-COCO models
We follow previous testbeds, using the same latent diffusion model [4] trained on the 100k MSCOCO dataset [04]. The model is the same as used in AbU [ and Georgiev et al. [53] and we use the test set from AbU, which contains 110 synthesized queries.
Rank prediction metrics (Q1). We collect ground-truth ranks with AbU+ on the 110 queries, where we rank every training data point. To train our model, we generate 5000 images, using other prompts in MSCOCO. To build our dataset, for each query, we select the top 10k nearest neighbor candidates in the DINO feature space and rank candidates with AbU+, totaling 50M attribution ranks. We take 4900 queries for training and 100 for validation. We measure predicedittion accuracy on the 110-query test set by mean average precision (mAP) in binary rntrival setup: the top L
5

---

MAP (1000): Before vs, After Tuning
(1000)
Functions
9.5
0.77
0. 0.979
0.224
6.8
0.737
0.724
MAP (1000)
9.4
0.520
MAP (1000)
0.6
0400
0.400
3.4
0:339
0.329
Before tuning After tuning
0.009
DINO
CLIP Image
CLIP Text
DINO-CLIP Text
Regression
Ordinal
Cross Entropy (Ours)
Figure 3: (Left) Feature spaces. We compare different feature spaces, before and after tuning for attribution. We measure mAP the ground truth ranking, generated by AbU+. While text-only embeddings perform well before tuning, image-only embeddings become stronger after tuning. Combining both performs best and our final method. (Right) Ranking loss functions. Simple MSE regression does not converge well. Ordinal loss works well, but does not support fast similarity search inference time. We use cross-entropy, which achieves performance similar to ordinal loss while supporting similarity search. We report I-standard error in the plots.
training points from the ground truth rank are labeled positive. We denote this mAP (L) for E {500, 1000, 4000). We note that this metric is different from conventional mAP@ K, where AP is evaluated for the first K data points only. Instead, use only define positives while evaluating over the full training set. We also report other ranking metrics in Appendix C.2
Counterfactual forgetting metrics (Q2). Following AbU, for each test query, we remove the top k: influential images (k- (k=500 500, 1000, 4000), retrain the model scratch, and measure how much the retrained model forgets via:
Loss change ( 0): the increase in query à loss under the retrained model relative to the original pretrained model.
Generation deviation AGę(E, c): the difference between the original and re-generated image (using the same noise seed e), where larger deviation indicates stronger forgetting [53] As in AbU, we report the difference in mean square error (MSE) and CLIP similarity.
Note that this test is expensive, as it involves retraining a model for each combination of synthesized image and attribution method, but is a gold-standard counterfactual test for attribution. We select and study various design choices of our ranking model using the inexpensive rank-predin rank-prediction metrics, then apply the much costly counterfactual forgetting evaluation to confirm that the tuned feature space indeed improves attribution.
Which feature space to tune? Figure (left) reports the effect of tuning different feature spaces. Before tuning, using text embedding (CLIP-text) beats image-only embeddings (DINO or CLIP). However, after tuning, the image-only embeddings achieve higher performance than text-only. The best results come from concatenating DINO and CL.IP-Text: after tuning this combined feature, we achieve the highest prediction accuracy, harnessing both visual and text signals.
Learning-to-rann objectives. Figure 3 (right) compares three objective functions. We first implement a simple mean-son seean--quared-eree (MSE) loss to regress ground-truth ranks, as used in previous work (60 103]. However, this formulation fails to converge and yields poor retrieval accuracy.
Next, we find that ordinal loss [59 delivers competitive ranking performance. However, as mentioned in Section 3.2 this loss function is incompatible with fast feature search at inference time. Finally, adopt cross-entropy loss on rank labels (normalized by K), which preserves the efficient cosine- similarity search, while matching the ordinal loss in mAP(T.) across all thresholds L.. Please see additional comparisons in Appendix C.2
Data scaling. Figure 4 (left) shows how rank-predictin accuracy improves as we increase the number of training samples. We see steep gains when moving from small to moderate dataset sizes, followed by diminishing returns as size grows further. This "elbow" behavior suggests that a few thousand attribution query examples suffice capture the bulk of the ranking signal.
Sampling images outside the neighbor set. Figure (right) explores the impact of injecting random "negative" samples-images not among the top-K ciahhbors-during training. A modest 10% sampling rate of non-neighbors raises accuracy from 0.709 to 0.724, but higher rates steadily degrade performance. This pattern suggests that occasional negatives help the model maintain global

---

(1000) Dataset
mAP (1000) vs. Non-candidate Ratio
0.706
0.732
6:224
6.74
0.724
MAP (1000)
MAP (L000)
0.72
0.709
0.710
3.6
0. 100
6.70
0.050
0.501
0.676
0.0
0.660
0.5
0.3
0.01
Dataset Size (number of queries)
Probability sampling non-candidates
Figure 4: (Left) Data scaling. We investigate the impact of the number of synthesized queries. Note that each synthesized query contains attribution scores with 10k training points. We find that the performance quickly improves and saturates. (Right) Sampling outside the neighbor set. We vary the probability of selecting non-nearest neighbor images when building the attribution dataset. Using a few randomly sampled, unrelated images from the training set helps keep the learned attribution model, while having too many impedes the learning. We report 1-standard error in the plots.
Fixed number queries
Fixed budget
0.724
0.214
0.04
PoMP (1000)
9.70
0.674
mAP (1000)
6.42
0.601 0.005
0.43
5500
0.83
0.586
3.03
8.38
3.55
0343
0.58
0.553
3.0
0
634
Random candidate selection (%)
Random candi selection (%)
Figure 5: Sampling strategies of dataset construction. (Left) For each query, we randomly select from the 10k neighbors to learn from. Reliable rankings can be learned, even with relatively fewer training images per query are provided in the dataset. (Right) Given a fixed budget of 2.45M query-training attribution ranks, we test trading off between fewer training images per query and more synthesized queries. We find that at this budget, more query images with fewer training images are beneficial. We report 1 1-standard error in the plots.
distances (assigning truly unrelated images worse ranks), yet samplng them distracts from the core task of iineggrained ranking among the most relevant candidates.
Random sampling of nearest-neighbor candidates. Figure (left) fixes the number of queries and varies the fraction of top-K candidates used for training. We observe that rank-prediction accuracy drops mildly until the candidate subset falls below 20% indicating that reliable relative rankings can be learned from a small fraction of neighbors. Figure 5 (right) explores a fixed compute-budget regime (constant total queries x selected candidates) of 2.45M query-training attribution ranks. (Recall our complete dataset has 50M). this setting, one can reduce the number of training images sampled per query and increase the number of distinct queries. We find that this boosts accuracy. This trade-off suggests that under budget constraints, it is better to sample fewer candidates per query to evaluate more queries. Accordingly, adopt 20% seighbor-sampling rate for collecting attribution examples in our Stable Diffusion experiments (Section 4.2
Runtime vs. counterfactual performance. In Table we compare performance and run-time against several baselines. Runtime and storage are shown on the left. We gather runtime values for each method by warmstarting 10 queries (if applicable) and then averaging over 20 queries. We run on a single Nvidia 100 80GB for benchmarking. The right columns of Table [ show the counterfactual performance retraining model from scratch without a subset of identified images from an attribution method - and sceing if the targeted synthesized image is damaged in the subsequent model (Loss deviation) generation (Image deviation).
The influence and unlearning-bait based methods offer tradeoff's between latency, performance, and storage. Unlearning-basee AbU, and our improved AbU+ variant achieve the highest attribution performance. However, the method requires hours to run, due to repeated function evaluations on each training image. While the influence function- based methods are faster, they are lower performing, with D-TRAK performing relatively well at fast inference times (46.7 sec). However, note that this is still longer than the time to generate an image (21.5 sec for 50-DDIM steps), and the method requires storage of the preconditioned gradients on the training set (30GB), which is larger than the images in the training set itself (19 GB).
7

---

Family
Method
Eficiency
Loss devtation (c...))
1x:00 -)
SISD (:10
Image deviution 20
CLP1
Latency
Storage
500
1000
1000)
Randum
3.50
3.47.303
4.09
4.07u000
4.05.11
7.86unus
7.85u18
7.85anro
TRAK
mis
250
4:18:338
7.85
bullueues
mis
200
4.440.11
4.87085
5.72.115
4.77.018
5.36
542.034
7.68
7.53
D-TRAK
46.7 sec
5.44
6.60u033
3,59.333
5.86
64Bua25
7.82.438
7.3tanto
7.060.09
6.44
Unleam.
AbU+
Abl1
2 br
438
1.9 GB
5.83
7.13
67
10.70a332
9.78-933
5:00-031
3.6
6.20u1g
6:00-433
7.54,923
7.15amu0
7,35
6.83.339
7.00114
5,80
Text
CLIP
ms
232 MB
3.82a011
4.19un
5:52.025
4.12.02p
4.30u
4.56.0.19
7.84auu
7.72...
7.4
FacE
605.6 ms
10218
3:99
336--311
4:00:351
7.85at0
7
7.66-6
CLIP
10.3 ma
232 MB
1.
4.
6.44.333
4.35
4:57
5.30.033
7.63400
7.54x101
6.79u0r6
Imnge
DINO<2
DINO
27.9 ms
1166 ms
464 MB
354 MB5
15
4.26mmss
15
5.00
8:06-2:5
6-26-513
4,5
4.304000
4.50w03
5.29w022
5,58-921
5/0-010 14
7.4
7.68.00
7:10:10s
766-141
6:27.0.10
6.98-cm
ABC. (CT.IP)
10.4 ms
232 MR
1.35-0
4.2-0000
4.55-000
5.05003
563-033
7.5
6.54-00
ABC (DINO)
11.7 ms
354 MB
4.75
5.53u1.00
8.11:2.1
4.78
4.95
5,81
7.5kmum
7.18a1.0
6.29
In. Teat
DINOsCLIPM: 1P
15.6 mp
578 MB
4.88
5:55
8.00 321
4,63 30
5.30
5.63
7,42an05
7:14:135
6.29.010
Durs
76. ms
354 MB
4.78
6:297434
7,87mo0
Table 1: Runtime and counterfactual leave-KKoon -out evaluations. We show the runtime and storage requirements of different attribution methods. Note that in this setting, an image takes 21.5 seconds to generate. Influence and unlearning- based methods are slower than generation, highlighted in yellow and red, while search-based embedding methods are faster than generation, shown in green. We show storage costs for attribution. Methods are colored relative to the storage cost of the training set (19 GB), more than the training set as red, within 10x as yellow, and significantly less as green. Following the "gold-standare attribution methods to predict counterfactually significant training images. That is given a synthesized image 2, we train leave- K -ou models for each of the attribution methods and track ( 0), the increase in loss change, and (G c), deviation of generation. We report results over I 10 samples, and gray shows the standard error. Across each efficiency regime (slower or faster than generation time), we bold the best method and underline methods that are within a standard error. Of the fast methods, our method performs the best at attribution.
metrics from Wang et al. (1), we measure the ability of different
On the other hand, embedding based methods offer low-storage (100s of megabytes), depending on the size of the embedding, and fast run-times (tens of milliseconds). As such, these methods can be tractable, as they are cheap to create relative to the generation itself. As the models we test are learning a mapping from text to image, we explore both text and image-based descriptors. As a baseline, we test if text descriptor is sufficient for attribution, using the CLIP text encoder. While it performs better than random, text-only is not sufficient, and the visual content is indicative which training images were used. While one can an off-the-shelf visual embedding and achieve visually similar images from the training set, they are not guaranteed to be counterfactually predictive and appropriate for attribution either. Previous work [ found DINO to be strong starting point.
From the ranking evaluation above, for our method, we select the best-performing model, which is tuned DINO+CLIIP Text feature with 0.1 probability sampling images outside the neighbor set, using all the data within the neighbor set. Figure [] we plot the attribution performance (loss change with k 500) vs. throughput (reciprocal of run-time). Our method achieves strong attribution performance overall, comparable to centennceffnnciinnn speed. We show qualitative results in Figure (left) and provide more results in Appendix C.2
methods, at orders of magnitude higher
4.2 Stable Diff'usion
We use Stable Diffusion v1.4 4 105 for our experiments. We collect generated images and captions from DiffusionDB [106] as attribution queries. Since retraining is prohibitive for such large models, we only report rank prediction metrics in this setting. In Figure 7 (right), we show a qualitative example of DINO + CLIP Text feature before and after tuning.
Rank prediction metrics (Q1). Same as the setup in MSCOCO (Section 4.1.. we collect ground truth queries and collect attribution examples of them using AbU+, where the queries are split for training, validation, and testing. Different from MSCOCO, it is prohibitively costly to run AbU+ over the entire dataset LLAION-400M (0). Hence, for each test query, we retrieve 100k nearest neighbor candidates from LAION-400M through CLIP index [2 and obtain ground truth attribution ranks for those candidates. For training and validation, following studies in Section 4. I for each query, we retrieve 100k nearest neighbors. To collect more diverse queries within a fixed budget, we only rank a 20% random selection of them for supervision. We collect 5000 queries for training and 50 queries for validation, for total of 101M query-training attribution ranks. We report mAP (L)
8

---

(4000): Before
Tuning
mAP (4000) Dataset Size
9.20
tuning After tuning
0.154
0.151
0.16
0.148
0.151
MAP (4000)
9.25
0.123
mAP (4000)
6:14
0.140
9.30
0.075
0.065
0.093
6:13
0.112
0:130
6.1
1200
2003
4002
5020
DINO
CUP Image
CLIP Text
DINO CUP Text
Dataset Size (number querles)
Figure 6: Stable Diffusion ranking results. (Left) Tuning performance. We see similar trends as with MS-COCO, with strongest performing embedding using both text and image features. (Right) Data scaling. Performance increases with query images, increasing additional gains with more compute dedicated to gathering attribution training data. We report I-standard error in the plots.
Ounenanes Barn "A plate is filial vite nouatee."
Genasied Sample sortrkk of day molion devy fir weaning magio indlurdd settaw. 2-3"
Crnand frets
MSCOCO
Stable Diflusion
Qualitative examples on MSCOCO (left) and Stable Diffusion (right). For each generated image and its text prompt on the left, we show top-5 training images retrieved by: DINO + CLIP-Text (top row), Ours (middle row), and the ground-truth influential examples via AbU+ (bottom row). Compared the untuned baseline, our distilled feature space yields attributions that match the ground-truth examples more closely.
Figure 7:
for L E (500, 1000, 4000}, where we use the tuned features to rank the 100k candidates and check whether this prediction aligns with the ground truth.
Which feature space to use? Figure (left) reports performance of different features before and after tuning. Similar to the findings in MSCOCO models, tuning feature spaces consistently improves the prediction results. However, in contrast to MSCOCO models, where image feature is an important factor for accurate rank prediction, indicate that text features are strictly necessary to yield good tuning performance. Using DINO and CLIP-Imare features significantly underperforms the ones with text features. This indicates that AbU+ tends to assign attribution scores that are more correlated with text feature similarities. We discuss this more in Appendix C.2
observe the opposite in Stable Diffusion. In fact, the results
Data scaling. Figure 6 (right) reports performance improvements with respect to dataset size. Similar to the findings in MSCOCO models, there are steep gains from small to moderate dataset sizes, and the rate of gain decreases as size grows further. However, we note that even with the full dataset at 5000 queries (100M data points), the performance increase has not saturated. With more compute, one could collect more data to improve ranking performance.
Discussion, Broader Impacts, and Limitations
Data attribution a quest to understand model behavior from a data-centric perspective. It can potentially aid practical applications such compensation models, which could help address the timely issue surrounding the authorship generative content [107] [108] [09 [10]. Our method
reduces the runtime and storage cost of the data attribution algorithm, which is a step towards making data attribution feasible solution for a compensation model.
While our work demonstrates good tradeoff between compute resources and attribution performance, there are additional avenues for future work. First, the learning to rank approach does not distill the raw attribution score, just the relative ranking in the training set, so the degree of influence and how diffuse or concentrated the influence may be lost as well. Further exploring and characterizing the degree of influence is an area of future work. Secondly, our method is distilled from a teacher method, failure modes will also be inherited. However, this work, we have shown that future broader improvements in attribution methods can benefit faster method through distillation. Besides developing a fast

---

attribution method for diffusion models, there are opportunities for applying attribution to other widely used models (e.g., flow matching [TT [12]. one-step models making attribution more explainable to the end users.
TT5
[66]
and
Acknowledgmments. We thank Simon Niklaus for the help the LAION image retrieval. We thank Ruihan Gao, Maxwell Jones, and Gaurav Parmar for helpful discussions and feedback on drafts. Sheng-Yu Yu Wang supported by the Google PhD Fellowship. The project was partly supported by Adobe Inc., the Packard Fellowship, the IITP grant funded by the Korean Government (MSIT) (No. RS-2024-00457882, National AI Research Lab Project), NSF IIS- 123-229,,,, and NSF ISS-2403303.
References
[1] Sheng-Yu Yu Wang, Aaron Hertzmann, Alexei A Efros, Jun-Yan Zhu, and Richard Zhang. Data attribution for text-tn-image models by unlearning synthesized images. In Conference on Neural Information Processing Systems (NeurIPS), 2024.
[2] Xiaosen Zheng, Tianyu Pang. Chao Du, Jing Jiang, and Min Lin. Intriguing properties of data attribution on diffusion models. In International Conference on Learning Representations (ICLR), 2024.
[3] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, lyya Sutskever, and Mark Chen. Glide: Towards photorcalistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:21 12. 10741, 2021.
[4] Robin Romhach, Andreas Blattmann, Dominik I Lorenz, Patrick Easer, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[5] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06123 2022.
[6] Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zinui Wang. Vijay Vasudevan, Alexander Ku, Yinfci Yang, Burcu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, and Yonghui Wu. Scaling autoregrossive models for content-rich text-to-imape generation. arXiv preprint arXiv:2206. 10789, 2022.
[7] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kam- yar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes. Tim Salimans, Ho Jonathan, David Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffussio models with deep language understanding. arXiv preprint arXiv:2205. 1487, 2022.
[8] Oran Gafni, Adam Polyak, Oron Ashual, Shelly Sheynin, Devi Parikh, and Yaniv Taigman. Make-a-scent Scene-based text-to-image generation with human priors. arXiv preprint arXiv:2203.131333 2022.
[9] Minguk Kang, Jun- Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for text-to-image synthesis. In JEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[10] Huiwen Chang, Han Zhang, Jarred Barber, Aaron Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang. Kevin Patrick Murphy, William T. Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan. Muse: Text-to-image generation via masked generative transformers. In International Conference on Machine Learning (ICML), 2023.
[11] Frank R. Hampel. The influence curve and its role in robust estimation. Journal of the American Statisticul Association, 1974.
[12] Pang Wei Koh and Percy Liang. Understanding black-hox predictions via influence functions. In International conference on machine learning, pages 1885-1894. PMLR, 2017.
[13] Andrea Schioppa. Polina Zablotskaia, David Vilar, and Artem Sokolov. Scaling up influence functions.
In Proceedings of the AAAI Conference on Artificial Intelligence, 2022.
[14] Sung Min Park, Kristian Georgiev, Andrew Iyys,, Guillaume Leclere, and Aleksander Madry. Trak: Attributing model behavior scale. arXiv preprint arXiv:2303. 14186, 2023.
[15] Sang Keun Choe, Hwijeen Ahn, Juhan Bae, Kewen Zhao, Minsoo Kang. Youngseog Chung, Adithya Pratapa, Willie Neiswanger, Emma Strubell, Teruko Mitamura, et al. What is your data worth to gpt? Ilm-scale data valuation with influence functions. arXiv preprint XxXv:22005.3955. 2024.
10

---

[16] Myeongseob Ko, Feiyang Kang, Weiyan Shi, Ming Jin, Zhou Yu, and Ruoxi Jia. The mirrored influence hypothesis: Efficient data influence estimation by harnessing forward passes. In IEEE Conference on Computer Vision and Puttern Recognition (CVPR). 2024.
[17] Masaru Isonuma and Ivan Titov. Unlearning reveals the influential training data of language models. In The Annual Meeting of the Association for Computational Linguisties (ACL). 2024.
[18] Adobe Inc. Adobe Firefly: Generative Al for Creatives. https://www/ adobe. con/ com/products/
firefly. hitelly.hmll 2025.
[19]
Midjourney, Inc. Midjoumey Website. tttpss//wwwt httpp:://w...icooouunayy... moc.oorreey.com/ooom com/home 2024.
[20] Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarezyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. 400m: Open dataset of clip- tip-filtered 400 million image-text text pairs. arXiv preprint arXiv:2111.02114, 2021.
[21]
Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. The faiss library. 2024.
[22] Lan Feng, Fan Nie, Yuejiang Liu, and Alexandre Alahi. Tarot: Targeted data selection via optimal transport. In International Conference on Machine Learning (ICML), 2025.
[23] Alon Albalak, Yanai Elazar, Sang Michael Xie, Shayne Longpre, Nathan Lambert, Xinyi Wang. Niklas Muennighoff, Bairu Hou, Liangming Pan, Haewon Jeong, Colin Raffel, Shiyu Chang, Tat- sunori Hashimoto, and William Yang Wang. A survey on data selection for language models. arXiv preprint 2402. 16827, 2024.
[24] Robert C. Moore and William Lewis. Intelligent selection of language model training data, Proceedings of the ACL 2010 Conference Short Pupers, 2010.
[25] Cody Coleman, Christopher Ych, Stephen Mussmann, Baharan Mirzasoleiman, Peter Bailis, Perey Liang, Jure Leskovec, and Matei Zaharia. Selection via proxy: Efficient data selection for deep learning. In International Conference on Learning Representations (ICLR), 2020.
[26] Logan Engstrom, Axel Feldmann, and Aleksander Madry. Dsdm: Model-aware dataset selection with datamodels. In International Conference on Machine Learning (ICML). 2024.
[27] Vishal Kaushal, Rishabh Iyer, Suraj Kothawade, Rohan Mahadev, Khoshrav Doctor, and Ganesh Ra- makrishnan. Learning from less data: A unified data subset selection and active learning framework for computer vision. In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2019.
[28] Jiachen Tianhao Wang. Tong Wu, Dawn Song, Prateek Mittal, and Ruoxi Jia. Greats: Online selection of high-quality data for Ilm training in every iteration. In Conference on Neural Information Processing Systems (NeurIPS), 2024.
[29] Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. Less: Selecting influential data for targeted instruction tuning. In International Conference Machine Learning (ICML., 2024.
[30] Baharan Mirzasoleiman, Jeff Bilmes, and Jure Leskovec. Coresets for data-efficient training of machine learning models. In International Conference on Machine Learning (ICML), 2020.
[31] Krishnateja Killamsetty, Sivasuhramanian Durga, Ganesh Ramakrishnan, Abir De, and Rishabh Iyer. Grad- match: Gradient matching based data subset selection for efficient deep model training. In International Conference on Machine Learning (ICML), 2021.
[32] Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on a data diet: Finding important examples early in training. In Conference on Neural Information Processing Systems (NeurIPS), 2021.
[33] Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy S Liang, Quoc V Le. Tengyu Ma, and Adams Wei Yu. Doremi: Optimizing data mixtures speeds up language model pretraining. In Conference Neural Information Processing Systems (NeurIPS), 2023.
[34] Simin Fan, Matteo Pagliardini, and Martin Jaggi. Doge: Domain reweighting with generalization estimation. In International Conference on Machine Learning (ICML), 2024.
[35] Simin Fan, David Grangier, and Pierre Ablin. Dynamic gradient alignment for online data mixing. arXiv preprint 2400422410.02498, 2024.

---

[36] Mayee Chen, Nicholas Roberts, Kush Bhatia, Jue Wang, Ce Zhang, Frederic Sala, and Christopher Ré. Skill-id a data-driven skills framework for understanding and training language models. In Conference on Neural Information Processing Systems (NeurlPS). 2023.
[37] Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, and Danqi Chen. Sheared Ilama: Accelerating language model pre-training via structured pruning. In International Conference on Learning Representations (ICLR), 2024.
[38] Alon Albalak, Liangming Pan. Colin Raffel, and William Yang Wang. Efficient online data mixing for language model pre-training. arXiv preprint Xiiv:2311..22006 2023.
[39] Amirata Ghorbani and James Zou, Data shapley: Equitable valuation of data for machine learning. In International Conference on Machine Learning (ICML), 2019.
[40] Ruoxi Jia, David Dao, Boxin Wang, Fa Hubis, Nick Hynes, Negin Gurel, Bo Wei, Costas J. Spanos Li, Dawn Song, and Costas J. Spanos. Towards efficient data valuation based on the shapley value. In International Conference on Machine Learning (ICML), 2019.
[41] Ferenc Illés and Péter Kerényi. Estimation of the shapley value by ergodic sampling, arXiv preprint 1XXi::9006.02244 2019.
[42] Jiachen T Wang, Prateek Mittal, Dawn Song, and Ruoxi Jia. Data shapley in one training run, arXiv preprint arXiv:2466. 1011, 2024.
[43] Mark A. Burgess and Archic C. Chapman. Approximating the shapley value using stratified empir- ical bernstein sampling. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 2021.
[44] W. Li and Y. Yu. Faster approximation of probabilistic and distributional values via least squares. In International Conference on Learning Representations (ICLR), 2024.
[45] Jinkun Lin, Anqi Zhang, Mathias Lécuyer, Jinyang Li, Aurojit Panda, and Siddhartha Sen. Measuring the effect of training data on deep learning predictions via randomized experiments. In International Conference on Machine Learning (ICMI.), 2022.
[46] Rory Mitchell, Joshua Cooper, Eibe Frank, and Geoffrey Holmes. Sampling permutations for shapley value estimation. Journal of Machine Learning Research, 2022.
[47] Ramin Okhrati and Aldo Lipani. multilinear sampling algorithm to estimate shapley values. 2020
25th International Conference on Pattern Recognition (ICPR), 2021.
[48] Vitaly Feldman and Chiyuan Zhang. What neural networks memotize and why: Discovering long tail via influence estimation. Conference on Neural Information Processing Systems (NeurIPS), 2020.
[49] Andrew Ilyas, Sung Min Park, Logan Engstrom, Guillaume Leclerc, and Aleksander Madry. Datamodels: Predicting predictions from training data. In International Conference on Machine Learning (ICML), 2022.
[50] Lloyd S Shapley. A value for n-person games. Princeton University Press Princeton, 1953.
[51] Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. Conference on Neural Information Processing Systems (NeurlPS), 33:19920- 19930, 2020.
[52] Shengchao Wang. Xuezhou Zhang, Junchi Yan, Tianle Liu, and Yisen Wang. Training data attribution via approximate unrolling. In Conference Neural Information Processing Systems (NeurIPS), 2024.
[53] Kristian Georgiev, Joshua Vendrow, Hadi Salman, Sung Min Park, and Aleksander Madry. The journey, not the destination: How data guides diffusion models. arXiv preprint arXiv:2312.06205, 2023.
[54] Roger Grosse, Juhan Bae. Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, et al. Studying large language model generalization with influence functions. arXiv preprint arXiv:2308.033: 002333008000..... 2023.
[55] Bruno Kacper Mlodozenice, Runa Eschenhagen, Juhan Bac, Alexander Immer, David Krueger, and Richard F. Tumnr. Influence functions for scalable data attribution in diffusion models. In International Conference Learning Representations (ICLR), 2025.
[56] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen Zhu, Yuanzhi Li. Shean Wang, Lu Wang. Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 2022.
12

---

[57] Tie- Yan Liu. Learning to rank for information retrieval. Foundations and Trends in Information Retrieval, 2009.
[58] Ping Li, Christopher J. Burges, and Qiang Wu. Mcrank: Learning to rank using multiple classification and gradient boosting. In Conference on Neural Information Processing Systems (NeurlPS), 2007.
[59] Koby Crammer and Yoram Singer. Pranking with ranking. In Conference Neural Information Processing Systems (NeurlPS), 2002.
[60] David Cossock and Tong Zhang. Statistical analysis of hayes optimal subset ranking. IEFE Transactions Information Theory, 2008.
[61] Yoav Freund, Raj Iyer. Robert E. Schapire, and Yoram Singer. An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research, 1998.
[62] Ralf Herbrich, Thore Graepel, and Klaus Obermayer. Large margin rank boundaries for ordinal regression. In Advances Large Margin Classifiers, 2000.
[63] Christopher J.C. Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. Learning rank using gradient descent. In Proceedings of the 22nd International Conference on Machine Learning (ICMI.,, 2005.
[64] Christopher Burges, Robert Ragno, and Quoe V. Le. Learning to rank with nonsmooth cost functions. Conference on Neural Information Processing Systems (NeurIPS), 2006.
[65] Maksims N. Volkovs and Richard S. Zemel. Boltzrank: Learning to maximize expected ranking gain, In International Conference on Machine Learning (ICML), 2012.
[66] Michael Taylor, John Guiver. Stephen Robertson, and Tom Minka. Softrank: Optimizing non-smoot smooth rank metrics. Proceedings of the International Conference on Weh Search and Data Mining (WSDM), 2008.
[67] Zhe Cao, Tao Qin, Tie Liu, Ming-Fen Tsai, and Hang Li. Learning to rank: From pairwise approach to listwise approach. In Proceedings of the 24th International Conference on Machine Learning (ICML), 2007.
[68] Christopher J.C. Burges. From ranknct to lambdarank to lambdamart: An overview. In Microsoft Research Technical Report
2010.
[69] Kalervo Järvelin and Jaana Kekäläinen, Cumulated gain- based evaluation of ir techniques. ACM Transactions on Information Systems, 2002.
[70] Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. Introduction to information retrieval. In Cambridge University Press, 2008
[71] Tao Qin, Tie-Yan Liu, Jun Xu, and Hang Li. Letor: A benchmark collection for research on learning to rank for information retrieval. Information Retrieval, 2010.
[72) William Hersh, Chris Buckley, T.J. Leone, and David Hickam. Ohsumed: An interactive retrieval evaluation and new large test collection for research. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1994.
[73] Alex Krizhevsky and Geoffrey E Hinton. Using very deep autoencoders for content-based image retrieval. ESANN, volume 1, page 2. Citeseer, 2011.
[74] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations (ICLR), 2015.
[75] Alexey Dosovitskiy, Lueas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021.
[76] Ross Girshick, Jeff Donahue. Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 580-587, 2014.
[77] Jason Yosinski. Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In Conference on Neural Information Processing Systems (NeurIPS), 2014.
13

---

[78] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman. Simran Arora, Sydney von Arx. Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models, arXiv preprint arXiv:2 07258, 2021.
[79] Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision (ICVV), (15())211--252 2015.
[80] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features a perceptual metric. In IEEE Conference Computer Vision and Puttern Recognition (CVPR), 2018.
[81] Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dreamsim: Learning new dimensions of human visual similarity using synthetic data. In arXiv preprint arXiv:2306. 09334, 2023.
[82] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE trunsactions on pattern analysis and machine intelligence, 15(8):1798-1825 2013.
[83] Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and compos- ing robust features with denoising autoencoders. Proceedings of the 25th international conference on Machine learning, pages (096-1103) 2008.
[84] Virginia De Sa. Learning classification with unlabeled data. Advances in neural information processing systems, 6, 1993.
[85] Carl Doersch, Abhinav Gupta, and Alexei A Efros. Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE International Conference on Computer Vision, pages 1422-1430, 2015.
[86] Xiaolong Wang and Abhinav Gupta. Unsupervised learning of visual representations using vidcos. In Proceedings of the IEEE international conference on computer vision, pages 2794-2802, 2015.
[87] Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. In European Conference Computer Vision (ECCV), 2016
[88] Deepak Pathak, Philipp Krahenbuhl. Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In IEEE Conference on Computer Vision and Pottern Recognition (CVPR), 2016.
[89] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 11597-6077 PMLR, 2020.
[90] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation lcarning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9738, 2020.
[91] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding, In European Conference on Computer Vision (ECCV), 2020.
[92] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou. Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In IEEE International Conference on Computer Vision (ICCV), 2021.
[93] lean-Bastedn Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new appreach to self-supervised learning. Advances in neural information processing systems, 33:21271 284, 2020.
[94] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transfcrable visual models from natural language supervision. In International Conference on Machine Learning (ICML), 2021.
[95] Sheng- Yu Wang, Alexei A. Efros, Jun Yun Zhu, and Richard Zhang. Evaluating data attribution for text-to-image models. In IEEE International Conference Computer Vision (ICCV), 2023.
14

---

[96] Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten. Certified data removal from machine learning models. In International Conference Machine Learning (ICML), 2020.
[97] Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Etemal sunshine of the spotless net: Selective forgetting deep networks. In IEEE Conference Computer Vision and Pottern Recognition (CVPR), 2020.
[98] Nupur Kumari, Bingliang Zhang, Sheng- Yu Wang, Eli Shechtman, Richard Zhang, and Jun-Yan Zhu. Ablating concepts in text-to-image diffusion models. In IEEE International Conference on Computer Vision (ICCV), 2023,
[99] Rohit Gandikota, Joanna Materzynska, Jaden Fiotto-Kaufman, and David Bau. Frasing concepts from diffusion models. In IEFF International Conference on Computer Vision (ICCV), 2023.
[100] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences (PNAS). 1 114(13):352133522 2017.
[101] Thomas George, César Laurent, Xavier Bouthillier, Nicolas Ballas, and Pascal Vincent. Fast approximate natural gradient descent in a kronecker factored eigenbasis. In Conference on Neural Information Processing Systems (NeurIPS), 2018.
[102] Han Guo, Nazneen Fatema Rajani, Peter Hase, Mohit Bansal, and Caiming Xiong. Fastif: Scalable influence functions for efficient model interpretation and debugging. arXiv preprint 117711 2020.
[103] Przemyslaw Pobrolyn, Tomasz Bartezak, Mikolaj Synowiec, Radoslaw Bialobrzeski, and Jaroslaw Bojar. Context-aware learning to rank with self-attention, arXiv preprint arXiv:2005. 10084, 2020.
[104] Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and Lawrence Zitnick. Microsoft coco: Common objects in context. European Conference on Computer Vision (ECCV), 2014.
[105] Stable diffusion. 2022.
[106] Zijie J. Wang, Evan Montoya, David Muncchika, Haoyang Yang, Benjamin Hoover, and Duen Horng Chau. DiffusionDB: A large-scale prompt gallery dataset for text-lo-image generative models. arXiv:2210.1444 14896 [cx]. 2022.
[107] Katherine Lee, A Feder Cooper, and James Grimmelmann. Talkin ai generation: Copyright and the generative-ai supply chain. arXiv preprint arXiv:2309.08/33, 2023.
[108] Niva Elkin-Koren Uri Hacohen, Roi Livni, and Shay Moran. Can copyright be reduced privacy? arXiv preprint arXiv:2305. 1422,, 2023.
[109] Uni Hacohen, Adi Haviv, Shahar Sarfaty, Bruria Friedman, Niva Elkin Koren, Roi Livni, and Amit Bermano. Not all similarities are created equal: Leveraging data-driven biases to inform genai copyright disputes. arXiv preprint 240iv:2403. 17691, 2024.
[110] Ziv Epstein, Aaron Hertzmann, Hany Farid, Jessica Fjeld, Morgan R. Frank, Matthew Groh, Laura Herman, Neil Leach, Robert Mahari, Alex "Sandy"" Pentland, Olga Russakovsky, Hope Schroeder, and Amy Smith. Art and the science of generative ai. Science, 38066650) 1111-1111,,111-11111111111111111 2023.
[111] Yaron Lipman, Ricky T. Q Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthw Le. Flow matching for generative modeling. In International Conference on Learning Representations (ICLR), 2023.
[112] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning generate and transfer data with rectified flow. arXiv preprint arXiv:2209.0300 03009,, 2022.
[113] Tianwei Yin, Michaël Gharbi, Tacsung Park, Richard Zhang, Eli Shechtman, Frédo Durand, and William T. Freeman. Improved distribution matching distillation for fast image synthesis. In Advances in Neural Information Processing Systems (N)urIPS), 2024.
[114] Tianwei Yin, Michael Gharbi, Taesung Park, Richard Zhang. Eli Shechtman, Fredo Durand, and William T. Freeman. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2024.
15

---

[115] Minguk Kang, Richard Zhang, Connelly Barnes, Sylvain Paris, Suha Kwak, Jaesik Park, Eli Shechtman, Jun- Yan Zhu, and Taesung Park. Distilling diffussin models into conditional gans. In European Conference on Computer Vision (ECCV). 2024.
[116] Yang Song, Chenlin Meng, and Stefano Ermon. Consistency models. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
[117] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Conference on Neural Information Processing Systems (NeurIPS). 2020.
[118] Yang Song, Jascha bohl-Diccsttei, Diederik Kingma, Abhishck Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic diffcrential equations. In International Conference on Learning Representations (ICLR), 2021.
[119] Jascha Sohl-Dickstein, Fric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference Machine Learning (ICML), 2015.
[120] Vinod Nair and Geoffrey Hinton. Rectified linear units improve restricted boltzmann machines. In International Conference on Machine Learning (ICML), 2010.
[121] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo. Marc Szafraniec, Vasil Khalidov, Pierre Fernandez. Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.077 07193, 2023.
[122] Zhe Li, Wei Zhao, Yige Li, and Jun Sun. Do influence functions work on large language models? arXiv preprint 2409 19998, 2024.
[123] Ekin Akyürek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, and Kelvin Guu. Towards tracing factual knowledge in language models back the training data. In Empirical Methods in Natural Language Processing (EMNLP), 2022.
A
Formulations
Here expand the main paper's formulations with more details.
A.1
Diffusion Models.
Section 3 of the main paper describes our formulation, which uses the loss L of the generative model being used. Our experiments are on diffusion models, and we describe the loss here. Diffusion models [177 118 [19 learnsto reverse a data noising process, defined as Xu...., Xy, where a clean image Xg is gradually diffused a pure Gaussian noise Xy over T timesteps. At timestep t E [0, T noise e NO, I) is blended into the clean image Xo to form noisy image x, X + VI- â,e, where a, defines the noise schedule. The training loss of the diffusion models is to encourage accurate denoising of x, by predicting the noise:
Exx.c,t [walle Ex c, ()
where W/ is weighting term typically set to [117) co is the denoising model, and c is the text condition. inference, the denosing model cy takes in random Gaussian noise and gradually denoises it to the learned data distribution.
Evaluating the diffusion loss in Equation 4 requires a Monte-Carlo estimate, and estimate by taking averages over different noise samples and timesteps. Section B provides implementation details of the Monte-Carlo estimate.
A.2 Influence function.
In the main text, we discussed that influence function methods suffer from a tradeoff between computation cost and attribution performance. To understand this, we briefly recap the influence function from Koh and Liang [12] Let
16

---

N R(0) = N Σ C(z,,)) 0), 8. = arg min R(),, = V)R(()).
(5)
where R. is the full training loss, {z:} are the training examples, and H is the Hessian of the training loss. We estimate the effect of removing a single training point z via a small perturbation et
0 arg min (R(0) =L(2,
(6)
where 0_z,e is the optimal model under the perturbed loss. 0 a= statistfies the stationary condition:
0 VoR(0-2. - -Vol(z,0022
From Equation 7 taking Taylor approximation around 0g and assuming convergence in model training, Koh and Liang show that the change in model weight A = 0 _2, - 80 would be:
A H Vo
8
Influence function is then defined by the rate of change of loss the testing point (or synthesized image) & with respect to the perturbation on the training point z. By the chain rule:
AC(2, 0 0=,e) 6
08_2.t
de
00 00_2.c
de
To make the computation of Equation tractable, recent works [14 Hessian by Generalized Gauss-Newiae (GGN) approximation, which essentially replaces the Hessian with the Fisher information matrix.
54 53 15 53 estimate the
However, even with this approximation, the tradeoff between computation cost and attribution performance remains. The size of the gradient (e.g.. VoC(z, Bo)) too big store. One could either
sacrifice runtime by computing training point gradients on the fly upon each query [54] 55 1. or
projecting gradients to a much smaller dimension that sacrifices performance
A.3 Eiienvalue- Kronccker-factored Approximate Curvature (EKFAC)
We collect attribution data using AbU+, an improved variant of AbU M. by replacing the diagonal Fisher-ifformation approximation with EKFAC [TOI) Below is a brief overview of EKFAC. We first discuss KFAC, the basis of EKFAC.
KFAC Kcronecker-factore Approximate Curvature) consists of two core ideas to reduce the space complexity of FIM: (1) approximating the Fisher information matrix (FIM) blockwise, where each layer corresponds to block, and (2) reducing each layer's FIM block to two smaller covariances. To illustrate the idea, for linear layer (no bias) layer é:
= W;0:-1,
(10)
a; = pi(si),
where yi is an element-wise nonlinearity (e.g., ReLU 120 0:-1 E Rdv, and W, Rdout
Writing 9i = V. L, the weight gradient
Vw 9; _ vec (Vc(Vw.C) a:-1 @ gb:
(11)
where vec(.) flattens 2D matrix column-wie miiunn-wiieeei into an vector. We use the fact that flattened outer product of two vectors can written as a Kronecker product the two vectors. The Fisher block becomes
17

---

F; E veel V w, L) vec(Vw,
E [a-- @ (9igg)
(12)
E [ak-1 & (;, 2 gi)²
2 E aaa-11
Elgig!
dy dia
dnx
dus X deat
The last step in the Equation 12 leverages properties of the Kronecker product. The final approxima- tion reduces space complexity from Odd dou)) to Oddy, + d since the only requirement is to compute the two smaller covariance matrices.
EKFAC refines KFAC by correcting the eigenvalues without increasing space complexity. We first take the eigen decompositions the two covariance matrices:
1llaa-104/11 UASAA E[ggg/ Up SB Up.
(13)
Then the FIM block becomes:
= A (UA
(UB
(14)
(UA @ UB) (SA @ SB) (UA @U))
EKFAC fixes the shared eigenbasis UΛ & Un but re-estimates the diagonal eigenvalue matrix S by projecting empirical gradients into this basis:
F (UA (UA (UA
(15)
Since UA E Rd Xdu, UB E Room xdout, and diagonal size doutout the O + dout) cost is preserved while reducing approximation error [IO..
In practice, we sample training images, noise vectors, and timesteps to estimate A, B, and the corrected eigenvalues S. Implementation details, including convolutional extensions, appear in Section
A.4 Learning-to-Rant Objectives
In Section the main text, we compare our cross-entropy objective with two other alternatives: MSE loss and ordinal loss. For the two losses, we follow the conventions from Pobrotyn et al. [103.
Using the notations from Section 3 of the main text, we want to predict the normalized ranked scores Ti E 1 2 k .... 1) for each training sample z,, where the most influential sample is assigned K and the least assigned 1.
MSE loss. This simply regresses the normalized rank:
m oa,3 (ro(2, (r)(2,2))) 2.)
E 2m 2,
(16)
where rg and the affine-scaled sigmoid (x) 11 (ar+())
defined in Section 3.2.
As reported in Section 4.1, training with this loss fails to converge and yields poor retrieval accuracy. We conjecture that regressing dense ranks over 10t candidates (vs. the usual 5 100) (60 103)
makes
the MSE formulation ill-suited to our setting.
Ordinal loss. In the ordinal-regression regression framework [59], each ground-truth (unnormalized) rank r E (l,..., K) is converted into binary vector of length via
(*() = [1, > hi 0, r h.
A k= 1.....,KK
(17)
Our ranker no now outputs logits l! "!....thh pk
which we transform into probabilities
(e
(18)
1
18

---

The ordinal loss is the sum of binary cross cross-entropies over thresholds:
K
Codd
8
(T))
logp" + -1 (m²)) log(1 - r]].
12-b((mm
(19)
2,
At inference, we recover scalar rank via
K-1 p
(20)
Directly using K s 10a would require 10 binary heads and thresholds, which is memory-prohibitive. Instead, we coarsen the rank range into 10 equal-width bins and apply ordinal loss over the B - thresholds, reducing the number of binary classifiers to 9, while preserving most of the ordinal structure.
In contrast to our setup in Section of the main text, the ordinal approach requires the network to emit multiple feature heads (one per threshold), compute a separate cosine similarity, affine transform, and sigmoid for each, and then sum all resulting probabilities to recover a scalar rank. This multi-step, multi-head pipeline increases parameters and computation and slows inference. Since our cross- entropy-based method matches ordinal loss in accuracy while only requiring one feature vector with no extra summation, we adopt it as our main method due its simplicity and efficiency.
B Implementation Details
B.1 MSCOCO Models
Collecting attribution data. We follow the AbU+ procedure from Wang et al. [1, performing single unlearning step that updates only the cross-attention key/value layers. When using EKFAC in place of a diagonal Fisher approximation, we set the Newton step-size to ON01/N, with N = 118287 (the size of the MSCOCO training set).
Within that step, we estimate the diffusion loss via Monte Carlo by sampling and averaging over 50,000 independent (noise, timestep) pairs. Attribution scores are then defined as the difference in loss between the unlearned and original models. To compute each loss, we evaluate at 20 equally spaced timesteps: at each timestep, we average the five losses obtained by combining the image with each of its five corresponding captions (using different random noise for each caption).
Training rank models. Our rank model is a 3-layer MLP with hidden and output dimensions of 768. We optimize using AdamW (learning rate 10-3, default betas 0.9, 0.999, weight decay 0.01) ) for 10 epochs the training set, without any additional learning-rate scheduling.
B.2 Stable Diffusion
Collecting attribution data. Similar to MSCOCO (Section B. we run AbU+ by performing single unlearning step that updates only the cross-attention key/value layers. We the Newton step-size to 0.002, and N = 400, 000, 000 (the size of LAION-40OM (20).
Within the unlearning step, we estimate the diffusion loss via Monte Carlo by sampling and averaging over 4,000 independent (noise, timestep) pairs. To compute the loss for attribution scores, we evaluate 10 equally spaced timesteps, and at each time step, sample random noise and the corresponding caption to assess the loss value.
Training rank models. We follow the exact same training recipe the rank model for MSCOCO,
which is described in Section[B..
B.3 Baselines
We describe the baselines used in our experiments. Most baselines follow those reported in AbU [1.
Pixel space. Following JourneyTRR TRAK's implementation 53 cosine similarity for attribution.
we flatten the pixel intensities and use
19

---

CLIP image and text features. We use the official ViT-B/32 model for image and text features.
DINO 192. We the official ViT-B/16 model for image features.
DINOv2 121 We use the official ViT-L.14 model with registers for image features.
CLIP (AbC) and DINO (AbC) [95]. We use the official models trained on the combination of object-centric and style-centric customized images. CLIP (AbC) and DINO (AbC) are selected because they are the best-performing choices of features.
TRAK [14] and Journey TRAK [53]. We adopt the official implementation of TRAK and Journey- TRAK and use random projection dimension 16384, the same as what they use for MSCOCO experiments.
D-TRAK [2]. We follow the best-performing hyperparameter reported in D-TRAK, using a random projection dimension of 32768 and lambda of 500. We use single model to compute the influence score.
AbU T and AbU+. For ABU, we follow the default hyperparameter reported in the paper. For AbU+, we follow the same hyperparameters the used for data curation, which is described in Section B.1
Licenses. Below we list the licenses of code, data, and models used for this project.
MSCOCO source model: collected from Georgiev et al. [53 which is under the MIT License.
MSCOCO dataset: Creative Commons Attribution 4.0 License.
MSCOCO synthesized images testset: collected from AbU [1], which is under CC BY-NC-SA
4.0.
Stable Diffusion: CreativeML Open RAIL++MM License.
DiffusionDB images: MIT License.
CLIP model: MIT License.
DINO model: Apache 2.0.
DINOv2 model: Apache 2.0.
AbC model: CC BY-NC-SA 4.0.
TRAK code: MIT License.
EKFAC code: Taken from the Kronfluence codebase, which is under Apache 2.0 License.
C
Additional Analysis
C.1
Compute Cost
Our experiments are all done by NVIDIA A100 GPUs. Below, we describe the runtime cost of the components of our project.
Data curation. On MSCOCO, attributing 100k candidates for 110 test queries at 2 hours/query took 220 GPU-hours, while curating 10k candidates for 5,000 train/val queries at 0.25 hour/query took 1,250 GPU-hours. For Stable Diffusion, attributing 20k candidates for 5,050 train/val queries at hours/query required 15,150 GPU-hours, and 140 test queries at 15 hours/query consumed 2,100 GPU-hours in total.
Training time for LTR models. Training one rank model on MSCOCO for 10 epochs takes approximately GPU-hour, while training the Stable Diffusion rank model for the same number of epochs requires about 6 GPU-hours.
C.2 Additional Results
Effectiveness of K-NN retrieval using off-the-shelf features. We study the effect of sampling from top- K neighbors for data collection, described Section 3.2 Table we report Table 2
counterfactual metries for AbU+ ran on (1) the full training set (AbU+) and (2) top neighbors after K-
20

---

Loss deviation AC(2,))
Image deviation AGo(e,
Method
Tx110 ?-
MSE fx TO
CLIP [x100
500
1000
4000
500
1000
4000
500
1000
4000
Random
3,5=000
3.5=0.0
3,5+0.0
4.1+0.11
4. 4.11
4.0:t:.1
7.9+0.00
7.9+0.0
7.9+0.0
D-TRAK
5.4=0.2
6.6=002
9.6+=0.3
5,9+002
6,4
7.8+
7.3++011
7.1+:011
6.4+0.1
AbU+
202
7.1 202
11.0
5.64:0.
6.2
7.5
7.2.00.1
6.8
5,8
AbU+ (K-NN)
5.8=02
7.1
9.7.20.4
5,440.
6.340033
6.9.00.3
7.1+0.1
6.7440.1
5.8
Table 2: Counterfactual leave-K-one -out evaluations. We report loss deviation and image deviation metrics K E {500, 10000,000). Values are scaled indicated in the headers; gray shows the standard error.
NN (AbU+ (K-NN)). We also copied numbers from D-TRAK (2nd best teacher) and random baselines reference. We find that applying K-NN retrieval does not introduce significant performance drop.
Predicting absolute attribution scores directly. We explore directly predicting the absolute attribu- tion instead of ranks, where the MIP regresses the absolute attribution scores (normalized by mean and standard deviation). This regression leads to worse ranking performance (0.009 mAP (1000)) than our Cearning-to-rane method (0.724 mAP (1000)).
Rank prediction evaluation. In Section we only include mAP (L) with one L value. Here, include mAP (500) and mAP (4000) for MSCOCO experiments in Figure 13 We include mAP (500) and mAP (1000) for Stable Diffusion experiments in Figure 16 All trends are similar to the ones reported in the main text.
MSCOCO qualitative results. Figure presents presents additional MSCOCO examples. Our rank model can retrieve influential training images as AbU+. Their influence is confirmed-removving those images, retraining, and regenerating the query leads large deviations.
Stable Diffusion qualitative results. Figure 9 shows additional examples on Stable Diffusion. Our model's top attributions follow those of AbU+, emphasizing prompt content over pure visual similarity. Since full counterfactual retraining is infeasible at this scale, we cannot definitively verify that these attributions reflect true influence. However, as shown in Section 4.2, our method reliably predicts AbU+'s 's ranks, indicating consistent attribution signal.
Establishing the ground-truth validity of this signal is left to future work. Prior studies suggest that influence-based methods may weaken on very large models (e.g., LLMs) [12 123]. However, developing more robust attribution algorithms for large-scale models-ann and distilling a rank model from stronger teacher methods using our method-and all promising directions ahead.
21

---

DINO CUIP Text
Our
Sample
"The dining tabke near the kistchss kas bowl of fruit
Ut
DIO CUP Text
Sample "a floots of gents
Our
come made watsking thew
Ground
DIN CUP Text
Ground Ours
Gonoratac
"A person, da a helinggh is
riding
skattebaard.""
DINO OUr Text
Generstec Sample living noow. that has
Curr
coushess and tables it*
Ground
Remove Top Retrain & Regenerate
Top Attributed Training Sample
Figure 8: MSCOCO qualitative results. For each synthesized sample (leftmost), compare three methods- DINO+CLIP Text (top row), our tuned rank model (middle row), and AbU+ ground truth (bottom row). Left block: after removing the top 500 attributed images and retraining. the model re- re-generates the query. Right block: each method's 's top-8 attributed training samples. Our method matches AbU+ in both forgetting behavior and retrieved examples.
22

---

"L.i7 awl. printing, and CLIP Text
DIN
"SPV Aves HeAd"
"amile Holees 4de stwwaa
"saller Helms 2.5 cdvin"
"An sien? Crece hdloudt head helinds soca.jt9
"Adult Dd Knight Milinee
"Adlall Ko-ight Helloua
"Aduli Knnight Hölieet
"Charnoter Workslkop Powale Knight"
Ganerated Semple "thoonter
heseedt, wresal enggine, realictio, witra sigaill
Kisight
exseliveel tniggtt helluue"
snnsur mundd barkesoycid"
mustle bertosqula"
Inloust
atmanet
wordlen Its.w. Knsight an Prise 2orhe
medel venttime FSR?
turbssuiss
entirtly medeled ie Mix [..]"
Ground Truth
"feetbail hnlinge
"Winnar Platomaal Arour M dlyed head "nooejho"
"WiG ridaliiis knlwet, Sunpper bassage Helindls"
"Sparthi warris weering kelinee and real oleeee
"[--] Helluue corpiny Cestumms Flard Invi masia L..]"
"Oarbon filer chanstropper helsur, and of Riiia"
"Sinapatrs Advassap Farial Rig (or Maya L.
Trooper Hellugh
DIC
"haxciig wilcinee
"Enchanded dente forett wallspter Mural?
"Lake night scene with grass sillowette
PRleting Masuigge Geeto by wedesssn harker
Paint Rouzzai Moww"
Audher Hilly Wlack Ho Malefieen Journay L.:"
"Magio Maan Tree Phone Wried"
"Eeönnd Cencibus Canvas by Alcxander Janasan convas Artworks"
eroled Sample "A lunar magicai onts Lookiing into reflections
L..]"
mode. awt coilage by maxl
"Sulatiee
"Slety Art Print"
"Mist Weeds) rine Art Original"
"hlggtt ssxis original Asnylis E..I"
mediaa, with awi aod mune,, hore"
willow Nigita Art Out L-]"
mixnd mutil Owi Art"
Loaves Maon SStare L.."
Ground Truth
"We codk lhe butt flond only
'L-J Digital wulf
*Faulasy Lauad soat with grass
"L vitlon waiding for the
"Colerful Pieturti of colorful Failetings"
"chriatiaw- mirrer"
"awd in ndgett sky.
L..""
Figure 9: Stable diffusion qualitative results. For each generated image (left), we compare the MINO+CLIP-Tex. baseline (top row), calibrated feature ranker (middle row), and AbU+ ground- truth attributions (bottom row). Both AbU+ and our method tend to retrieve images that reflect textual cues rather than visual similarity. Top: "warrior helmet" retrieved helmets rather than faces wearing helmets. Bottom: "lunar sage magical owls" retrieved owl-centric scenes despite no owls in the query images.
mAP (500): Before After Tuning
mAP (4000): Before
After Tuning
0.8
0.663
6.8
0.664
0.670
0.704
MAP (500)
3.6
mAP (4000)
6.6
0.494
0.4
0.392
6.4
0.388
0:256
0,250
3.2
tuning After tuning
5.2
tuning After tuning
3.2
5.0
Image
Image
Figure 10: mAP across different feature spaces. We compare different feature spaces, before and after tuning for attribution. We report mAP (500) and mAP (4000) to the ground truth ranking, generated by AbU+. The trend is similar to Figure 3 (left) of the main text.
23

---

mAP (500) Across Loss Functions
mAP (4000) Across Loss Functions
0.8
0.670
0.663
0.713
0.704
MAP (500)
0.0
mAP (4000)
0.005
0.035
Regression sion
Ordinal
Cross Entropy (Ours)
Regression
Ordinal
Cross Entropy (Ours)
Figure 11: mAP across different learning-to-rank. to-rank losses. Simple MSE regression does not converge well. Our cross-entropy method achieves performance similar to ordinal loss while supporting similarity search. We report mAP (500) and mAP (4000), and the trend is similar to Figure 3 (right) of the main text.
mAP (500)
Dataset Size
mAP (4000) Dataset Size
277
0.61o
0.643
0.450
0. 0,6
0.692
0.696
MAP (500)
3.6
PuAP (4000)
0.619
0.639
2.
0.504
0.421
3/
0.50
23
0.266
1000
2002
2000
4002
2000
2000
2002
4000
5002
Dataset Size (number of queries)
Dataset Size (numbee queriees)
Figure 12: mAP across different dataset sizes. We find that the performance quickly improves and saturates as the dataset size grows. We report mAP (500) and mAP (4000), and the trend similar to Figure 4 (left) of the main text.
Non-candidate Ratio
mAP (4000) vs. Non-candidate Ratio
mvAr (500)
3.43
3.65
0.663
0.648
maP (4000)
0.725
0,089
3.64
0.625
6.44
3.62
0.606
3.4
0.800
sampling non-candidates
Probability sampling non-candidates
Figure 13: mAP across probability of non-candidate sampling. Using few randomly sampled, unrelated images from the training set helps keep the learned attribution model, while having too many impedes the learning. We report mAP (500) and mAP (4000), and the trend is similar to Figure 4 (right) of the main text.
Fixed number of queries
Fixed number queries
MAP (500)
0.663
0.650
0.625
MAP (00D0)
6.70
0.704
0.688
0.687
0.675
0.63
001
0.80
0366
0:67
candidate selection
Random candi selection
Figure 14: mAP vs. random subset ratio with a fixed number of queries. Reliable rankings be learned, even with relatively fewer training images per query. We report mAP (500) and mAP (4000). and the trend is similar to Figure 5 (left) of the main text.

---

Fixed budget
Fixed budget
MMP (5000
254
3.53
0.505
0.525
522
mAP (4000)
0.66
0.631
0,638
2.3
0.492
0.612
3.4
0.459
0.43
200
46
Random candidate selection ratio (%)
Random candidate selection ratio (%)
Figure 15: mAP vs. random subset ratio with a fixed budget. We find that at a fixed budget of 2.45M, more query images with fewer training images are beneficial. We report mAP (500) and mAP (4000), and the trend is similar to Figure 5 (right) of the main text.
mAP (500): Before
After Tuning
mAP (1000); Before After Tuning
0.20
0.2
Before tuning
Before tuning
MAP 1500)
0.15
After tuning
MAP (1000)
0.1
After tuning
9.30
0.19
0.20
0.014
0.013 9.019
0.035 0.044
0.041
0.03
0.013
0.025
0.019
0.032
0.053
0.013
0.064
DINO
CUP Image
CLIP Text
DIND CLIP Text
DIND
CUP Image
CLIP Text
DINO+CLPP Text
Figure 16: Stable Diffusion ranking results (tuning feature spaces). We see similar trends as with MS-COCO, with the strongest performing embedding using both text and image features. However, text features are more necessary to yield strong performance in this setting. We report mAP (500) and mAP (1000), and the trend is similar to Figure 6 (right) of the main text.
mAP (500) vs. Dataset Size
MAP (1000) vs. Dataset Size
3.455
0,041
0.062
0.064
PMP (500)
0.039
MAP (1000)
0.06
3,959
0,034
0.053
0.029
0.05
3005
0.025
0.04
0.06
5.22
1000
2002
2003
4000
5000
1002
2020
3100
4002
5000
Dataset Size (number of queries)
Dataset Size (number of queries)
Figure 17: Stable Diffusion ranking results (dataset sizes). Performance increases with query images, increasing additional gains with more compute dedicated gathering attribution training data. We report mAP (500) and mAP (1000), and the trend is similar to Figure 6 (right) of the main text.
25

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: In the abstract and introduction, the claims are made to reflect the paper's contributions and scope accurately.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: See Section S
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [NA]
26

---

Justification:
The paper presents empirical findings and does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide all the information for reproducing our main results in the main
paper (Section 3 and the supplemental material. We will also release the code upon
publication.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
27

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We release code, model, and data htttpss///ptteeraaggii..c 10/
FastGDA
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.cc.
aublicc/qudees/Codestbbaas www.eeeesuumissioooooooooooo for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ wttps/cccppppccggua.aeaeaeccuuep
Saaeee/CCoSSSubisss..S.. www.ssssiicgcyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provided these details in the main paper (Section and the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report 1-standard-ent -eroor error bars in Figure 3 and Table We state that we are reporting error bars representing standard error in the captions.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
28

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report compute resources for our experiments the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer:
[Yes]
Justification: We fully conform with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss societal impacts in Section
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
29

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We conduct experiments existing generative models and data. We will release datz-attributioo--speiii feature encoder, which has low risk for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
In the supplemental material.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
30

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, batasets.papersstt paperswi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: Yes]
Justification:
We will release the code upon publication. No assets provided for this submission.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
31

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
32


Reviews:
[{"review_id": "TmQz8Sdv5E", "reviewer": "Reviewer_PCEG", "summary": "This paper addresses the computational bottleneck in data attribution for text-to-image models by proposing a learning-to-rank approach that distills slow but accurate attribution methods into fast feature embeddings. The authors leverage Attribution by Unlearning (AbU) as a teacher method and train a feature space that can identify influential training images through simple cosine similarity search. Their method achievessignificant speedup compared to existing approaches while maintaining competitive attribution performance.", "questions": "- Have you explored modifications to preserve absolute attribution scores rather than just rankings? This could be achieved through regression with appropriate normalization or by learning a calibration function. This information could be valuable for understanding whether influence is concentrated in a few images or distributed across many.\n\n- Also wondering if ensembling multiple different teacher ()attribution) models may achieve better results?", "limitations": "- This method can only work if one has access to the pretrained T2I model\n- One additional limitation, is the computational cost of the data curation phase, which still requires running expensive attribution methods", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "streghts first:\n- The paper tackles a critical challenge where existing attribution methods have oversized cost (compute/time), making real-world use infeasible. The proposed speedup makes attribution practical.\n- The two-stage data strategy (coarse retrieval followed by fine-grained ranking) balances computational efficiency with attribution accuracy. The learning-to-rank framework is appropriate for distilling ranking information from expensive teacher methods.\n- The evaluation is thorough, including both efficient rank prediction metrics and expensive counterfactual evaluation. The ablation studies investigate key design choices (feature spaces, loss functions, sampling strategies) with clear insights.\n- Finally the method achieves attribution performance comparable to influence-function and Unlearning-based methods while operating at millisecond latencies with minimal storage overhead.\n\nWeaknesses:\n- The approach is inherently limited by the teacher method's quality. Any biases, errors, or failure modes in AbU+ can be inherited, limiting the path to improve attribution beyond the teacher.\n- This is minor, nonetheless, from the perspective of training data creators, by learning only relative rankings rather than absolute attribution scores, the method loses potentially important information about influence magnitude and concentration. This could matter for applications like fair compensation where the degree of influence is critical. But that in itself would require massive scale and doesn't undermine the efficacy of this method or the results.\n- The initial k-NN retrieval using off-the-shelf features may miss influential images that are semantically related but visually dissimilar, creating an irrecoverable performance ceiling.", "was_revised": true, "final_justification": "The paper tackles a critical challenge where existing attribution methods have oversized cost (compute/time), making real-world use infeasible. The proposed speedup makes attribution practical. The rebuttal also quelled many weakness and questions i had.", "rebuttal": "Thank you for acknowledging that our paper addresses a critical challenge, proposes an appropriate learning-to-rank framework, offers a thorough evaluation, and achieves low-latency attribution with minimal storage requirements. We address your comments below.\n\n---\n**\"The approach is inherently limited by the teacher method's quality.\"**\nYes, we agree and have discussed this in the limitation (L333). This would be true for all distillation methods. Like all distillation methods, our method can be generally applied to different teacher methods, enabling future improvements in (slow) teachers to be distilled into fast students, using our procedure. \n\n---\n**\"Ensembling multiple different teacher (attribution) models may achieve better results?\"**\nThank you for your suggestion. We also believe that if an ensemble leads to better attribution rankings, we expect our method to perform better by distilling from those rankings. We plan to explore it in future work.\n\n---\n**Predicting relative instead of absolute attribution scores.**\nYes, we agree and have discussed this in the limitation (L330). Following your suggestion, we conducted a preliminary experiment on regressing the absolute attribution scores (normalized by mean and standard deviation). This regression leads to worse ranking performance than our learning-to-rank method. Below, we report mAP (1000) for comparison (higher is better):\n\n| Regress Absolute Score  ||| Ours |\n|:---:|--|--|:---:|\n| 0.009 ± 0.000 ||| **0.724 ± 0.021** |\n\n\nPredicting accurate absolute attribution scores remains challenging, and it will be interesting to explore improvements in this in the future. We will add this discussion in the revision. \n\n---\n**Effectiveness of K-NN retrieval using off-the-shelf features.**\nFollowing the reviewer’s suggestion, we study the effect of applying K-NN for data collection. As in Table 1 in the main text, we report counterfactual metrics for AbU+ ran on (1) the full training set (**AbU+**) and (2) top neighbors after K-NN (**AbU+ (K-NN)**). We also copied numbers from D-TRAK (2nd best teacher) and random baselines as reference:\n\n|  | Loss Change $\\uparrow$ [K=500] ($\\times 10^{-3}$) | Loss Change $\\uparrow$ [K=1000] ($\\times 10^{-3}$) | Loss Change $\\uparrow$ [K=4000] ($\\times 10^{-3}$) | MSE $\\uparrow$ [K=500] ($\\times 10^{-2}$) | MSE $\\uparrow$ [K=1000] ($\\times 10^{-2}$) | MSE $\\uparrow$ [K=4000] ($\\times 10^{-2}$) | CLIP $\\downarrow$ [K=500] ($\\times 10^{-1}$) | CLIP $\\downarrow$ [K=1000] ($\\times 10^{-1}$) | CLIP $\\downarrow$ [K=4000] ($\\times 10^{-1}$) |\n|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n| Random                | 3.5±0.0 | 3.5±0.0 | 3.5±0.0 | 4.1±0.1 | 4.1±0.1 | 4.0±0.1 | 7.9±0.0 | 7.9±0.0 | 7.9±0.0 |\n| D‑TRAK                | 5.4±0.2 | 6.6±0.2 | 9.6±0.3 | **5.9±0.2** | **6.4±0.3** | **7.8±0.3** | 7.3±0.1 | 7.1±0.1 | 6.4±0.1 |\n| AbU+                  | **5.8±0.2** | **7.1±0.2** | **11.0±0.3** | 5.6±0.2 | 6.2±0.2 | 7.5±0.2 | 7.2±0.1 | 6.8±0.1 | **5.8±0.1** |\n| AbU+ (K-NN)             | **5.8±0.2** | **7.1±0.2** | 9.7±0.4 | 5.4±0.2 | 6.3±0.3 | 6.9±0.3 | **7.1±0.1** | **6.7±0.1** | **5.8±0.1** |\n\n\nWe find that applying K-NN retrieval does not introduce a significant performance drop. We are happy to include this study in the revision.\n\n---\n**Access to the pretrained T2I model.**\nYes, our method assumes full access to the model, dataset, and training loss. This setup directly applies to a company running attribution on its in-house model. We also note that recent works on data attribution all follow the same setup [1,2,3].\n\n[1] Georgiev et al. The Journey, Not the Destination: How Data Guides Diffusion Models.\n\n[2] Zheng et al. Intriguing Properties of Data Attribution on Diffusion Models.\n\n[3] Wang et al. Data Attribution for Text-to-Image Models by Unlearning Synthesized Images.\n\n---\n**Computational cost of the data curation phase.**\nYes, the cost of data curation remains high, and we will discuss this in the limitations. We will clarify that our work focuses on achieving efficient attribution, with a low storage cost, **at inference time**. Currently, most attribution methods cannot handle user requests in real time, making it difficult to deploy in real-world use cases. In scenarios where an attribution system handles many daily user requests, our method is more favorable, since it provides a cost-efficient and fast solution at inference time."}, {"review_id": "75LdA1iXrm", "reviewer": "Reviewer_sTkC", "summary": "This paper proposes a scalable and efficient data attribution method for text-to-image generative models. The key idea is to distill an unlearning-based attribution method into a task-specific embedding space, enabling fast retrieval of influential training images via similarity search. The learned embedding is trained with a learning-to-rank objective over attribution scores generated by AbU+. The method is benchmarked on both MSCOCO and Stable Diffusion, demonstrating superior attribution accuracy while achieving high speedup over compared attribution methods.", "questions": "1. How does the teacher model limit the overall performance?\n2. Why are these two metrics selected, and are others infeasible due to the problem setting?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengthens**\n1. The idea of learning an embedding space specifically optimized for data attribution is novel. It bridges the gap between accurate attribution and similarity-based methods.\n2. The proposed method achieves faster attribution compared to unlearning or influence-function-based methods, making it practical for deployment in real-world applications.\n3. The experiments are comprehensive. They include ablations on feature types, loss functions and sampling strategies.\n\n**Weaknesses**\n1. The proposed method is fully supervised by AbU+ scores. However, the paper does not analyze whether the teacher model itself is stable or accurate across modalities. Any failure case in AbU+ would be learned.\n2. While the method achieves fast inference, it sacrifices the ability to interpret absolute attribution scores (only rankings are learned). Furthermore, constructing the training set still requires significant computational cost.\n3. Both evaluation metrics (ranking prediction accuracy and counterfactual forgetting) are aligned with the design of this paper. It remains unclear whether these metrics correlate well with real attribution quality.", "was_revised": true, "final_justification": "The paper proposes a novel and practical attribution method with solid empirical results. While it depends on AbU+ as the teacher and does not recover absolute scores, the design is well-motivated and the evaluation is thorough. Despite some limitations, the contribution is clear and valuable.", "rebuttal": "Thank you for acknowledging that our paper proposes a novel idea, offers a practical attribution application, and conducts comprehensive experiments. We address your comments below.\n\n---\n**\"Any failure case in AbU+ would be learned.\"**\nYes, we agree and have discussed this in the limitation (L333). This would be true for all distillation methods. Like all distillation methods, our method can be generally applied to different teacher methods, enabling future improvements in (slow) teachers to be distilled into fast students, using our procedure.\n\n---\n**Predicting relative instead of absolute attribution scores.**\nYes, we agree and have discussed this in the limitation (L330). Following Reviewer PCEG’s suggestion, we conducted a preliminary experiment on regressing the absolute attribution scores (normalized by mean and standard deviation). This regression leads to worse ranking performance than our learning-to-rank method. Below, we report mAP (1000) for comparison (higher is better):\n\n| Regress Absolute Score  ||| Ours |\n|:---:|--|--|:---:|\n| 0.009 ± 0.000 ||| **0.724 ± 0.021** |\n\nPredicting accurate absolute attribution scores remains challenging, and it will be interesting to explore improvements in this in the future. We will add this discussion in the revision. \n\n---\n**Computational cost of the training set construction.**\nYes, the cost of data curation remains high, and we will discuss this in the limitations. We will clarify that our work focuses on achieving efficient attribution, with a low storage cost, **at inference time**. Currently, most attribution methods cannot handle user requests in real time, making it difficult to deploy in real-world use cases. In scenarios where an attribution system handles many daily user requests, our method is more favorable, since it provides a cost-efficient and fast solution at inference time.\n\n---\n**\"Do these metrics correlate well with real attribution quality?\"**\nYes, the counterfactual forgetting metric directly measures the real attribution quality, which is widely adopted by AbU and other recent works [1,2,3,4]. As described in AbU [1], the metric evaluates attribution in the following way:\n\n>If an attribution algorithm accurately identifies influential training images for the generated image $\\hat{\\textbf{z}}$, then a model trained without those images would be incapable of generating or representing $\\hat{\\textbf{z}}$.\n\nMeanwhile, as discussed in L227-231, we use cheap rank prediction metrics (e.g., mAP (1000)) to evaluate design choices for ranking models. To reduce computation cost, we only perform expensive counterfactual metrics on the best-ranking models to confirm that our method indeed yields good attribution quality. We will clarify this in the revision. We are happy to consider other metrics suggested by the reviewer.\n\n[1] Wang et al. Data Attribution for Text-to-Image Models by Unlearning Synthesized Images.\n\n[2] Ilyas et al. Datamodels: Predicting Predictions from Training Data.\n\n[3] Park et al. TRAK: Attributing Model Behavior at Scale.\n\n[4] Georgiev et al. The Journey, Not the Destination: How Data Guides Diffusion Models."}, {"review_id": "pahIQZJhjK", "reviewer": "Reviewer_viaY", "summary": "This paper presents a method for efficiently identifying the training images that most influenced a generated image from a text-to-image model. \nThe primary challenge with existing data attribution techniques is their high computational cost, which makes them impractical for large-scale use.\nThe authors' key contribution is an approach that significantly speeds up data attribution. \nThey achieve this by \"distilling\" the knowledge from a slow but relatively accurate attribution method, AbU+, into a lightweight feature embedding model. \nOnce trained, this new model can quickly find influential training images by performing a fast similarity search in its learned feature space, eliminating the need to run the expensive attribution algorithm for every new query.\nThe paper demonstrates through extensive experiments on both the MSCOCO and the large-scale LAION datasets that their method achieves attribution performance comparable to or better than existing methods, but is thousands of times faster.", "questions": "See weaknesses", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. The paper addresses a highly relevant and timely problem. \nAs generative models become more widespread, methods for tracing generated content back to training data are crucial for addressing issues of copyright, artist compensation, and model transparency.\nExisting attribution methods are often too computationally expensive for practical deployment. \nThis work significantly contributes by dramatically reducing the runtime of attribution to a few milliseconds, making it faster. \n\n2. The authors validate their method on MSCOCO using counterfactual evaluation, which is the \"gold standard\" for attribution. \nThis involves removing the top-k identified training images, retraining the model, and measuring the degradation in its ability to produce the query image. \nThis is a costly but powerful way to demonstrate true influence. \nThe authors perform a systematic study of various design choices, providing clear justifications for their final model. This includes comparing different feature spaces to tune (e.g., DINO, CLIP Text, and their combination), evaluating multiple learning-to-rank objectives (MSE, Ordinal, Cross-Entropy), and analyzing the impact of data-scaling and sampling strategies. \n\n3. The successful application to Stable Diffusion/LAION demonstrates the method's scalability and its ability to handle modern, large-scale systems. \n\n4. The paper is well-written and easy to follow. The methodology is presented clearly, with helpful illustrations like Figure 2 that effectively communicate the overall pipeline. \n\nWeaknesses\n1. The core idea of distilling a slow data attribution method into a fast, feature-based model might not be novel. Previous papers have explored similar paradigms, and the authors did not discuss them [1, 2]. \n\n2. The LTR approach distills the relative ranking of influential examples but loses the raw, calibrated attribution scores produced by the teacher. This means that while the method can identify which images were most influential, it cannot quantify how much more influential one image is than another or determine if influence is concentrated in a few examples or diffuse among many. \n\n3. The authors did not adopt the widely used Linear Datamodeling Scores as on of their evaluation metrics, making it a bit difficult to compare with previous methods.\n\n[1] MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models. NeurIPS 2024\n\n[2] Data Selection via Optimal Control for Language Models. ICLR 2025", "was_revised": true, "final_justification": "The response has effectively addressed my primary concerns.", "rebuttal": "Thank you for acknowledging that our paper addresses a timely problem, provides a systematic study of design choices, demonstrates a scalable method, and presents clear methodology. We address your comments below.\n\n---\n**Comparison to previous works.**\nThanks for the suggestion! We will discuss them in the revision. There are several differences between their approaches and ours:\n\n- **The task is different.** They focus on data selection for a specified downstream task, where they assign a single influence score to each training point and select ones with high influence. We focus on training point influence for *any given synthesized image*. The training point influence is different per query synthesized image.\n\n- **The distillation method is different.** They take the training point feature as input and regress the single influence score. We use feature similarity between the training and synthesized image to predict influence, where we finetune features using learning-to-rank objectives to encourage similarities to have the correct rank.\n\n- **Modality is different.** They focus on language models, and we focus on text-to-image models.\n\n---\n**\"The LTR approach distills the relative ranking.\"**\nYes, we agree and have discussed this in the limitation (L330). Following Reviewer PCEG’s suggestion, we conducted a preliminary experiment on regressing the absolute attribution scores (normalized by mean and standard deviation). This regression leads to worse ranking performance than our learning-to-rank method. Below, we report mAP (1000) for comparison (higher is better):\n\n| Regress Absolute Score  ||| Ours |\n|:---:|--|--|:---:|\n| 0.009 ± 0.000 ||| **0.724 ± 0.021** |\n\n\nPredicting accurate absolute attribution scores remains challenging, and it will be interesting to explore improvements in this in the future. We will add this discussion in the revision.\n\n---\n**Linear Datamodeling Scores (LDS).**\nLDS was introduced by Park et al. [1], with the assumption that attribution methods are additive, where the influence of a group of training points is the sum of the individual training point influences. As noted by AbU [2], while the additive assumption sometimes holds true (e.g., influence functions), it does not hold true for many methods (e.g., AbU, feature similarity). \n\nMeanwhile, we did compare previous methods with the counterfactual forgetting metric, as used by many prior works [1,2,3,4]. We will clarify this in the revision.\n\n[1] Park et al. TRAK: Attributing Model Behavior at Scale.\n\n[2] Wang et al. Data Attribution for Text-to-Image Models by Unlearning Synthesized Images.\n\n[3] Ilyas et al. Datamodels: Predicting Predictions from Training Data.\n\n[4] Georgiev et al. The Journey, Not the Destination: How Data Guides Diffusion Models."}, {"review_id": "p27sxbZkfF", "reviewer": "Reviewer_QpB6", "summary": "This paper proposes a novel approach for scalable and efficient data attribution. The key idea is to distill a slow, unlearning-based attribution method to a feature embedding space for efficient retrieval of highly influential training images. Good experimental results have been reported. The authors claim \"our method can achieve better or competitive performance in a few seconds, faster than existing methods by 2,500× ∼ 400,000×.\"\nIf this is true, it's impressive. On the other hand, it's not possible to verify here. The author stated at the end of their abstract: \"Our code, models, and datasets will be made publicly available.\"", "questions": "The paper reports good performance of their methods; however, the justification is not convincingly presented. The proposed method is not rigorously presented either. See early comments on this. Since the source codes are not made available, it is impossible to validate their results. If the authors want to make a convincing case, they may want to consider releasing their code and the code for their experiments, so that others can reproduce their results.", "limitations": "There are some discussions under Section 5 Discussion, Broader Impacts, and Limitations", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The paper is okay up to the bottom of page 3. Then it becomes very vague to understand. \nFor example, in equation (1) at the bottom of page 3, the symbols in the equation are not adequately defined. Readers have to imagine what they are by following some traditions. However, it's the authors' responsibility to declare them. \nThe following content, from the top of page 4 to the end of Section 3, is presented in a handwaving style without providing a rigorous description. \nPerhaps the authors could improve by providing a clear and readable algorithm here. \nWithout knowing exactly what the method is, it is hard to appreciate its reported success. \n\nThis reviewer decides to recommend negatively to this submission, mainly because, based on the submitted work, it is hard to see what the proposed algorithm is. Additionally, since the work is not reproducible, it is impossible to validate the numerical success declared in the abstract, namely \"demonstrating that our method can achieve better or competitive performance in a few seconds, faster than existing methods by 2,500× ∼ 400,000×.\" Considering the paper's main contribution is the declared experimental success, which is not verifiable.", "was_revised": true, "final_justification": "See my response to the authors.", "rebuttal": "Thanks for the helpful suggestions and comments. Below are our responses.\n\n**Code release.** As mentioned in the abstract and in the review, we will release code, data, evaluation script, and models to ensure reproducibility upon publication.\n\n**Writing improvement.** Thank you for suggesting writing improvement for our method. While other reviewers didn’t raise issues with the writing, we agree that it can be polished, and we are happy to provide any further clarifications during the discussion period.\n\n**Symbols in Equation 1 (AbU+).**\nIn Eqn 1, $\\theta_0$ and $\\hat z$ are defined in L138, and $\\theta_{-\\hat z}$, $\\alpha$, and $F$ are defined in L139. $N$ is the size of the training set. $\\mathcal L$ is the training loss of text-to-image diffusion model, as defined in Section A.1 in the supplemental. In the revision, we will more clearly refer to Section A.1 and define all the symbols.\n\n**\"Clear and readable algorithm.\"** For reference, we provide a rigorous algorithm block below, which will be added to the revision.\n\n\n\n---\n*(We use mathjax in this response. Please refresh the page if it's not properly rendered, thanks!)*\n>**Algorithm 1 Data-curation (Sec. 3.1).**\n>\n>For distillation, we curate a teacher set with **AbU+**. For each synthetic query, we first retrieve the top-$K$ neighbors to narrow the search space, then compute AbU+ scores only on that subset (optionally subsampling to size $M$) to improve efficiency. Finally, we rank all scored neighbors for that query and store the ranks. We normalize the rank to $[0, 1]$.\n>\n> **Input**  \n> – $ \\mathcal D=\\{(x_i,c_i)\\}_{i=1}^{N} $:&nbsp;&nbsp; Training set (image–caption pairs) \\\n> – $ \\phi $:&nbsp;&nbsp; Frozen feature encoder  for retrieval  \\\n> – $ \\tau(\\hat z,z) $:&nbsp;&nbsp; Attribution scores from slow teacher AbU+  \\\n> – $ \\mathcal C $:&nbsp;&nbsp; Caption set  to generate query images  \\\n> – $ K $:&nbsp;&nbsp; Retrieval cut-off \\\n> – $ M $:&nbsp;&nbsp; Subsample size\n>\n>  **Algorithm**\n> 1. Initialize teacher attribution set $ \\mathcal U \\leftarrow \\varnothing $.  \n> 2. **For each** caption $ c\\in\\mathcal C $:  \n> 3. &nbsp;&nbsp;&nbsp;&nbsp;Generate synthetic image $ \\hat x $ and form query $ \\hat z=(\\hat x,c) $.  \n> 4. &nbsp;&nbsp;&nbsp;&nbsp;Retrieve top-$K$ neighbors $ \\mathcal D_{\\hat z}\\subset\\mathcal D $ of $ \\hat z $ using $ \\phi $.  \n> 5. &nbsp;&nbsp;&nbsp;&nbsp;Uniformly subsample $ \\tilde{\\mathcal D_{\\hat z}}\\subset\\mathcal D_{\\hat z} $ with $ |\\tilde{\\mathcal D_{\\hat z}}| = M $.  \n> 6. &nbsp;&nbsp;&nbsp;&nbsp;Initialize list $ \\mathcal S_{\\hat z}\\leftarrow\\varnothing $.  \n> 7. &nbsp;&nbsp;&nbsp;&nbsp;**For each** $ z_i\\in\\tilde{\\mathcal D_{\\hat z}} $:  \n> 8. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Compute score $ s_i=\\tau(\\hat z,z_i) $ and append $s_i$ to $ \\mathcal S_{\\hat z} $.  \n> 9. &nbsp;&nbsp;&nbsp;&nbsp;Sort $ \\mathcal S_{\\hat z} $ in **descending** order by $ s_i $ to assign ranks $ \\pi^i_{\\hat z}=1,2,\\dots,|\\mathcal S_{\\hat z}| $ to each training point $z\\in\\tilde{\\mathcal D_{\\hat z}}$.  \n>10. &nbsp;&nbsp;&nbsp;&nbsp;**For each** $ z_i\\in\\tilde{\\mathcal D_{\\hat z}} $ and its rank $\\pi^i_{\\hat z}$:  \n>11. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Normalize and update $ \\pi^i_{\\hat z} \\leftarrow \\dfrac{\\pi^i_{\\hat z}}{|\\mathcal S_{\\hat z}|} $.  \n>12. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Append $ (\\hat z,z_i, \\pi^i_{\\hat z},\\mathcal D_{\\hat z}) $ to $ \\mathcal U $.  \n>\n> **Output** Curated teacher set $ \\mathcal U $ containing tuples $ (\\hat z,z_i, \\pi^i_{\\hat z},\\mathcal D_{\\hat z})$.\n>\n>(We introduce a new symbol $\\mathcal U$ for clarity, and other symbols follows the same convention in the submission.)\n\n\n---\n\n>**Algorithm 2 Training the fast attribution embedder  (Sec. 3.2).**\n>\n>Using the teacher set $\\mathcal U$, we finetune features so that the feature similarity between the training and synthesized image predicts the ranks.\n>\n> **Input**  \n> – $ \\mathcal D$:&nbsp;&nbsp; Training set   (from Algorithm 1) \\\n> – $ \\mathcal U $:&nbsp;&nbsp; Teacher set  (from Algorithm 1) \\\n> – $ \\phi $:&nbsp;&nbsp; Frozen encoder \\\n> – $ g_\\psi $:&nbsp;&nbsp; Trainable MLP \\\n> – $ (\\alpha,\\beta) $:&nbsp;&nbsp; Trainable logit scale \\\n> – $ p_n $:&nbsp;&nbsp; Probability of sampling non-neighbor images *(for learning to assign lower rank to unrelated image)* \\\n> – $ \\eta $:&nbsp;&nbsp; Learning rate \\\n> – $T$ :&nbsp;&nbsp; Training iteration \n>\n> **Algorithm**  \n> 1. **Repeat** for $T$ iterations:  \n> 2. &nbsp;&nbsp;&nbsp;&nbsp;Sample $ (\\hat z, z_i, \\pi^i_{\\hat z},\\mathcal D_{\\hat z}) $ from $ \\mathcal U $.\n> 3. &nbsp;&nbsp;&nbsp;&nbsp; `Sample non-neighbor` With probability $ p_n $: \\\n> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;  (a) `replace data with a non-neighbor` $z_i \\leftarrow z \\sim \\mathcal D /\\mathcal D_{\\hat z}$ \\\n> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;  (b) `set the lowest rank to the non-neighbor` $\\pi^i_{\\hat z} \\leftarrow 1$ \n> 4. &nbsp;&nbsp;&nbsp;&nbsp;`Get features` $f_\\psi(\\hat z) = g_\\psi\\bigl(\\phi(\\hat z)\\bigr),\\quad f_\\psi(z_i) = g_\\psi\\bigl(\\phi(z_i)\\bigr) $.  \n> 5. &nbsp;&nbsp;&nbsp;&nbsp;`Similarity` $ r_\\psi(\\hat z, z_i) = \\cos\\bigl(f_\\psi(\\hat z), f_\\psi(z_i)\\bigr) $.  \n> 6. &nbsp;&nbsp;&nbsp;&nbsp;`Predicted rank`  $ \\sigma_{\\alpha,\\beta}(r_\\psi(\\hat z, z_i)) = \\operatorname{sigmoid}(\\alpha\\odot r_\\psi(\\hat z,z_i) + \\beta) $.  \n> 7. &nbsp;&nbsp;&nbsp;&nbsp;`BCE Loss`  $ \\mathcal L(\\psi, \\alpha, \\beta) = \\ell_\\text{BCE}\\bigl(\\pi^i_{\\hat z}\\ ,\\ \\sigma_{\\alpha,\\beta}(r_\\psi(\\hat z, z_i))\\bigr) $.  \n> 8. &nbsp;&nbsp;&nbsp;&nbsp;Update $ \\psi,\\alpha,\\beta $ using $ \\nabla\\mathcal L $ with AdamW optimizer with learning rate $ \\eta $.\n> 9. `Store the optimized parameters` $ \\psi^\\* \\leftarrow \\psi$, $ \\alpha^\\* \\leftarrow \\alpha$, $ \\beta^\\* \\leftarrow \\beta$.\n>\n> **Output** Trained $ \\psi^\\*, \\alpha^\\*, \\beta^\\* $; pre-compute and index $ f_{\\psi^\\*}(z_i) $ for all training images $z_i \\in \\mathcal D$.\n>\n>(Besides $\\mathcal U$, we introduce $p_n$, $T$ for clarity, and other symbols follow the same convention in the submission.)"}]

Author rebuttal:

Meta-review: All reviewers are in agreement that the proposed method of distilling attribution methods into a feature embedding. There was concern about the presentation and the justification, but given that multiple reviewers recommend the presentation to be a strength of the paper, which the AC's quick read also aligns with, the AC recommends accepting the paper.
Decision: Accept (poster)