Paper: DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models (ICLR-2024)

Reviewer comment (Evaluation):
One of the main applications of influence functions is to find the most influential samples whose removal would significantly change model's behavior in inference time. The way this paper evaluates the most influential samples is not appropriate. In fact, we do expect influential samples to come from the same label but that is not enough.

Author response:
We greatly appreciate the reviewer for these thought-provoking comments. While we have systematically evaluated the quality of the influence function methods through approximation error analysis and mislabeled data detection tasks with repetitions, we agree with the reviewer that a good influence method is desired to perform well in the data selection task. To address your comment, we have added the data selection experiment in Appendix F (the last page of the revision)

As a brief summary, we find that DataInf is very effective in data selection, demonstrating that it yields better model performance than existing influence function methods. In particular, a subset selected by DataInf often yields a better model than that based on the full dataset when the dataset is noisy. Furthermore, we find that a subset selected by DataInf achieves the best model performance in the first few epochs and is able to accelerate the model training. We believe DataInf selectively finds a good subset by excluding ambiguous or mislabeled data points, and hence it can help accelerate model training.

Thank you very much again for your time and constructive feedback! We hope you would consider increasing your score if we have answered your questions. Please let us know if you have additional questions and we are happy to follow up. Thanks!

Reviewer comment (Experiments):
There is a recent line of work that casts doubt on the usefulness of approximating influence functions. They argue that for image generation tasks, utilizing an off-the-shelf visual encoder can be more helpful in identifying influential samples. I would expect to see a comparison between the proposed method and those kinds of methods [1, 2].

Author response:
Thank you for this comment. Data contribution estimation for general deep neural network models, not just image generation models, is an important open question, so we would like to highlight that our main focus is to propose a practical approximation for the influence function that can be applied to any deep neural network model. As a result, our method is designed to be more general. Moreover, [2] has not been published in a conference yet.

As the reviewer mentioned, existing influence function methods have been recognized as less useful, and we believe it is because the approximation is inaccurate. It has been actually observed in our work as well: Figure 1 demonstrates that existing methods are less accurate than DataInf, and Figure 2 illustrates DataInf outperforms them in downstream tasks.

Reviewer comment (Evaluation):
Are there any specific model configurations where you would advise not using DataInf?

Author response:
We believe there is no particular model configuration that limits the use of DataInf. However, as our Theorem 1 shows, the dimension of parameters can largely affect the approximation errors. Hence, parameter-efficient fine-tuning techniques (e.g., LoRA) are recommended when DataInf is applied.

Reviewer comment (Experiments):
Do LoRA finetuning methods typically only use low ranks like 2,4,6 as used in the experiments in the paper?

Author response:
The optimal selection of low rank should highly depend on datasets and downstream tasks. However, in Table 6 of Hu et al. (2021), the original authors mentioned that a low rank (r=1,2,4, and 8) can be sufficient to achieve competitive performance for general NLP downstream tasks. Moreover, Figure 6 of our paper empirically supports this conclusion: the downstream performance does not vary much across different rank values.

Reviewer comment (Experiments):
The paper has rightly pointed out that there aren't many qualitative metrics for measuring the utility of influence scores. The authors try to address it through proxies. The results in Figure 2 are presented with only rank 4, what do the results look like for other rank values?

Author response:
Thank you for this question. The submitted manuscript actually included mislabeled data detection results for different rank values. Specifically, Figure 6 shows three different cases with the low-rank equals 1, 2, or 8. We observed a consistent pattern with different ranks: DataInf achieves better AUC than LiSSA and hessian-free methods, and sometimes better than the exact influence function. We have clarified this in the revision.

Reviewer comment (Presentation):
Curious as to what is the computational and memory complexity of Hessian free methods like Tracin which are omitted from the table?

Author response:
In terms of computational and memory complexity, the Hessian-free method has exactly the same rate as ours. We decided not to include this method as it essentially ignores the Hessian part of the influence function, yielding too crude approximation.

Reviewer comment (Reproducibility):
How did you obtain multiple training checkpoints required for techniques like tracin in your experiments? Esp for the llama model, are the checkpoints publicly released or did you do your own fine-tuning and use the fine-tuning checkpoints? Do the results for tracin improve with more checkpoints?

Author response:
Thank you for this question. We used only the last checkpoint of a fine-tuned model. We expect using multiple checkpoints may better describe the influence of data points, but it requires more expensive memory costs. For instance, storing all gradients of a pre-trained Llama2 model at one checkpoint requires 25 GB in our sentence transformation task. If we use $k$ checkpoints, it will require $25 \times k$ GB.

Reviewer comment (Theory):
One assumption made by the paper is the first equation on page 3, which states that the expectation of hessian is equal to the expectation of the second moment of gradients. This assumption only holds true when the loss function is $-\log P(y|f_\theta(x))$, where $f_\theta(x)$ denotes the output probability of network parameterized by $\theta$. However, consider the cross entropy loss function, which is $y \log f_\theta(x)$. There is no discussion of how the assumption applies to common loss functions.

Author response:
We would like to clarify that the cross-entropy loss function is actually a negative log-likelihood function in classification settings. To be more specific, suppose we have a $K$-class classification problem and $f_{\theta} (x) \in [0,1]^{K}$ is a probability estimate for an input data $x$. Then, the correct mathematical notation for the cross-entropy loss function is the $y$-th element of $ -\log f_{\theta} (x) $, not $ y \log f_{\theta}(x)$, when the associated label $y \in \set{1, \dots, K}$. That is, the cross-entropy loss function is a negative log-likelihood function, and for this reason, we use it in our experiments.

For a general loss function, the Hessian matrix is not simplified since Bartlett’s second identity does not hold. As a result, DataInf might not be applicable to a general loss function, but we would like to emphasize that a negative log-likelihood function (i.e., the cross-entropy loss function) is the most commonly used loss function—many popular LLMs have utilized the cross-entropy loss function. We will include this discussion in the revision.

Reviewer comment (Experiments):
Section 4.1: What is the correlation analysis result when the data points are all clean?

Author response:
We greatly appreciate the reviewer's constructive suggestion We have added a new correlation analysis result with a clean dataset in the revision (Figure 5). We observed that DataInf is more correlated with the exact influence function than existing methods, demonstrating the consistency of our results regardless of the presence of noisy data.

Reviewer comment (Evaluation):
Section 4.2: Why does exact influence sometimes underperform the approximation methods? How does the result vary with the mislabeling ratio?

Author response:
Thank you for this question. It is expected that mislabeled data are well detected by the exact influence function as they likely have a large negative impact on validation accuracy. However, we would like to emphasize that it is not necessarily the most effective method to detect low-quality data points. The exact influence function is not guaranteed to show the best performance (as shown Jiang et al. (2023), Park et al. (2023), and Ilyas et al. (2022)), and our method shows promising results for some datasets. 

Thank you again for your time! We hope you would consider increasing your score if we have answered your questions. Please let us know if you have additional questions and we are happy to follow up. Thanks!

#### Reference
- Jiang, K. F., Liang, W., Zou, J., & Kwon, Y. (2023). OpenDataVal: a Unified Benchmark for Data Valuation. arXiv preprint arXiv:2306.10577., 
- Park, S. M., Georgiev, K., Ilyas, A., Leclerc, G., & Madry, A. (2023). Trak: Attributing model behavior at scale. arXiv preprint arXiv:2303.14186.
- Ilyas, A., Park, S. M., Engstrom, L., Leclerc, G., & Madry, A. (2022). Datamodels: Predicting predictions from training data. arXiv preprint arXiv:2202.00622.