Enhancing Training Data Attribution for Large Language Models with Fitting Error Consideration (2024)

Abstract: The black-box nature of large language models (LLMs) poses challenges in interpreting results, impacting issues such as data intellectual property protection and hallucination tracing. Training data attribution (TDA) methods are considered effective solutions to address these challenges.Most recent TDA methods rely on influence functions, assuming the model achieves minimized empirical risk. However, achieving this criterion is difficult, and sourcing accuracy can be compromised by fitting errors during model training. In this paper, we introduce a novel TDA method called Debias and Denoise Attribution (DDA), which enhances influence functions by addressing fitting errors. Specifically, the debias strategy seeks to improve the performance of influence functions by eliminating the knowledge bias present in the base model before fine-tuning, while the denoise strategy aims to reduce discrepancies in influence scores arising from varying degrees of fitting during the training process through smoothing techniques.Experimental results demonstrate that our method significantly outperforms existing approaches, achieving an averaged AUC of 91.64%. Moreover, DDA exhibits strong generality and scalability across various sources and different-scale models like LLaMA2, QWEN2, and Mistral.

## introduction large language models (llms) represent a novel paradigm of pre-training followed by fine-tuning, which is premised on initially training a model with a massive number of parameters on an extensive corpus to establish a foundational model @xcite . llms achieve a significant breakthrough in the scale of training data and the quantity of model parameters @xcite , thereby substantially enhancing the overall capability of model in executing complex tasks @xcite @xcite . however, llms still face a series of challenges and issues in practical applications. the complexity, black-box nature, and randomness of the training process of these models make it difficult to accurately understand and control their behavior and output @xcite . this presents significant challenges for the interpretability of llms, the protection of intellectual property rights associated with data, and the tracking of model-generated hallucinations, thereby posing potential risks to the deployment and use of llms @xcite @xcite . the numerous challenges and issues faced by llms in practical applications have garnered widespread attention from the research community @xcite . to ensure their lawful, fair, and trustworthy use, researchers are continuously exploring methods to address these problems @xcite . previous work has partially explained the model outputs from the perspective of training data, including methods such as influence functions @xcite @xcite @xcite and watermark embedding @xcite . however, the watermark embedding methods assume an overly idealized scenario, requiring specific interventions during the training phase, which significantly limits its applicability @xcite . consequently, for tda in deep learning models, the methods based on influence functions are preferred due to their robust theoretical foundation, ability to provide high-precision, fine-grained attribution, and the absence of a need for additional modifications to the training data or the model itself, making them applicable across a wide range of scenarios. nevertheless, the methods based on influence functions are highly dependent on the accuracy of gradient approximation @xcite @xcite . due to the training process of llms often failing to meet the criteria for empirical risk minimization (erm) @xcite , methods based on tda utilizing influence functions are easily impacted by fitting errors encountered during model training, resulting in reduced effectiveness in attributing training data. to address issues of influence functions, we propose a method called debias and denoise attribution (dda) to correct the influence function by reducing the impact of fitting errors. first, we continue training llms from a base model to obtain multiple versions of llms with different fitting errors (training losses). in the debias phase, we correct each influence score by subtracting the influence score of the base model. next, in the denoise phase, we eliminate discrepancies in the sourcing results by averaging the rectified influence scores of all llm versions. theoretically, we prove that dda can align the influence functions more closely with the erm theoretical conditions, thereby enhancing the effectiveness of influence function-based tda tasks. however, evaluating training data attribution tasks is not well-defined and very hard @xcite , thus we transform it into a hallucination tracing task. specifically, we construct a hallucination dataset named "hallucination xsum" and assess the performance of llms in tracing training data under hallucination conditions to measure the effectiveness of the baseline and dda methods. this transformation approach allows us to more effectively evaluate the accuracy and reliability of training data attribution methods. experimental results show that dda significantly outperforms methods such as bm25 @xcite , tracin @xcite , trak @xcite , and cea @xcite in training data attribution, with an auc value as high as 93.49%. moreover, dda demonstrates its generality and scalability across models like llama2 @xcite , qwen2 @xcite . our main contributions can be summarized as follows: ## related work the existing methods for training data attribution can be broadly categorized into two types: watermark-based and influence-based attribution. ## preliminaries in this section, we will introduce the background information and foundational knowledge necessary for understanding the core content of the paper. at the same time, we will define some symbols, variables, and abbreviations. ## analysis in this section, we conduct additional experiments to provide a more comprehensive analysis of dda. first, we examine the effect of the bias coefficient β in eq. ( 9 ) on the performance of dda. next, we perform ablation experiments to analyze the tda benefits brought by two different strategies in dda. finally, we conduct a case analysis of dda to comprehensively understand the tda phenomenon of dda on llms. ## conclusions the interpretability of llms, protection of data intellectual property, and tracing of model hallucinations are currently facing significant challenges. to ensure the legal, fair, and trustworthy use of training data, current methods typically rely on influence functions for training data attribution. however, these influence functions assume that the model training process adheres to empirical risk minimization, a condition often unmet during llm training. consequently, significant fitting errors arise during the training process, which severely impacts the accuracy of tda methods that rely on influence functions. to address this issue, we propose an attribution method called dda, which combines debias and denoise strategies to reduce the impact of fitting errors on tda performance. our method not only achieves sota performance compared to baseline methods but also exhibits strong robustness across various parameter scales and different llm sources. additionally, we conduct detailed analytical experiments to investigate the effects of hyperparameters and various strategies in dda. our research provides a novel perspective for enhancing the accuracy and reliability of tda methods for llms. it significantly contributes to ensuring the interpretability and data security of llms and offers a potential solution to the hallucination problem in llms.