Quantifying Memorization and Detecting Training Data of Pre-trained Language Models using J apanese Newspaper (2024)

Abstract: Dominant pre-trained language models (PLMs) have demonstrated the potential risk of memorizing and outputting the training data. While this concern has been discussed mainly in English, it is also practically important to focus on domain-specific PLMs. In this study, we pre-trained domain-specific GPT-2 models using a limited corpus of Japanese newspaper articles and evaluated their behavior. Experiments replicated the empirical finding that memorization of PLMs is related to the duplication in the training data, model size, and prompt length, in Japanese the same as in previous English studies. Furthermore, we attempted membership inference attacks, demonstrating that the training data can be detected even in Japanese, which is the same trend as in English. The study warns that domain-specific PLMs, sometimes trained with valuable private data, can ”copy and paste” on a large scale.

## introduction as pre-trained language models (plms) have become increasingly practical, critical views on the memorization of plms are emerging in security and copyright @xcite @xcite . prior research has indicated that neural networks have the property of unintentionally memorizing and outputting the training data @xcite @xcite @xcite . in particular, @xcite demonstrated that memorized personal information can be detected from gpt-2 models @xcite . this can lead to an invasion of privacy, reduced utility, and reduced ethical practices @xcite . if there is no novelty in the generation, there would be a problem with copyright @xcite . 1 an early version of this study was accepted for nonarchival track of the fourth workshop on trustworthy natural language processing @xcite . research on memorization of plms has been intensively advanced, and empirical findings have been reported @xcite . initial studies remain on the qualitative side @xcite , and subsequent studies have begun to focus on quantitative evaluations. according to one of the first comprehensive quantitative studies @xcite , the memorization of plms is strongly related to the string duplications in the training set, model size, and prompt length. benchmarking of memorized string detection has also progressed, including constructing evaluation sets @xcite @xcite . these studies were conducted in english, and their reproducibility is uncertain under domainspecific conditions. domain-specific plms are sometimes built on rare private corpora and have smaller pre-training corpora than general plms. when the data size is small, models tend to be pre-trained in multiple epochs. however, increasing the number of epochs is equivalent to string duplications, which risks increased memorization. furthermore, security and copyright considerations become increasingly important, as the memorized contents tend to be more specific than general corpora. we, therefore, pose the following practically significant questions about domain-specific plms: how much of the pre-training data is memorized, and is the memorized data detectable? this study is the first attempt to quantify the memorization of domain-specific plms using a limited corpus of japanese newspaper articles. our research objective is to identify the memorization properties of domain-specific plms. first, we developed a framework for quantifying the memorization and detecting training data of plms using japanese newspaper articles (section 3). we then pre-trained domain-specific gpt-2 models and quantified their memorization (section 4). furthermore, we addressed membership inference at-tacks @xcite , which predicts whether the output string was included in the training data (section 5). the main findings and contributions of this paper are summarized as follows. ## related work this section reviews related work and highlights the position of this study. ## problem statement & methodology this section explains the problem addressed in this study and the methodology (figure 2 ). we use a prompt pre-training generation candidate ■■■■■■■■ ■■■■■■■■ ■■■■■ …… ■■■■■■■■ ■■■■■■■■ ■■■■■■■■ public part is a prompt. newspaper paywall: private part can be used for evaluation. reference training set evaluation new texts figure 2: the procedure of quantifying the memorization and training data detection of plms in this study. first, we pre-trained gpt-2 models using newspaper articles as a training set. we then generated strings using the public part as a prompt. the memorization was quantified using the private part. we also tackle the training data detection task, using articles used for pre-training as positive examples and not as negative examples. ## experiment 1: quantification this section reports our findings from experiments under various conditions. first, multiple plms and the evaluation set were prepared, and then memorization was quantified. we analyzed the results from a quantitative and qualitative perspective. ## experiment 2: detection this section demonstrates that memorized strings can be detected from japanese plms. specifically, we investigated whether detecting training data from japanese plms is possible using the proven min-k% prob in english. we targeted the domainspecific gpt-2 models (1, 5, 15, 30, and 60 epochs) described in the previous section. ## conclusion this study is the first attempt to quantify the memorization and detect training data of domain-specific plms that are not english but japanese. although our study has some limitations, this is a major step forward, as there is even a scant discussion of string similarity concerning the memorization of domainspecific plms.