## 3 Training Data Extraction for Large Language Models
### Ranking Strategy  
After generating many text samples, it is necessary to rank these samples to determine which samples have a higher probability of belonging to the training dataset. This ranking is typically achieved through a perplexity-based membership inference attack.  
**Loss-based MIA [34]**: Loss-based MIAs exploit the output loss of a model to deduce whether a particular sample is included in its training dataset. Specifically, the attacker compares the target sample with samples in the training dataset by observing the loss value of the model. A close alignment between the loss value of the target sample and those of training data samples strongly suggests that the target sample is likely a member of the training data.  
**Zlib entropy-based MIA [35]**: Zlib entropy-based MIAs leverage the Zlib compression algorithm [36] to evaluate the information entropy of text samples, thereby inferring whether the text belongs to the model's training dataset. Specifically, for a given generated sample, the Zlib entropy is calculated as follows:  
\[E_{Zlib}=\frac{S_{original}}{S_{compressed}} \tag{7}\]  
where \(S_{original}\) and \(S_{compressed}\) denote the file sizes of the generated sample before and after compression through the Zlib compression algorithm, respectively.  
Generally, lower Zlib entropy in the original text indicates more information, whereas higher Zlib entropy suggests less information in the original text. Zlib entropy-based MIAs typically follow two implementation approaches, as calculated in Equations (8) and (9),respectively, where \(P^{\prime}\) is the perplexity of the generated sample after Zlib entropy processing. One is to divide perplexity by Zlib entropy, favoring generated text with higher Zlib entropy, which indicates greater repetition within the text.  
\[P^{\prime}=\frac{P}{E_{Zlib}} \tag{8}\]  
The other is to multiply the perplexity by Zlib entropy, favoring generated text with lower Zlib entropy, thereby allowing the generated text to contain more information.  
\[P^{\prime}=P*E_{Zlib} \tag{9}\]  
In the sorting process, generated texts with the smaller \(P^{\prime}\) will be selected as training data.  
**Neighborhood comparison**: Drawing inspiration from reference-model-based membership inference attacks and addressing the challenge of requiring high-quality training data, [PERSON] et al. proposed a novel neighborhood-based membership inference attack [20]. Given a target sample \(x\), multiple highly similar neighborhood samples are generated by substituting words using a pretrained masked language model, specifically BERT, in this paper. Both the generated neighborhood samples and the original sample are inputted into the target model to calculate their respective loss values. Since the target model is prone to overfitting to the training data, the difference between the loss value of the target sample for the training data and the average loss value of the neighborhood samples is expected to be smaller than for other samples. The decision rule is as follows:  
\[A_{f_{\theta}(x)}=1[(L(f_{\theta},x)-\sum_{i=1}^{n}\frac{L(f_{\theta},\vec{x} _{i})}{n})<\gamma] \tag{10}\]  
where \(\{\widetilde{x}_{1},\ldots,\widetilde{x}_{n}\}\) is a set of n neighbors for \(x\).