## 3 Training Data Extraction for Large Language Models
### Supfix Generation Strategy  
In the first stage of training data extraction, we can generate texts by adjusting different suffix generation strategies. Different suffix generation strategies affect the results of generated text and thus the accuracy of extraction, and these suffix generation strategies can also be combined with each other to generate text. Language models can generate new text \(\widehat{x}_{i+1}\sim f_{\theta}(x_{i+1}\mid x_{1},\ldots,x_{i})\) through iterative sampling, and then re-input the sampled text \(x_{i+1}\) into the model to obtain \(\widehat{x}_{i+2}\sim f_{\theta}(x_{i+2}\mid x_{1},\ldots,x_{i+1})\). This process is repeated until a stopping condition is met. In the domain of large language model generation, the most common decoding strategy is to maximize the probability of text generation, usually referred to as the greedy strategy. While this method is effective in ensuring high-probability text output, ostensibly catering to the objective of extracting training data from large language models, it inevitably sacrifices textual diversity. Moreover, the pursuit of theoretically optimal sequences from the model is impractical due to the infeasibility of exhaustively generating and ranking all potential sequences. Therefore, a prevalent alternative involves the employment of beam search techniques. **Beam-search**: Beam search [28] retains only a predetermined optimal subset of partial solutions. Specifically, beam search maintains a candidate set of size \(n\). At each step, it selects the top \(n\) words from the probability distribution of each candidate sequence and then retains the \(n\) candidate sequences with the highest overall probabilities. This method strikes a balance between the quality and diversity of the generated text. However, due to its tendency to retain only the top \(n\) optimal solutions at each step, beam search often results in generated text that lacks diversity leading to outputs that may appear overly conservative and unnatural. **Top-k [29]**: This method involves sampling from the top \(k\) tokens, thereby granting lower probability tokens an opportunity for selection. This introduction of randomness often enhances the diversity of the generated text. Specifically, at each step, random sampling is executed from the \(k\) highest probability words, excluding those with lower probabilities. The sampling probability among the top \(k\) words is determined by their respective likelihood scores. **Top-p [30]**: At each step, random sampling is conducted only from the smallest set of words whose cumulative probability exceeds a certain threshold \(p\), disregarding words with lower probability. This approach is similar to the top-k method, but top-p dynamically adjusts the size of the token candidate list. Therefore, this strategy helps to mitigate the selection of inappropriate or irrelevant words while preserving the potential for interesting or creative outcomes. **Typical-p**: The typical-p [31] sampling method, grounded in information theory, constructs the sampling space by initially including words with the lowest conditional entropy. This process continues until the cumulative probability of all words in the sampling space surpasses a predefined threshold \(p\). Specifically, this method begins by calculating the conditional entropy for each word and subsequently sorting the vocabulary based on these entropy values. Words with the lowest conditional entropy are progressively selected until their cumulative probability exceeds the threshold. Sampling is then performed according to this newly established probability distribution. The primary advantage of the typical-p sampling method is its ability to enhance the typicality of the generated text, thereby producing sequences that are more representative of natural language. **Temperature [32]**: Temperature \(T\) represents a strategy for adjusting probability distributions, employing a local re-normalization method with an annealing factor.