## 3 Materials and Methods
### Methods
#### 3.2.4 Evaluation Metrics  
The evaluation of large language models covers multiple dimensions [38], including language generation ability, task performance, comprehension depth, and logical reasoning. Automatic evaluation employs computational metrics to quantitatively assess both the quality of generated text and overall task performance. Common evaluation metrics include BLEU [39], ROUGE [40], BERTScore [41], QAGS [42], FactCC [43], and FEQA [44], which measure aspects such as text similarity, semantic consistency, and fluency. However, BLEU rely on precise n-gram matching and are less sensitive to semantic equivalence and syntactic variations. Furthermore, although QAGS and FEQA can indirectly measure text fidelity through question-answer generation, their question-generation process often introduces significant uncertainty, and their correlation with human judgment remains limited in practical applications. For these reasons, we have chosen not to use BLEU, QAGS, and FEQA, and instead focus on metrics that more accurately capture semantic and factual consistency. In this study, we employ ROUGE, BERTScore, and FactCC as automatic evaluation metrics to assess model performance. These three methods are both efficient and objective, providing a comprehensive measure of the model's performance on generation tasks. ROUGE primarily measures the overlap between generated and reference text, making it particularly suitable for assessing summarization accuracy and other text generation tasks. BERTScore leverages a deep language model to embed both the generated text and the reference text, then calculates the cosine similarity between these embeddings to capture semantic information. FactCC evaluates factual consistency by using a supervised binary classifier to determine whether the factual statements in the generated text align with authoritative data, effectively identifying hallucinations and factual errors. In summary, these indicators effectively capture text quality and semantic accuracy while reducing the subjectivity and time costs associated with manual evaluation, thereby providing a more reliable basis for the performance evaluation of large-scale language models. ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is primarily used to measure the similarity between generated and reference texts based on \(N\)-gram overlap. An \(N\)-gram is a contiguous sequence of \(N\) items--such as characters, syllables, or words--in a text, where \(N\) is a positive integer denoting the sequence length. ROUGE is widely applied in automatic summarization and machine translation tasks. Common ROUGE variants include _ROUGE-\(N\)_ and _ROUGE-L_. _ROUGE-\(N\)_ quantifies the \(N\)-gram overlap between generated and reference texts. The equation is:  
\[ROUGE-N=\frac{\sum\limits_{S\in\{ReferenceSummaries\}}\sum\limits_{gram_{N} \in S}Count_{match}(gram_{N})}{\sum\limits_{S\in\{ReferenceSummaries\}} \sum\limits_{gram_{N}\in S}Count(gram_{N})} \tag{2}\]  
The numerator of the equation calculates the minimum number of matches of all \(N\)-grams between the generated text and the reference text, while the denominator is the total number of \(N\)-grams in the reference text. _ROUGE-L_ is based on the longest common subsequence (LCS) to capture the structural similarity between the generated text and the reference text. The equation is expressed as follows:  
\[ROUGE-L=\frac{LCS(gen,ref)}{length(ref)} \tag{3}\]  
where \(LCS(gen,ref)\) is the length of the longest common subsequence (LCS) between the generated text and the reference text, and \(Length(ref)\) is the length of the reference text; ROUGE-L is mainly used to calculate the similarity between the generated text and the reference text in terms of sentence structure, which is applicable to the scenario of structured information. BERTScore is a metric designed to measure text similarity. It evaluates the similarity between two sentences by calculating the cosine similarity between the embedding vectors produced by a pre-trained BERT model.