## 5 Discussion  
Accurate and automatic DRG improves clinical efficiency, and fine-tuning a pre-trained LLM is indispensable for realizing this specific application task. However, anisotropy degeneration or semantic inconsistency of sentence embeddings has been massively observed  
Figure 4: The effect of control threshold \(\rho\) on the text generation diversity (\(\rho=0.01\), dotted red line with \(\Diamond;\rho=0.10\), dashed blue line with \(\circ\)). \begin{table}
\begin{tabular}{c c c} \hline \hline
**Team** & **Main Procedure in Diagnosis Report Generation** & **Score** \\ \hline A & CPT-base + noise-aware similarity bucketing + fine-tuning & 2.327 \\ B & BART-large + GBPQ + fine-tuning & 2.297 \\ C & (CPT-base + BART-base) + RAG + fine-tuning & 2.285 \\ D & BART-large + fine-tuning & 2.272 \\ E & BART-large + fine-tuning & 2.263 \\ F & BART-large + fine-tuning & 2.249 \\ \hline ours & GPT2-Chinese + fine-tuning + CLpCEwDCS decoding & 2.135 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Current achievement of the first-tier teams on the competition. from unnatural and undesirable text generation. To address this issue, a CLpCEwDCS decoding framework is proposed and evaluated on this challenging task. In any case, a supplementary metric (maxTRR) is designed to evaluate the token diversity in text summarization, which is also important in DCS decoding. The CLpCE improves the consistency and accuracy of the sentence embeddings. In comparison to the CE objective function, the proposed CLpCE function leads to higher DRG quality regardless of the decoding methods. It increases the vales of evaluation metrics (Figure 3) and obtains superior performance when \(\beta=0.60\) (Table 1). Specifically, when using DCS for decoding, CLpCE (\(\beta=0.60\)) enhances 0.03 on BLEU-1 and BLEU-2, 0.02 on BLEU-3 and BLEU-4, 0.01 on METEOR and ROUGE, and 0.09 on CIDER over the CE objective function (\(\beta=0.00\)). Notably, this phenomenon can also be observed when using other decoding methods. It indicates that CLpCE can quantitatively improve the DRG quality from fragment accuracy, word matching, semantic similarity and content consistency. The main reason is the penalty term. CL is a self-supervised representation learning method by contrasting semantically similar and dissimilar pairs of samples [25]. Its purpose is to minimize the distance of samples from same distributions and to maximize the distance of samples from different distributions. Consequently, in the sentence embedding space, intra-class tokens could be close, and inter-class tokens could be kept a long distance. Thereby, the CL penalty term benefits LLM fine-tuning and guides the procedure towards a specific application task, and in this study, it improves DRG quality. The DCS decoding method relieves the anisotropy degeneration issue by decreasing the frequency of token repetition. It achieves competitive DRG quality with the CS and the GS decoding methods (Table 1). Most importantly, it leads to more candidate outputs of text summarization in the token space (Figure 4) and decreases token repetition ratio (Table 2) by using the minimum of the maxTRR values, while the generation cohesion and accuracy are maintained well (Table 4). Of particular concern is the proposed metric maxTRR (Equation (12)). Its value is applied to determine the final token sequence output (Equation (11)) in an automated fashion. Additionally, two case examples further reveal that DCS decoding provides more candidate outputs of text generation with lower repeats and frequent tokens (Table 3). It should be admitted that there is discrepancy between the human and model word distributions, and further training on more data could not rectify this discrepancy [26; 53]. Interestingly, the DCS decoding shows the potential to decrease the discrepancy by improving the output diversity. It keeps the accuracy and coherence as the CS decoding method and outperforms other traditional methods [19].