# Large-Language-Model-Enabled Text Semantic Communication Systems
## 4 Numerical Results
### The Performance of BLEU and Sentence Similarity  
To evaluate the performance of LLM-SC in comparison with existing popular methods in semantic communication systems, we utilized BLEU and sentence similarity as our evaluation metrics. Unlike the modulation method previously discussed, the methods presented in [5] employ various modulation schemes. To ensure a fair comparison with the work in [5], we used bits per transmitted symbol (bps) as the metric for coding efficiency, which quantifies the number of information bits conveyed by each modulation symbol. According to [5], their coding efficiency is approximately 1.07 bps. Hence, for LLM-SC to achieve comparable coding efficiency, we employed a 15 bits per token encoding strategy coupled with an 8-QAM modulation scheme, wherein each token is represented by five modulation symbols in a high-dimensional constellation. This approach ensures that the coding efficiency of LLM-SC aligns with that of the DeepSC method proposed in [5], thereby maintaining equivalence in the number of transmitted characters for a given number of symbols. Figure 2 illustrates the relationship between the BLEU score and the SNR for identical bps over AWGN and Rayleigh fading channels. The comparison includes traditional techniques such as Huffman coding with RS (30,42) in 8-QAM, 5-bit coding with RS (42,54) in 64-QAM, Huffman coding with Turbo coding in 64-QAM, 5-bit coding with Turbo coding in 128-QAM, [PERSON] coding with Turbo coding in 8-QAM, and the DNN-based JSCC trained over AWGN channels and Rayleigh fading channels, as reported in [5], alongside the DeepSC approach, which employs trained modulation maps. It can be observed in Figure 2a that AI-based methods generally outperform traditional source-channel separation coding methods. Specifically, at SNR levels below 3 dB, the DeepSC model demonstrates the best performance in terms of BLEU score, indicating its superior adaptability to channel noise under low-SNR conditions. Conversely, for SNR levels above 3 dB, the LLM-SC model exhibits superior BLEU performance across all n-grams, significantly surpassing traditional coding schemes. This suggests that, under high-SNR conditions, the LLM-SC model is more effective in recovering semantic information. Notably, the DeepSC model fails to achieve completely error-free transmission even at very-high-SNR levels, implying that DeepSC cannot ensure the reliable transmission of all semantic information. In contrast, the LLM-SC model achieves a BLEU score of 1 at high-SNR levels, with near-zero errors at SNRs above 10 dB. Additionally, the BLEU performance of LLM-SC improves rapidly with increasing SNRs. When comparing BLEU scores across multiple n-grams, it is evident that the degradation of the BLEU score with increasing n in n-grams is slowest for the LLM-SC method. This is reflected in the diminishing advantage of DeepSC over LLM-SC with increasing n at low-SNR levels, whereas at high-SNR levels, the advantage of LLM-SC over DeepSC strengthens with larger n-grams. This indicates that the LLM-SC method is more favorable for semantic coherence, prioritizing the decoding of semantically continuous tokens, which aligns with the characteristics of LLMs. Figure 2b shows the performance of LLM-SC under Rayleigh fading channels. Traditional encoding and decoding schemes experience significant performance degradation in Rayleigh channels, with the highest BLEU score not exceeding 0.6 at an SNR of 18 dB. In contrast, DeepSC shows better adaptability to Rayleigh fading channels, indicating that neural network-based semantic encoding and decoding systems can effectively learn channel characteristics. However, DeepSC still fails to achieve error-free transmission, even when BLEU scores are high. The LLM-SC model demonstrates performance under Rayleigh fading channels similar to that in AWGN channels, with no noticeable degradation, consistent with the conclusion of Equation (10). Furthermore, LLM-SC achieves error-free transmission at high-SNR levels and outperforms all compared models.