# Large-Language-Model-Enabled Text Semantic Communication Systems
## 3 System Model and Mechanism
### System Model
By designing a high-dimensional constellation diagram for \(W\) and modulating it, \(W\) can becomes \(S\). Itshould be noted that because the vocabulary size is generally large, it is difficult to represent it with one modulation symbol. Therefore, multiple modulation symbols are needed to represent a token, which we call a high-dimensional constellation diagram. Since \(W\) and \(S\) are in one-to-one correspondence, they share identical probability distributions. To train a semantic system for optimal decoding, the loss function is formulated as follows:  
\[\mathcal{L}_{f}=-\sum_{i=1}^{t}\log P(w_{i}\mid w_{i-1},w_{i-2},\ldots,w_{max (i-N,1)})-\sum_{i=1}^{t}\log P(o_{i}\mid w_{i}), \tag{16}\]  
The two components of Equation (16) are evidently independent, enabling separate training of the system parts. The second component can be expressed as follows:  
\[-\sum_{i=1}^{t}\log P(o_{i}\mid w_{i})=-\sum_{i=1}^{t}\log P(o_{i}\mid s_{i}), \tag{17}\]  
Equation (17) represents the channel condition transition probability. Once the channel and modulation scheme are determined, this equation can be specified as Equation (10). For modulation schemes akin to those in classical communication (e.g., QAM or QPSK), Equation (17) can be directly applied without additional training. Therefore, language sequences can be semantically encoded through the tokenization and training of LLMs. Probabilistic modeling is subsequently performed on these encoded sequences. Achieving a zero loss in LLM training suggests optimal decoding within this transmission system, where loss indicates deviation from optimal decoding. Notably, through the training of tokenization and LLMs, an optimal transmission system can be attained across various coding rates, obviating the need for further LLM fine-tuning. This training process aligns seamlessly with the typical training methodologies of LLMs, requiring no adjustments for semantic communication. The system structure depicted in Figure 1 illustrates the pre-training process. We constructed the dataset by collecting all possible transmitted contents from both the sender and receiver. A tokenizer with a vocabulary size of \(V\) was trained using algorithms such as BPE and WordPiece. This tokenizer converts the text into tokens, which are later used to train an LLM. The loss function for the LLM is defined in Equation (15). Upon completion of the LLM training, the model serves as a knowledge base for semantic communication by providing \(P(W)\) to supply shared knowledge to the receiver. At the receiving end, the LLM plays a crucial role in the decoder by offering priors for the transmitted sequence. The method for achieving optimal decoding will be discussed in subsequent sections. The LLM-SC has the following characteristics:  
* The system assumes that \(D\) comprises all possible content that can be transmitted between the transmitter and receiver. While achieving this in real life poses challenges, current training corpora for LLMs strive to encompass all languages used by humans. Furthermore, this assumption is becoming increasingly feasible as LLM technology evolves rapidly. * The system prioritizes decoding sentences with higher probabilities of occurrence in the real world. Sequences for which \(P(s_{1},s_{2},\ldots,s_{T})=0\) cannot be transmitted, as the receiver would be unable to decode such sequences where the probability is zero. Consequently, the system cannot transmit sentences that are impossible in the real world, as these sentences convey no meaningful information. Therefore, the system integrates both technical and semantic information.