# Reliable Semantic Communication System Enabled by Knowledge Graph
## 3 System Model
### Semantic Restoration Method  
The proposed semantic restoration method--similar to the proposed semantic extraction method--uses deep learning to generate sentences from the received knowledge graph. The generated sentence can help the receiver understand the semantics of the transmitted sentence. Let \(G2S_{\varphi}(\bullet)\) be the function of the proposed semantic restoration. The input of \(G2S_{\varphi}(\bullet)\) is the received knowledge graph \(\hat{G}\) and its output is the restored sentence \(\hat{S}\). The deep learning network structure for the semantic restoration method is shown in Figure 4.  
At first, we encoded the received knowledge graph \(\hat{G}\) to convert it to the embedding, which could be processed by the deep learning network. Specifically, we used the graph attention network (GAT) [42] to calculate the embedding of the received knowledge graph \(\hat{G}\). GAT is a representative graph convolutional network that can encode the knowledge graph by introducing the attention mechanism into the knowledge graph. Therefore, the embedding of \(\hat{G}\) can be represented as  
\[h=\text{GAT}(\hat{G}) \tag{11}\]  
Figure 4: The deep learning network structure for the semantic restoration method. After obtaining the embedding \(h\), we used the recurrent neural network (RNN) and the attention mechanism to generate the sentence word by word. Each step of RNN can produce a word embedding. In the \(i\)th step, the embedding \(b_{i}\) can be represented as  
\[b_{i}=\text{RNN}(b_{i-1},w_{i-1}) \tag{12}\]  
where \(w_{i-1}\) is the \(i-1\)th word in the generated sentence, \(b_{i-1}\) is the embedding produced in the \(i-1\)th step. To improve the accuracy of the generated sentence, the attention mechanism was used to obtain the embedding of contextual information. The formula can be described as  
\[c_{i}=\text{ATTENTION}(b_{i},h) \tag{13}\]  
where \(c_{i}\) denotes the contextual information of the \(i\)th word. Then we fed the word embedding \(b_{i}\) and the contextual information \(c_{i}\) into a multilayer perceptron (MLP) to generate the \(i\)th word \(w_{i}\). Consequently, the generation of \(w_{i}\) based on the received knowledge graph \(\hat{G}\) and all previously generated words \(w_{<i}\) was fulfilled by predicting the word \(w_{i}\) through MLP with the assistance of the word embedding \(b_{i}\) and the contextual information \(c_{i}\). Thus, the probability of recovering word \(w_{i}\) can be represented as  
\[p(w_{i}|w_{<i},\hat{G})\propto\exp(\text{MLP}([b_{i};c_{i}])) \tag{14}\]  
In summary, the probability of generating a sentence from the received knowledge graph \(\hat{G}\) is equivalent to the product of the probability of generating each word. The probability can be described as  
\[p(\hat{S}|\hat{G})=\prod p\big{(}w_{i}\mid w_{<i},\hat{G}\big{)} \tag{15}\]  
Similarly, we used the negative log-likelihood loss to denote the loss function of the proposed semantic restoration method according to the probability \(p(\hat{S}|\hat{G})\). The loss function can be represented as  
\[ \begin{split}\mathcal{L}_{\text{G25}}(\varphi)&= \mathbb{E}[-\log p(\hat{S}\mid\hat{G};\varphi)]\\ &=\mathbb{E}\big{[}-\log\prod p\big{(}w_{i}\mid w_{<i},\hat{G}; \varphi\big{)}\big{]}\end{split}  \tag{16}\]  
where \(\varphi\) is the network parameter set of the deep learning network, which is shown in Figure 4. Finally, the gradient descent can be used to find the optimal parameter set \(\varphi^{*}\) for minimizing the loss function \(\mathcal{L}_{\text{G25}}(\varphi)\). The details of the proposed semantic restoration process are summarized in Algorithm 2.