# Reliable Semantic Communication System Enabled by Knowledge Graph
## 3 System Model
### Semantic Extraction Method  
To represent the semantic information correctly, the semantic extraction module at the transmitter uses a deep learning network to extract the knowledge graph from the transmitted sentence. Let \(S2G_{\textit{g}}(\bullet)\) be the function of the proposed semantic extraction method, which takes the sentence \(S=[w_{1},w_{2},\cdots,w_{m}]\) as input and its corresponding output is the knowledge graph \(G\), where \(w_{m}\) is the \(m\)th word in the sentence. The deep learning network structure for the semantic extraction method is shown in Figure 3.  
In particular, we used the pipeline method to extract the knowledge graph, which means extracting the entities in \(S\) and then predicting the relations between entities. Firstly, we used a well-established named entity recognition model (NER) to extract the entities [40]. This model is based on the conditional random field classifier and Gibbs sampling. The conditional random field classifier combines the characteristics of the maximum entropy model and the hidden Markov model, and it is often used to deal with sequence labeling tasks, such as parts-of-speech tagging and named entity recognition. Gibbs sampling is a method of generating Markov chains that can be used for Monte Carlo simulations. Based on the conditional random field classifier and Gibbs sampling, NER is trained by using a large amount of manually annotated text and can recognize entities from given sentences. Therefore, the entities in the transmitted sentence can be expressed as  
\[E=[en_{1},en_{2},\cdots,en_{i},\ldots,en_{L}]=\text{NER}(S) \tag{5}\]  
where \(en_{i}\) represents the \(i\)th entity in the sentence, \(L\) is the total number of entities contained in the sentence. After extracting entities from \(S\), we predict the relations between the two entities. Firstly, the embedding of each word \(w_{j}\) in the entity \(en_{i}\) is averaged to obtain the entity's embedding. The embedding of \(w_{j}\) can be obtained by using a long short-term memory model (LSTM) [41] to encode \(w_{j}\) and its context. The formula is given as follows  
\[\text{emb}\big{(}w_{j}\big{)}=\text{LSTM\_encode}\big{(}w_{j},w_{<j},w_{>j} \big{)} \tag{6}\]  
Therefore, the \(i\)th entity's embedding \(e_{i}\) can be represented as  
\[e_{i}=\frac{1}{\text{Len}(en_{i})}\sum_{w_{j}\in en_{i}}\text{emb}\big{(}w_{ j}\big{)} \tag{7}\]  
where \(\text{Len}(en_{i})\) is the number of words in the entity \(en_{i}\). Then we feed the entity embeddings into a multi-label classification layer MLCL(\(\bullet\)) to predict the relations. The multi-label classification layer MLCL(\(\bullet\)) can take in two entities and predict the possible relation set. To prevent these two entities from being irrelevant, the relation set includes the \"no-relation\" type. The relation set between the \(i\)th entity and the \(j\)th entity can be represented as  
\[r_{ij}=\text{MLCL}\big{(}e_{i},e_{j}\big{)} \tag{8}\]  
Since the knowledge graph is made of entities and relations, the probability of extracting a graph from a given sentence is equivalent to the product of the probability of extracting the relation set given any two entities. The formula can be expressed as  
Figure 3: The deep learning network structure for the semantic extraction method. \[p(G\mid S)=\prod_{i=0}^{L}\prod_{j=0}^{L}p\big{(}r_{ij}\mid e_{i},e_{j},S\big{)} \tag{9}\]  
Based on the probability \(p(G\mid S)\), we can denote the loss function of the proposed semantic extraction method by using the negative log-likelihood loss, which can be formulated as  
\[ \begin{split}\mathcal{L}_{\texttt{S2G}}(\theta)&= \mathbb{E}[-\log p(G\mid S;\theta)]\\ &=\mathbb{E}\Bigg{[}-\log\prod_{i=0}^{L}\prod_{j=0}^{L}p\big{(}r_ {ij}\mid e_{i},e_{j},S;\theta\big{)}\Bigg{]}\end{split}  \tag{10}\]  
where \(\theta\) is the network parameter set of the deep learning network, which is shown in Figure 3.  
Utilizing the loss function \(\mathcal{L}_{\texttt{S2G}}\), the optimal parameter set \(\theta^{*}\) can be easily found using the gradient descent method.