# Sentence Representation Method Based on Multi-Layer Semantic Network
## 3 Methods
### Design of Semantic Representation Network Based on Multi Attention
#### 3.2.1 Natural Language Sentence Vectorization  
The function of sentence vectorization is to obtain the initial embedded representation of the sentence and the semantic representation network's embedding layer. This part is composed of the Seq2 Word module, Word2 Char module, and embedded layer. The network structure is shown in Figure 3.  
Suppose input \(S=(W_{1},W_{i},\dots,W_{L})\), depending on the word \(W_{i}\) in sentence \(S\) input into the embedding layer of multi-layer semantic representation network, and calculate the initial embedding representation \(H\) of sentence \(S\), where \(W_{i}\) (\(i=1,\ 2,\dots,\ L\)) is the \(i\)th word of sentence \(S\), and \(L\) is the length of sentence. The calculation process of each module is as follows:  
1. Seq2 Word module  
Figure 3: Network structure diagram of the embedded layer.  
The module is mainly used to obtain the embedded representation of the word vector corresponding to each word in the sentence \(S\), the word-level word vectorization processing. The sentence is mapped from the natural language space to the vector space by splicing all the word vector representations of the sentence. The specific steps are as follows:  
1. Word2 vec technology [10] combined with GloVe-840B-300D [34] was used to train the word vector embedding representation of the dictionary database;
2. According to the word's position in the dictionary, the embedding vector w corresponding to each word in the sentence is obtained \(w_{i}\in R^{d_{w}\times 1}\).  
2. Word2 Char module  
The module uses convolution networks of different sizes to extract character features in words [35] and obtain word embedding representation at the character level. The specific steps are as follows:  
1. According to the given character dictionary (including 24 letters and common punctuation and special characters), the word \(W_{i}\) is split into a list of characters;
2. After a single-layer convolution network and pooling layer, the character vector \(c_{i}\in R^{m\times d_{c}}\) of the word \(W_{i}\) is obtained, where \(d_{c}\) is the dimension of each letter mapping, and \(m\) is the number of pooling layers. The convolution layer uses \(n\ast h\) convolution kernel (\(n\) is the dimension of character embedding vector, \(h\) is the size of convolution kernel window).  
3. Embedded layer  
Firstly, the word vector \(w_{i}\) obtained from the Seq2 Word module and the character vector \(c_{i}\) obtained from the Word2 Char module are combined to form the semantic vector \(e_{i}\in R^{d_{c}}\) of the word \(w_{i}\). Finally, the word vector sequence \(x=\{e_{1},\cdots,e_{t},\cdots,e_{L}\}\) of sentence \(S\), where \(x\in R^{L\times d_{c}}\), \(d_{c}\) is the dimension of the semantic vector \(e_{i}\), _and_\(d_{c}\) is the sum of \(d_{w}\) and \(d_{c}\).  
The \(N\)-layer BiLSTM network is used to obtain further information of the word vector sequence \(x\) of the sentence to obtain the context information in the sentence. For each layer of the network, the hidden layer state of the forward network and the backward network \(\overset{\rightarrow}{h}_{t}\) and \(\overset{\leftarrow}{h}_{t}\), are as follows:  
\[\overset{\rightarrow}{h}_{t}=LSTM(e_{t},\overset{\rightarrow}{h}_{t-1}), \overset{\leftarrow}{h}_{t}=LSTM(e_{t},\overset{\leftarrow}{h}_{t-1}) \tag{7}\]  
splicing \(\overset{\rightarrow}{h}_{t}\) and \(\overset{\leftarrow}{h}_{t}\) to obtain the hidden layer state \(h_{t}=\left[\overset{\rightarrow}{h}_{t},\overset{\leftarrow}{h}_{t}\right] \in R^{1\times 2u}\), where \(u\) is the number of neurons in the forward (backward) network of each layer. After \(N\)-layer cycle, the hidden layer state of the top-level network is output as the embedded representation \(H\in R^{N\times 2u}\) of sentence \(S\).