# A Sentence-Matching Model Based on Multi-Granularity Contextual Key Semantic Interaction
## 3 Multi-Granularity Contextual Key Semantic Interaction Model
### Multi-Granularity Feature Extraction Layer  
In the field of natural language processing (NLP), convolutional neural networks (CNNs) have proven to be effective in extracting local features from sentences. They operate by applying convolutional operations to input sentences, which allows them to capture n-gram features and generate latent semantic representations that are rich in information for various downstream tasks. However, when dealing with Chinese text, words and phrases can vary in length and carry different meanings based on the number of characters they consist of. Using a fixed-size convolutional kernel might not be sufficient to capture the diverse range of phrase features, potentially resulting in the omission of crucial semantic information. To address the limitations of a single fixed-size convolutional kernel in capturing diverse phrase features in Chinese text, this study proposes the use of multiple-scale convolutions to capture semantic features at different granularity levels, including individual words, phrases, and entire sentences. This allows for a comprehensive representation of sequence samples. The multi-scale CNN performs convolutions on the input sentence using a series of differently sized kernels, each targeting specific n-gram semantic features. By employing multiple CNNs, the model can capture distinct features contained in different combinations of n-grams, overcoming the limitations of a single CNN. The network architecture is depicted in Figure 2.  
Figure 2: Structure of multi-scale CNN. Moreover, combinations of different phrases and expressions carry higher-level semantic information. By stacking different-scale CNNs, the multi-level CNN can capture advanced semantic features from low-level character sequences. This allows for the extraction of deeper semantic information. The network structure is illustrated in Figure 3.  
The multi-scale CNN and multi-level CNN have the ability to capture different aspects of sentences. The multi-scale CNN captures a broader range of semantic information, while the multi-level CNN captures deeper semantic information. These two approaches complement each other. Similar to other CNN models used in NLP tasks, this study employs fixed-width convolutions to extract local features. The width of the convolutional kernel matches the hidden dimension of the input sequence. For a given sequence of sentences, let us assume \(\epsilon_{i}\in\mathbb{R}^{d}\) represents the contextual representation of the \(i\)-th word in the sentence, where \(d\) denotes the dimension of the representation. The input sentence consists of \(n\) words. To extract meaningful local features, we employ convolutional filters of different scales, denoted as \(S=\{s_{1},s_{2},\ldots,s_{t}\}\), with a fixed kernel width of \(l\). For example, generating feature \(m_{ij}^{S_{i}}\) through window \(\epsilon_{ii+S_{i}-1}\):  
\[m_{ij}^{S_{i}}=GELU\Big{(}\mathbf{W}_{j}^{S_{i}}\big{(}\epsilon_{ii+S_{i}-1}\big{)} +b_{j}^{S_{i}}\Big{)} \tag{6}\]  
where \(\mathbf{W}_{j}^{S_{i}}\in\mathbb{R}^{S_{i}\times d}\) represents the weight of the \(j\)-th feature map at scale \(s_{i}\), \(GELU\) denotes the activation function, and \(b_{j}^{S_{i}}\) represents the bias of the feature map. The convolutional kernel, denoted as \(s_{i}\), slides over the entire padded contextual representation matrix, denoted as \(\mathbf{C}\), to generate feature maps. This process is illustrated below:  
\[m_{j}^{S_{i}}=\left[m_{1j}^{S_{i}},m_{2j}^{S_{i}},\ldots,m_{nj}^{S_{i}}\right] \tag{7}\]  
The local n-gram features of the sentences have been extracted using the multi-scale CNN. By concatenating the outputs of the convolutional kernel \(s_{i}\), we obtain the input for the key semantic alignment layer, which is represented as follows:  
\[\mathbf{M}^{S_{i}}=\left[m_{1}^{S_{i}},m_{2}^{S_{i}},\ldots,m_{n}^{S_{i}}\right] \tag{8}\]  
where \(h\) represents the number of feature maps, and \(\mathbf{M}^{S_{i}}\in\mathbb{R}^{n\times h}\).