# A Sentence-Matching Model Based on Multi-Granularity Contextual Key Semantic Interaction
## 3 Multi-Granularity Contextual Key Semantic Interaction Model
### Key Semantic Alignment Layer
#### 3.3.1 Multi-Head Self-Attention  
The transformer model utilizes the scaled dot-product attention (SDA) mechanism for self-attention. As transformer-based pre-trained language models have gained popularity, the use of the scaled dot-product function has become more prevalent. The process begins by calculating the dot product between \(\mathbf{Q}\) and \(\mathbf{K}\), which serves as a measure of similarity between them. A higher value indicates a stronger correlation. This result is then divided by the scaling factor \(\sqrt{d}\), where \(d\) represents the dimension of the vectors. The introduction of the scaling factor addresses the issue of gradient vanishing or exploding that may occur when dealing with large vector dimensions during the dot product computation. This ensures smoother and more manageable attention scores, promoting training stability. Subsequently, the softmax function is applied to ensure that the weighted coefficients sum up to one, achieving attention normalization and distribution. Finally, the attention  
Figure 4: Structure of key semantic alignment layer.  
expression is obtained by multiplying it with the matrix v. The SDA function is defined as follows:  
\[SDA(\mathbf{Q},\mathbf{K},\mathbf{V})=Softmax\left(\frac{\mathbf{Q}\mathbf{K}^{\top}}{\sqrt{d}} \right)\mathbf{V} \tag{11}\]  
By introducing the multi-head mechanism, sequence features are mapped into different linear spaces, allowing for the extraction of relevant information from various representation spaces. This integration of information from different spaces enhances the expressive power of attention. The formula can be represented as follows:  
\[MultiHead(\mathbf{Q},\mathbf{K},\mathbf{V})=Concat(\mathbf{Head}_{1},\dots,\mathbf{Head}_{t}) \mathbf{W} \tag{12}\]  
\[\mathbf{Head}_{i}=SDA\left(\mathbf{Q}\mathbf{W}_{i}^{Q},\mathbf{K}\mathbf{W}_{i}^{K},\mathbf{V}\mathbf{W} _{i}^{V}\right) \tag{13}\]  
where \(\mathbf{W}\), \(\mathbf{W}_{i}^{Q}\), \(\mathbf{W}_{i}^{\mathbf{x}}\), and \(\mathbf{W}_{i}^{V}\) are learnable parameters, \(t\) denoting the number of attention heads.  
Using multi-head self-attention, the key semantic features \(\mathbf{S}_{P}^{S_{i}}\) for the sentence \(\mathbf{P}\) itself can be computed as follows:  
\[\mathbf{S}_{P}^{S_{i}}=Concat\left(\mathbf{S}_{P_{1}}^{S_{i}},\dots,\mathbf{S}_{P_{l}}^{S_ {i}}\right)\mathbf{W} \tag{14}\]  
\[\mathbf{S}_{P_{i}}^{S_{i}}=SDA\left(\mathbf{M}_{P}^{S_{i}}\mathbf{W}_{i}^{Q},\mathbf{M}_{P}^{S _{i}}\mathbf{W}_{i}^{K},\mathbf{M}_{P}^{S_{i}}\mathbf{W}_{i}^{V}\right)=Softmax\left(\frac{ M_{P}^{S_{i}}\mathbf{W}_{i}^{Q}\left(\mathbf{M}_{P}^{S_{i}}\mathbf{W}_{i}^{K}\right)^{\top}}{ \sqrt{h}}\right)\mathbf{M}_{P}^{S_{i}}\mathbf{W}_{i}^{V} \tag{15}\]  
In this equation, the dimension of \(\mathbf{S}_{P}^{S_{i}}\) matches the dimension of the input feature \(\mathbf{M}_{P}^{S_{i}}\), \(\mathbf{S}_{P}^{S_{i}}\in\mathbb{R}^{m\times h}\). Similarly, the crucial semantic features \(\mathbf{S}_{Q}^{S_{i}}\in\mathbb{R}^{n\times h}\) for the sentence \(\mathbf{Q}\) can be calculated.