## 2 Methods
### Feature Extraction Network Construction
#### 2.1.2 Feature Extraction Enhanced Network Module with Self-Attention  
The Self-Attention mechanism allows the model to focus on different parts of the sequence when processing the sequence data instead of fixing the same weights for all parts, which is particularly helpful for dealing with long-distance dependencies. The self-attention mechanism can be used in the Transformer model, which is a deep-learning model that uses the self-attention mechanism to capture global information in the input sequence. [PERSON] et al. [28] were the first to propose the transformer module for application in computer vision and showed through experimental results that the transformer module outperforms the convolutional neural network in the task of image restoration. Therefore, in this paper, we use the self-attentive transformer module to improve the DRE-Net network and apply it in the multi-view stereo matching 3D reconstruction feature extraction session to improve the image features through the self-attention mechanism, with the intention of enriching the semantics and improving the completeness of the reconstructed point cloud.  
The structure of the self-attention module transformer used in this paper is shown in Figure 3. The structure first converts the image into an input in the form of image blocks and slides the convolution kernel in order to extract the image blocks, with the total number of image blocks \(N=\left((H-l)/s+1\right)\times\left((W-l)/s+1\right)\), where the convolution kernel is of length and width \(l\) and the step size is \(s\). Then, flattening is performed to get the input as \(x^{\prime}\in R^{N\times D}\), and the self-attention mechanism is applied to these blocks of images to be processed later. First, the image blocks are processed through the fully connected network layer to obtain \(\left(key\right)x_{k}\), \(\left(value\right)x_{p}\), \(\left(query\right)x_{q}\) in the self-attention mechanism, and the key and query are multiplied pointwise to obtain the weight matrix of the image block \(x_{w}=x_{q}\cdot{x_{k}}^{T}\), where \(D^{\prime}\) is the feature dimension of the \(D\)-transform.  
\[x_{k} =x^{\prime}W_{k},W_{k}\in R^{D\times D^{\prime}},x_{k}\in R^{N \times D^{\prime}} \tag{1}\] \[x_{q} =x^{\prime}W_{q},W_{q}\in R^{D\times D^{\prime}},x_{q}\in R^{N \times D^{\prime}}\] (2) \[x_{v} =x^{\prime}W_{v},W_{v}\in R^{D\times D^{\prime}},x_{v}\in R^{N \times D^{\prime}} \tag{3}\]  
The weight matrix is normalized by the Softmax function and multiplied with (query) to get the result \(x_{att}=x_{w}\cdot x_{v}\) of the image block after the action of the self-attention mechanism. All the image blocks are processed by the transformer module and the feature dimensions are changed to \(D^{\prime}\), and finally the feature matrix \(x_{att}\in R^{N\times D^{\prime}}\) is transformed into the feature image resolution pattern to get \(F\in R^{H\times W\times D^{\prime}}\), where \(H\), \(W\), and \(D^{\prime}\) are the length, width, and feature dimensions of the output.  
Figure 3: Structure of the self-attention mechanism module.  
Through the above process, the codec structure is constructed to learn the features in the image in the feature extraction session, and the important features in the image block are enhanced by considering the relationship between each pixel point in the image and the surrounding pixel points through the self-attentive transformer module [29].