# Shear Wave Velocity Prediction Based on the Long Short-Term Memory Network with Attention Mechanism
## 2 Method Theory
### Long Short-Term Network with Attention Mechanism (Attention-LSTM)  
The attention mechanism first came from the Transformer structure [45] proposed by [PERSON] in 2017. The core idea is to give different parts of the information different attention when processing it. Attention mechanisms are usually implemented by learning a set of weights that determine the importance of each part of the input data. In this way, the attention mechanism helps the model focus on which features are most important for the task.  
Attention-LSTM is a shear wave prediction method developed based on the long short-term network. This method mainly adds an attention computing module between the long short-term network and the input data to make up for the defect that the long and short-term network cannot focus on the input data. With the addition of a self-attention mechanism, the long short-term memory network can reconstruct the information in a fixed time step. This method can extract adequate information from the input data and remove the strong noise in the input data, meaning that the input data are more effective.  
Assuming there are n log curves related to shear wave velocity and the time step is m, the calculation flow of the network is as follows:  
1. Calculate the attention results of every logging curve (_r_ = 1,2, \(\ldots\), _n_).  
\[\begin{pmatrix}\mathrm{y_{1r}}\\ \mathrm{y_{2r}}\\ \vdots\\ \mathrm{y_{mrr}}\end{pmatrix}=\begin{pmatrix}A_{11}^{r}&A_{12}^{r}&\cdots&A_ {1m}^{r}\\ A_{21}^{r}&A_{22}^{r}&\cdots&A_{2m}^{r}\\ \vdots&\vdots&\ddots&\vdots\\ A_{m1}^{r}&A_{m2}^{r}&\cdots&A_{mm}^{r}\end{pmatrix}\begin{pmatrix}x_{1r}\\ x_{2r}\\ \vdots\\ x_{mrr}\end{pmatrix}\begin{pmatrix}A_{11}^{r}&A_{12}^{r}&\cdots&A_{1m}^{r}\\ A_{21}^{r}&A_{22}^{r}&\cdots&A_{2m}^{r}\\ \vdots&\vdots&\ddots&\vdots\\ A_{m1}^{r}&A_{m2}^{r}&\cdots&A_{mm}^{r}\end{pmatrix}\textit{is trainable matrix} \tag{8}\]  
2. Form a new input matrix.  
\[\begin{pmatrix}y_{11}\\ y_{21}\\ \vdots\\ y_{m1}\end{pmatrix}^{\prime}\begin{pmatrix}y_{12}\\ y_{22}\\ \vdots\\ y_{m2}\end{pmatrix}\cdots\begin{pmatrix}y_{1n}\\ y_{2n}\\ \vdots\\ y_{mn}\end{pmatrix}\Rightarrow\begin{pmatrix}y_{11}&y_{12}&\cdots&y_{1n}\\ y_{21}&y_{22}&\cdots&y_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ y_{m1}&y_{m2}&\cdots&y_{mm}\end{pmatrix}=\begin{pmatrix}y_{1}\\ y_{2}\\ \vdots\\ y_{m}\end{pmatrix} \tag{9}\]  
3. Calculate the output of Attention-LSTM.  
\[\textit{prediction value}=LSTM(y_{1},y_{2},\ldots,y_{m}) \tag{10}\]  
Figure 4 shows the organizational chart of the Attention-LSTM network. The first part of this network is the attention module, which performs self-attention weighting for each log through the attention matrix. The second part is the long short-term memory network, which accepts the result matrix from the first part of the attention calculation. Through the dynamic combination of these two parts, the first part increases the proportion of important information in the data. It suppresses the interference information, while the second part extracts the sequence information. Finally, the shear wave prediction value of the last time step is obtained. The flow of the entire process can be represented in Figure 5.