# Shear Wave Velocity Prediction Based on the Long Short-Term Memory Network with Attention Mechanism
## 2 Method Theory
### Long Short-Term Memory Network (LSTM)  
The LSTM network is a recurrent neural network specially designed for processing time series data proposed by [PERSON] and [PERSON] [44]. Recurrent neural networks can retain the relevant information of the previous sequence and pass it to the following sequence in turn. Therefore, the LSTM network can synthesize the context information of the original input data to infer the results. Based on the sequence characteristics of log data, the LSTM network is suitable for shear wave velocity prediction. The LSTM sets three threshold structures (input gate, forget gate, and output gate) to obtain contextual long-term dependency information for input data. The input gate selects the valuable information in the current input, the forget gate determines what information should be removed from the memory of the previous moment; and the output gate controls what valid information needs to be output at the current point in time. The LSTM network structure is shown in Figure 2.  
The unit structure of LSTM includes the input gate (\(i_{t}\)), forgetting gate (\(f_{t}\)), status update layer (\(c_{t}\)), output gate (\(o_{t}\)), hidden state at time \(t-1\) (\(h_{t-1}\)), hidden state \(h_{t}\) at t time, and input \(x_{t}\) at \(t\) time. They can be expressed as follows:  
\[\mathrm{i_{t}}=\sigma(\mathrm{W_{i}}[\mathrm{x_{t}},\mathrm{h_{t-1}}]+\mathrm{ b_{i}}) \tag{2}\]  
\[\mathrm{f_{t}}=\sigma(\mathrm{W_{f}}[\mathrm{x_{t}},\mathrm{h_{t-1}}]+\mathrm{ b_{f}}) \tag{3}\]  
\[\mathrm{C_{t}}=\mathrm{f_{t}}*\mathrm{C_{t-1}}+\mathrm{i_{t}}*\mathrm{tanh}( \mathrm{W_{c}}[\mathrm{x_{t}},\mathrm{h_{t-1}}]+\mathrm{b_{c}}) \tag{4}\]  
Figure 2: LSTM network internal structure. \[\mathrm{o_{t}}=\sigma(\mathrm{W_{o}}[\mathrm{x_{t}},\mathrm{h_{t-1}}]+\mathrm{b_{o}}) \tag{5}\]  
\[\mathrm{h_{t}}=\mathrm{o_{t}}*\mathrm{tanh}(\mathrm{C_{t}}) \tag{6}\]  
where \(W_{i}\), \(W_{f}\), \(W_{c}\), \(W_{o}\) and \(b_{i}\), \(b_{f}\), \(b_{c}\), \(b_{o}\) are the weight matrix and bias values of the input gate, forgetting gate, status update gate, and output gate, respectively, which are obtained through network training; \"\(\sigma\)\" is the sigmoid activation function, \(h_{t}\) is the hidden layer state at time t, and \"\(*\)\" represents the dot product (two vectors of the same dimension multiply the corresponding positions to form a new vector of the same dimension). \"[A, B]\" represents how vector A is concatenated to the head of vector B. The computing of the input gate is responsible for controlling whether to update the contents of the memory unit; the forget gate \(f_{t}\) is used to determine what information should be discarded from the memory unit of the previous moment. The status update layer \(c_{t}\) is the core of the LSTM network, which is used to store and transmit long-term information. The output gate \(o_{t}\) determines which data need to be output in the hidden state \(h_{t}\) of the current time step through the activation function tanh. Suppose \(x_{i}=[v_{1}\), \(v_{2}\), \(\ldots\), \(v_{m}]\) (\(i=1\), 2, 3, \(\ldots\) \(n\); \(m\) represents the number of selected logging curves) is the set of log vectors at the same time point, then the matrix formed by adjacent sampling points \([x_{1}\), \(x_{2}\), \(x_{3}\), \(\ldots\), \(x_{n}]\) can be used as the input matrix of the long and short-term memory (LSTM) network. The LSTM network consists of n LSTM cells with adjacent time steps, each of which, in turn, passes information to the next cell core. The data inputted can be defined as follows:  
\[Input\ \mathrm{data}= \begin{pmatrix}v_{1}^{x_{1}}&v_{2}^{x_{1}}&\cdots&v_{m}^{x_ {1}}\\ v_{1}^{x_{2}}&v_{2}^{x_{2}}&\cdots&v_{m}^{x_{2}}\\ v_{1}^{x_{3}}&v_{2}^{x_{3}}&\cdots&v_{m}^{x_{3}}\\ \vdots&\vdots&\vdots&\vdots\\ v_{1}^{x_{n}}&v_{2}^{x_{n}}&\cdots&v_{m}^{x_{n}}\\ \end{pmatrix}  \tag{7}\]  
In Figure 3, with a time step of 5 as an example, the hidden state \(\mathrm{h_{5}}\) contains the information the first four cell cores should retain.