# Vector Magnetic Anomaly Detection via an Attention Mechanism Deep-Learning Model
## 2 Methods  
### Network Design and Architecture  
Magnetic signals are sequential time series consisting of different local (targets presence) and more global (e.g., instrument background noise and daily variation of magnetic field) features. Hence, we can consider MAD as a classification problem at each time point. Traditionally, recurrent neural networks have been used for such sequence modeling. Relatively long-duration magnetic signals require downsampling prior to feeding to the recurrent layers to decrease the computational complexity. Here, we introduce several modules in our model. The general architecture is based on U-Net, which has presented an excellent performance in the segmentation task. The U-Net network structure is symmetrical and resembles the letter U. The most important advantage of U-Net for our network design is that it can build a very deep encoder by downsampling. Traditionally, recurrent neural networks (RNN) have been used for such sequence modeling; however, downsampling prior to the recurrent layers is imperative to manage the computational complexity when input is relatively long duration signals. Moreover, U-Net's skip layers can help keep detailed information.  
Our deep-learning model for MAD consists of an encoder and decoder, including 1D convolutions, residual connections (Res CNN) [21], bi-directional long-short-term memories (BiLSTM), and a transformer (Figure 1).  
Since excessively long sequences in the self-attentive mechanism cause computation to increase exponentially, we carried out downsampling through max-pooling layers. We added three skip connection layers to extract convolutional features at different levels. These features were accepted through the concatenate layers in the decoder for learning the picking task hierarchically. In this way, the magnetic field signal was transformed from a sequence in the time domain into a high-dimensional feature representation.  
Next, we processed these expressions by two attention modules. The first global attention module guided the network attention to the part where the target appeared by assigning different weights. The second local attention module is the last part of the encoder. It mainly transmits the signal of the target appearance to the decoder. After passing through a very deep encoder, the signal reaches the bi-directional long short-term memory (BiLSTM) blocks. Long short-term memory (LSTM) is a specific type of RNN generally used in modeling longer sequences. The main element in the LSTM cell is a memory cell. At each time step, the LSTM cell receives an input, outputs a hidden state, and updates the memory cell according to the gate mechanism. Compared with the regular RNN, the LSTM could keep the long-period information, which is helpful for the attention mechanism. BiLSTM is used to assist the attention module in distinguishing long sequence signal features by incorporating time-positional information. As shown by the brown squares in Figure 1, the double arrows indicate the recurrence direction of the BiLSTM. Decoders then use this information to map the high-level features by a sequence of target existence probabilities.