# Joint Geoeffectiveness and Arrival Time Prediction of CMEs by a Unified Deep Learning Framework
## 2 Methods
### Geoeffectiveness Prediction of CMEs  
#### 2.1.1 Deep Residual Network Embedded with the Attention Mechanism  
The proposed deep residual network embedded with the attention mechanism is used to extract the feature map from each observation image. Our network is designed based on the traditional convolutional neural networks and modified to make it more suitable for CME observation images. The main features are as follows--compared with ImageNet, the features of the CME image are more sparse, so we select larger convolution filters; using the residual learning to train a deeper network; introducing the attention mechanism to extract key area features. The input image will first pass through data preprocessing. It consists of scaling, normalization and gray scale conversion. After that, the input image will be a gray scale image and pixel size is \(256\times 256\), and the weight of each pixel is between [0, 1]. Next, our network will extract the feature map from the input image. The architecture of our network is shown in Table 1. The first layer is a \(11\times 11\) convolution with 64 filters and the stride is 2. Compared with the ImageNet data set, the features of CME image are relatively sparse, using convolution filters with big kernel size can better fuse the features. The second layer is the max pooling layer, which can downsample the feature dimensions while retaining the main features. Next layers are composed of six residual blocks with embedded attention. Each residual block introduces residual learning to make deeper neural networks easier to train, and attention mechanism is introduced to extract key regional features. The residual block with embedded attention has two forms. The first form is shown in Figure 2, and it is used when the input and output are of the same dimensions. The formula is as follows:  
\[y=F(x,\{W_{i}\})+x, \tag{1}\] \begin{table}
\begin{tabular}{c c c} \hline \hline
**Layer Name** & **Layer** & **Output Size** \\ \hline Conv1 & \(11\times 11\), 64, stride 1 & \(256\times 256\) \\ \hline Max Pool & \(3\times 3\), max pool, stride 2 & \(128\times 128\) \\ \hline Conv2\_x & \(\left[\begin{array}{c}5\times 5,\,64\\ 5\times 5,\,64\\ Channel\ Attention\\ Spatial\ Attention\end{array}\right]\times 2\) & \(128\times 128\) \\ \hline Conv3\_x & \(\left[\begin{array}{c}5\times 5,\,128\\ 5\times 5,\,128\\ Channel\ Attention\\ Spatial\ Attention\end{array}\right]\times 2\) & \(64\times 64\) \\ \hline Conv4\_x & \(\left[\begin{array}{c}5\times 5,\,256\\ 5\times 5,\,256\\ Channel\ Attention\\ Spatial\ Attention\end{array}\right]\times 2\) & \(32\times 32\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The architecture of the deep residual network embedded with the attention mechanism. where \(x\) and \(y\) are the input and output vectors of the residual blocks, \(F(x,\{W_{i}\})\) represents the residual mapping to be learned. In our network, the formula of \(F(x,\{W_{i}\})\) is as follows:  
\[F(x,\{W_{i}\})=SpatialAtt(ChannelAtt(W_{2}\sigma(W_{1}x))), \tag{2}\]  
where \(W_{1}\) and \(W_{2}\) denote the convolution operation with the filter size of \(5\times 5\), and the convolution operation is followed by batch normalization [37] by default. Batch normalization allows each layer of the network to learn the wanted distribution, reduces the dependence on parameter initialization and can effectively alleviate the phenomenon of gradient vanishing/gradient exploding. \(\sigma\) denotes the activation function RELU [38], \(ChannelAtt\) and \(SpatialAtt\) represent the channel attention mechanism and the spatial attention mechanism, respectively. Here, the channel attention mechanism is performed first, and then the spatial attention mechanism. The second form is used to match dimensions when the input and output are different dimensions, it can be expressed as follows:  
\[y=F(x,\{W_{i}\})+W_{s}x, \tag{3}\]  
where \(W_{s}\) represents a convolution operation with the filter size of \(1\times 1\).