# A Novel Interpretable Deep Learning Model for Ozone Prediction
## 2 Materials and Methods  
### Methods  
The attention mechanism is commonly used in various domains and is effective in solving sequence-prediction problems. To learn temporality, we incorporated the attention mechanism to capture the temporal information from the previous sequence of the target monitoring station. Specifically, the past sequence is sent to the temporal attention layer and the corresponding attention weight vector quantifying the relation between each past moment and current prediction is obtained. After computing the dot product of the past sequence and the attention weight vector, temporal information, namely the temporal context vector, is derived. Intrinsically, by using temporal attention, we established a link between the present prediction and every previous moment. With this connection, the knowledge from previous sequences can assist in forecasting. This inspired us to employ the idea of using the attention mechanism to construct bridges that facilitate the flow of information between multiple stations in spatial learning.  
When predicting ozone for a certain station, we believe that its surrounding stations can provide assistance to improve prediction accuracy, because ozone pollution is usually a regional air quality issue. Thus, we learned spatial information from some specific moments in the past sequence of each surrounding station. Here, we propose to employ particular moments, not the whole sequence as it can reduce computation and alleviate disturbances from unimportant moments. We utilized the temporal attention layer to select the past moment with the highest attention weight for each station and introduced another attention layer, the spatial attention layer, to calculate the spatial context vector. The reason we used the temporal attention layer to make selections is that this layer contains knowledge about how to evaluate the importance of a past moment compared to the current prediction concerning the target station. Thus, the attention layer with learnable parameters is recommended. After determining these specific moments, the spatial attention layer calculates the attention score using decoding hidden states and the geographical location of that neighboring monitoring station for each selected moment to obtain the spatial context vector. In this manner, the spatial attention layer builds a connection between the target station and its neighbors and the knowledge of the past sequences of each station eventually affects the prediction.  
Along with learning the temporal and spatial information, the spatiotemporal attention-based STAGRU (Spatiotemporal Attentive Gated Recurrent Unit) model was proposed. Details of the STAGRU are shown in Figure 1. As shown in Figure 1, there are two modeling parts in the STAGRU. Historical air quality data and meteorological data of the target station are fed into the Encoder and encoded into the hidden states. A temporal context vector is derived from those hidden states by the temporal attention layer when decoding. For spatial modeling, the decoding hidden states with the highest attention weight are selected to learn the spatial knowledge and obtain the spatial context vector. After this, the temporal and spatial context vectors are concatenated with the current decoding hidden state and sent to the DNN to make a prediction.