# An Attention-Based Deep Convolution Network for Mining Airport Delay Propagation Causality
## 3 Methodology
### Delay Prediction Model Based on Deep Convolutional Network
#### 3.2.2 Attention-Based Time Series Prediction Architecture  
The goal of airport delay propagation causality discovery is to find directly correlated airports that help predict delays at the target airport. Whether it is the Granger causality test used in the literature [15; 31] to find the causality of delayed propagation, or the transfer entropy used in the literature [32], the prediction is essentially the evaluation. These methods are used to examine the variation of delay time series at one airport and whether it is helpful to predict future delays at another airport. However, these methods only mine causal relationships between pairs of airports, without considering the whole airport network, and the mining results may have indirect or confounding causal relationships. This section will comprehensively consider the airport network perspective, and introduce an attention mechanism into the deep time series convolutional network (DTCN). The delay time series of all airports are used as the input of DTCN, and the delay time series of a single airport is used as the output to the airport network. A DTCN prediction model is constructed for each airport, as shown in Figure 5. The processing flow from left to right is to input the delay time series \(\{X_{1},X_{2}\cdots,X_{N}\}\) of all airports, perform attention mechanism processing on them (obtain the sequence \(\{X_{1}^{\prime},X_{2}^{\prime},\cdots,X_{N}^{\prime}\}\)), and perform single-channel depth convolution on each time series (obtain the sequence \(\{X_{1}^{\prime\prime},X_{2}^{\prime\prime},\cdots,X_{N}^{\prime\prime}\}\)), and point convolution (output delay prediction vector \(\overline{X}_{j}\) of target airport \(j\)). In order to predict the delay in the \(t\)-th time interval of the airport, a prediction network is trained for each airport with the delay information of all airports in the \(\tau_{\max}\) periods before the \(t\)-period as input. Let \(X_{j}^{(t-1)}=\left[x_{j}^{t-\tau_{\max}},x_{j}^{t-\tau_{\max}+1},\cdots,x_{j} ^{t-1}\right]\) represent the delay time subsequence of the airport \(j\) in the time interval \([t-\tau_{\max},t-1]\), and the delay prediction model based on the deep time domain convolutional network is expressed as follows:  
\[\left[ \begin{array}{c}x_{1}^{t}\\ x_{2}^{t}\\ \vdots\\ x_{N}^{t}\end{array} \right]=DTCN\Big{(}X_{1}^{(t-1)},X_{2}^{(t-1)},\cdots,X_{N} ^{(t-1)}\Big{)}, \tag{1}\]  
where \(DTCN(\cdot)\) represents a deep convolutional network model that needs to be learned. Equation (1) is a general model and cannot distinguish the influence of input variables on output variables. In order to find the directly related airports that help to predict the delay of the target airport, the input variables need to be separated for model  
Figure 5: The DTCN prediction model architecture based on the attention mechanism. training. \(N\) airports need to train \(N\) deep time domain convolution prediction models \(\{DTCN_{1}(\cdot),DTCN_{2}(\cdot),\cdots,DTCN_{N}(\cdot)\}\), then Equation (1) can be written as:  
\[\left\{ \begin{array}{c}x_{1}^{t}=DTCN_{1}(X_{1}^{(t-1)},X_{2}^{(t-1)},\cdots,X _{N}^{(t-1)})\\ \vdots\\ x_{j}^{t}=DTCN_{j}(X_{1}^{(t-1)},X_{2}^{(t-1)},\cdots,X_{N}^{(t-1)})\\ \vdots\\ x_{N}^{t}=DTCN_{N}(X_{1}^{(t-1)},X_{2}^{(t-1)},\cdots,X_{N}^{(t-1)}).\end{array} \right. \tag{2}\]  
Further, an attention mechanism is introduced to measure the impact of delays at different airports on the target airport. For any target airport \(j\), an \(N\)-dimensional vector \(\mathbf{w}_{j}=[w_{1j},w_{2j},\cdots,w_{Nj}]\) is introduced to perform point-by-point multiplication with the input time series of \(N\) airports. Call \(\mathbf{w}_{j}\) the attention score vector (or contribution vector) of all airports (including the target airport itself) to the target airport. When the neural network based on the attention mechanism processes a large amount of input information, it will select a part of the key information for processing, and ignore the information that is irrelevant to the output.