# An Attention-Based Deep Convolution Network for Mining Airport Delay Propagation Causality
## 3 Methodology
### Causality Verification
#### 3.3.1 Candidate Causality Filtering Based on Attention Scores  
In Section 3.2, we propose an airport delay prediction model integrating an attention mechanism by introducing the attention mechanism from the perspective of the airport network. In the model training phase, the attention score is used as a parameter of the prediction model, which is trained simultaneously with the convolution parameters. After training the model, the attention score ranges between \(-\infty\) and \(+\infty\). In order to make the attention score reflect the influence degree of the airport more intuitively, the following semi-binarization function is used to process the arbitrary attention score vector \(\mathbf{w}_{j}=[w_{1j},w_{2j},\cdots,w_{Nj}]\):  
\[m_{ij}=\left\{\begin{array}{ll}\frac{e^{w_{ij}}}{\sum_{i}^{N}e^{w_{ij}}}& \text{if }w_{ij}\geq w_{0}\\ 0,&\text{if }w_{ij}<w_{0}\end{array}\right., \tag{4}\]  
where represents the attention score threshold. If \(m_{ij}=0\), it is considered that the delayed airport \(i\) will not affect the delay of the airport \(j\); if \(m_{ij}>0\), it is considered that delayed airport \(i\) will affect the airport \(j\), and the degree of influence can be objectively measured by the value of \(m_{ij}\).  
It should be noted that the selection of the threshold \(w_{0}\) is intuitive and important. If the value \(w_{0}\) is too small, it will lead to too many causal relationship pairs of candidate airport delay propagation, which makes the constructed causal relationship network too complicated and inconvenient for analysis and utilization. Conversely, if the value \(w_{0}\) is  
\begin{table}
\begin{tabular}{l} Input data: \\ All airport delay time series: \(X_{1},\cdots X_{i},\cdots,X_{N}\) \\ Predicting target airport delay time series: \(X_{j}\) \\ Parameter: Hidden layer number \(L\); training times Epochs; learning rate learning-rate; \\ convolution kernel size \(k\); expansion coefficient \(d\); initialization attention factor \(W\); optimizer \\ Outputs: \\ Predicting target airport delay time series \\ \end{tabular}  
Training steps: \\ Step 1: Read in data: including all airport delay time series and target airport delay time series. \\ Step 2: The network initialization parameters are sampled from a normal distribution with a mean of 0 and \\ a standard deviation of 0.1. Predict the target airport delay time series based on all the input airport delay time series. \\ Step 3: Calculate the distance between the predicted value of the network and the real target airport delay time series according to the MSE error function. \\ Step 4: Backpropagation is performed to calculate the gradient value of the loss function for the network parameters. Zero the gradient values in the optimizer before backpropagation. \\ Step 5: Modify the network parameters according to the gradient value and the definition of the optimizer, so that the distance between the predicted value of the network and the real value becomes smaller and smaller. \\ Step 6: Repeat steps 3–5 until the number of model training times is reached. \\ \end{tabular}
\end{table}
Table 1: Training steps of the deep time-domain prediction model.  
too large, although the number of candidate causal relationship pairs is reduced, which is beneficial to subsequent analysis and application, some real causal relationships will be filtered out. Since the purpose of excavating the causal relationship of delay propagation between airports is to provide a scientific basis for air traffic managers to implement flow management, too many causal relationship pairs are inconvenient to operate in practice. Too few causal relationship pairs ignores some key airport pairs, resulting in poor regulation. To this end, this paper will select a certain number of candidate causal relationship pairs corresponding to the number of threshold parameters as threshold parameters by taking different values through experiments.