# Automated Anomaly Detection and Causal Analysis for Civil Aviation Using QAR Data
## 4 Multi-Type Anomaly Detection in Civil Aviation
### Model Design  
The XGBoost algorithm is a machine learning algorithm based on Gradient Boosting Tree, which uses a technique called tree boosting to construct models. XGBoost, as an efficient and scalable gradient boosting algorithm, performs well in handling multiple classification problems. For multiple classification tasks, XGBoost trains a decision tree for each category, synthesizes the outputs of each sub-tree, and finally outputs the classification results for the samples.  
Define the training dataset as \(\mathcal{D}\), and the number of parameters in \(\mathcal{D}\) as \(d\).  
\[(X_{i,j},y_{i})_{i=1}^{\mathcal{D}} \tag{2}\]  
where \(X_{i,j}=\{x_{1,j},x_{2,j},\ldots,x_{d,j}\}\), \(j\in[1,L]\), \(L\) is the number of attribute columns in \(\mathcal{D}\), and \(y_{i}\in\{0,1,\ldots,N-1\}\).  
If \(y_{i,n}\) represents the true value of the \(i\)-th sample on the \(n\)-th class, then the following statement can be made:  
\[y_{i,n}=\left\{\begin{array}{ll}1&\quad y_{i}=n\\ 0&\quad\text{otherwise}\end{array}\right. \tag{3}\]  
and then \(\hat{y}_{i,n}\) represents the predicted value of the \(i\)-th sample on class \(n\).  
XGBoost is an additive model of boost, where each iteration optimizes only the sub models in the current step. The step \(m\) can be defined as follows:  
\[F_{m}(x_{i})=F_{m-1}(x_{i})+f_{m}(x_{i}) \tag{4}\]  
where \(f_{m}(x_{i})\) is the sub-model of the current step, and \(F_{m-1}(x_{i})\) gives the first \(m-1\) sub-models that have completed training and are fixed. Therefore, its objective function is defined as follows:  
\[L^{(t)}=\sum_{i=1}^{M}l(y_{i},\hat{y}_{i}^{(t-1)}+f_{t}(x_{i}))+\Omega(f_{t}) \tag{5}\]  
where \(L^{(t)}\) represents the objective function of the \(t\)-th anomaly category, \(l(y_{i},\hat{y}_{i}^{(t-1)}+f_{t}(x_{i}))\) represents the loss function of the \(i\)-th sample, and \(\Omega(f_{t})\) denotes the value of the penalty term to prevent overfitting.  
In the method proposed in this paper, the Optuna module uses weighted F1 values as the objective function, and we use the cross-entropy loss function in the XGBoost model. The objective function of the MAD-XFP method is as follows:  
\[\begin{split}&\text{{Objective}}=L^{(t)}-\sum_{i=1}^{n}\sum_{j=1}^{k}w_{i,j} \log(p_{i,j})\\ &=\Omega(f_{t})-\sum_{i=1}^{M}\sum_{n=0}^{N-1}(y_{i,n}+w_{i,n}) \log(\hat{p}_{i,n})\end{split} \tag{6}\]  
where \(N\) is the number of the category of anomaly events, \(y_{i,n}\) represents whether the true label of the \(i\)-th sample belongs to the \(n\)-th class, and \(\hat{p}_{i,n}\) represents the predicted probability that the \(i\)-th sample belongs to the \(n\)-th class.  
\[w_{i,n}=\frac{(1+\beta 2)\cdot\text{Precision}_{n}\cdot\text{Recall}_{n}}{ \beta 2\cdot\text{Precision}_{n}+\text{Recall}_{n}} \tag{7}\]  
where \(\beta\) is a weight parameter, and its default value is one. \(\text{Precision}_{n}\) and \(\text{Recall}_{n}\) are, respectively, the precision and recall of the \(n\)-th class.  
The XGBoost model also provides a quantitative way to demonstrate the importance of features that affect the model, which measures the value of features in the construction of decision trees for model improvement. Figure 5 shows the characteristic parameters calculated by the XGBoost model that are very relevant to the abnormal events in the approach phase. The quantization sorting is based on the average gain of each feature on the predicted results in all trees and reflects the splitting ability of the feature at each node, because the stronger the splitting ability of a feature at each node, the greater its contribution to the final prediction result.  
Overall, the improved XGBoost model proposed in this paper is capable of establishing complex decision tree structures and performs well in capturing complex relationships in data, improving classification accuracy.