# Defect Recognition in Ballastless Track Structures Based on Distributed Acoustic Sensors
## 5 XGBoost-Based Defect Recognition Approaches for Ballastless Tracks
### XGBoost  
XGBoost [20] is an implementation of gradient-boosted decision trees (GBDTs) that can quickly solve prediction and classification problems in data science. The main theory of XGBoost is to continually add regression trees and obtain results by adding the value of each tree.  
(1) Objective function  
Suppose the dataset has \(n\) samples and \(m\) features;  
\(D=\left\{(x_{i},y_{i})\right\}\)\(\left(|D|=n,x_{i}\in\mathbb{R}^{m},y_{i}\in\mathbb{R}\right)\), the predictive function of XGBoost can be expressed as:  
\[\hat{y}_{i}=\sum\limits_{k=1}^{K}f_{k}(x_{i}),\;\;f_{k}\in\mathcal{F} \tag{3}\]  
where \(\mathcal{F}=\left\{f(x)=w_{q(x)}\right\}\)\(\left(q:\mathbb{R}^{m}\to T,w\in\mathbb{R}^{T}\right)\) is the CART space, \(q\) represents the structure of each tree, mapping samples to corresponding leaf nodes and \(T\) is the number of leaf nodes in the tree. Each \(f_{k}\) corresponds to a tree whose leaf node weight is \(w\).  
To learn the set of functions used in the model, it is necessary to minimize the objective function with a regularization term:  
\[\begin{array}{l}\zeta(\phi)=\sum\limits_{i}l(\hat{y}_{i},y_{i})+\sum\limits_ {k}\Omega(f_{k})\\ \Omega(f)=\gamma T+\frac{1}{2}\lambda\|w\|^{2}\end{array} \tag{4}\]  
In Equation (4), \(\hat{y}_{i}\) represents the predicted value of the model and \(y_{i}\) represents the class label of the \(i\)th sample. The first term is the loss function and the second term is the regularizer, which is used to control the tree and avoid overfitting.  
(2) Taylor expansion  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Domain** & **Feature Description** & **Number** \\ \hline \multirow{2}{*}{Time} & Average amplitude, Standard deviation, Zero & \multirow{2}{*}{6} \\  & crossing rate, Root mean square, Skewness, Kurtosis & \\ Frequency & Average amplitude, Standard deviation, Skewness, & \\  & Kurtosis, Spectral entropy, Spectral centroid & \\ Wavelet packet & Wavelet packet energy spectra & 8 \\ Mel scale & Mel-frequency cepstral coefficients & 13 \\ Total & & 33 \\ \hline \hline \end{tabular}
\end{table}
Table 2: The extracted features for each frame.  
From Equation (3), the prediction result of sample \(i\) after the \(t\)th iteration is known as:  
\[\begin{array}{l}\hat{y}_{i}^{(0)}=0\\ \hat{y}_{i}^{(1)}=f_{1}(x_{i})=\hat{y}_{i}^{(0)}+f_{1}(x_{i})\\ \hat{y}_{i}^{(2)}=f_{1}(x_{i})+f_{2}(x_{i})=\hat{y}_{i}^{(1)}+f_{2}(x_{i})\\ \cdots\\ \hat{y}_{i}^{(t)}=\sum\limits_{k=1}^{t}f_{k}(x_{i})=\hat{y}_{i}^{(t-1)}+f_{t} (x_{i})\end{array} \tag{5}\]  
A new function \(f_{t}(x_{i})\) that minimizes the following objective function is introduced:  
\[\zeta^{(t)}=\sum\limits_{i=1}^{n}l\Big{(}y_{i},\hat{y}_{i}^{(t-1)}+f_{t}(x_{i} )\Big{)}+\Omega(f_{t}) \tag{6}\]  
Then, the Taylor expansion is performed on the objective function in Equation (6); the first three terms are taken and the high-order infinitesimal terms are removed. We arrived at:  
\[\zeta^{(t)}\approx\sum\limits_{i=1}^{n}\bigg{[}l\Big{(}y_{i},\hat{y}^{(t-1)} \Big{)}+g_{i}f_{t}(x_{i})+\frac{1}{2}h_{i}f_{t}^{2}(x_{i})\bigg{]}+\Omega(f_{ t}) \tag{7}\]  
where \(g_{i}=\partial_{g^{(t-1)}}l\Big{(}y_{i},\hat{y}^{(t-1)}\Big{)}\) and \(h_{i}=\partial_{g^{(t-1)}}^{2}l\Big{(}y_{i},\hat{y}^{(t-1)}\Big{)}\) are the first and second derivatives of the loss function, respectively.  
This does not affect the optimization of the function, since \(l\Big{(}y_{i},\hat{y}_{i}^{(t-1)}\Big{)}\) is a constant in Equation (7). By removing all of the constant terms, the simplified objective function can be obtained:  
\[\widetilde{\zeta}^{(t)}=\sum\limits_{i=1}^{n}\bigg{[}g_{i}f_{t}(x_{i})+\frac{ 1}{2}h_{i}f_{t}^{2}(x_{i})\bigg{]}+\Omega(f_{t}) \tag{8}\]  
Only \(g_{i}\) and \(h_{i}\) need to be considered in Equation (8), so the first and second derivatives of the loss function for each round can be calculated to obtain \(f_{t}(x_{i})\).