# IgA Nephropathy Prediction in Children with Machine Learning Algorithms
## 2 Materials and Methods
### Model  
In this section, XGBoost has been introduced as the best performing algorithm. For processing high-dimensional data, dimensionality reduction, feature extraction, etc., it has a higher accuracy than traditional algorithms. XGBoost is an improved gradient boosting algorithm [26]. The innovation of XGBoost lies in the optimization of the objective function with the second-order Taylor expansion. It merges multiple weak classifiers in order to evolve into a strong classifier, and the base classifier is a classification and regression tree (CART). The objective function of XGBoost consists of a loss function and a regularization term, which are defined, as follows:  
\[Obj=\sum\limits_{i=1}^{n}l(y_{i},\hat{y}_{i})+\sum\limits_{k=1}^{K}\Omega(f_{ k}) \tag{2}\]  
where \(f_{k}\) is the function expression of the \(k\)-th tree model and \(y_{i}\) and \(\hat{y}_{i}\) are the true label and predicted value of the \(i\)-th sample \(x_{i}\), respectively. XGBoost is an additive model, so the predicted value is the sum of the predicted values of each tree, i.e., \(\hat{y}_{i}=\sum\limits_{k=1}^{K}f_{k}(x_{i}),f_{k}\in F\). The sum of the complexity of \(K\) trees is used as a regularization term for preventing the model from over-fitting. Assuming that the tree model that is trained on the \(t\)-th iteration is \(f_{t}\), then:  
\[\hat{y}_{i}^{(t)}=\sum\limits_{k=1}^{t}f_{k}(x_{i})=\hat{y}_{i}^{(t-1)}+f_{t}( x_{i}) \tag{3}\]  
Substitute Equation (3) into Equation (2) to obtain Equation (4). \[Obj^{(t)}=\sum\limits_{i=1}^{t}l(y_{i},\hat{y}_{i}^{(t)})+\sum\limits_{i=1}^{ t}\Omega(f_{i})=\sum\limits_{i=1}^{t}l(y_{i},\hat{y}_{i}^{(t-1)}+f_{t}(x_{i}))+ \Omega(f_{t}) \tag{4}\] \begin{table}
\begin{tabular}{c c c} \hline \hline
**Variable** & **Score** & **\(p\)-Value** \\ \hline IgA & 0.025 & 0.875 \\ IgG & 0.816 & 0.366 \\ C4 & 0.001 & 0.973 \\ C1q & 3.444 & 0.063 \\ Hypertension & 11.592 & 0.001 \\ ACEI\_ARB & 0.211 & 0.646 \\ Immunosuppressive therapy & 0.232 & 0.630 \\ Lipid lowering & 9.725 & 0.002 \\ Tonsillectomy & 0.488 & 0.485 \\ Loop necrosis & 0.024 & 0.877 \\ Arterial hyaline degeneration & 0.012 & 0.911 \\ Medullary interstitial fibrosis & 5.655 & 0.017 \\ Thickening and stratification of elastic layer of interlobular artery & 6.373 & 0.012 \\ Vacuolar degeneration of arteriole smooth muscle cells & 0.490 & 0.484 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: _Cont._Expand the loss function \(\sum\limits_{i=1}^{t}l(y_{i},\hat{y}_{i}^{(t)})\) to second-order Taylor  
\[ \begin{split} Obj^{(t)}\cong\sum\limits_{i=1}^{t}\left[l(y_{i}, \hat{y}_{i}^{(t-1)}+f_{t}(x_{i}))\right]=\\ \sum\limits_{i=1}^{t}\left[l(y_{i},\hat{y}_{i}^{(t-1)}+g_{i}f_{t} (x_{i})+\frac{1}{2}h_{i}f_{t}^{2}(x_{i}))\right]+\Omega(f_{t})\end{split}  \tag{5}\]  
where \(g_{i}\) and \(h_{i}\) are the first-order partial derivative and second-order partial derivative of the loss function l with regard to \(\hat{y}_{i}^{(t-1)}\), respectively, and \(\Omega(f_{t})=\gamma T+\frac{1}{2}\lambda\sum\limits_{j=1}^{T}\omega_{j}{}^{2}\). Define the leaf node \(I_{j}=\{i|\hat{q}(x_{i})=j\}\), and the objective function is finally reduced to  
\[Obj^{(t)}=\sum\limits_{j}^{T}\left[\left(\sum\limits_{i\in I_{j}}g_{i}\right) \omega_{j}+\frac{1}{2}\left(\sum\limits_{i\in I_{j}}h_{i}\right)\omega_{j}{}^{ 2}\right]+\gamma T \tag{6}\]  
During the training process of the XGBoost model, when the \(t\)-th tree is established, the greedy strategy is adopted in order to split the tree nodes. Every time the tree node splits into two left and right leaf nodes, it will bring gain to the loss function, which is defined, as follows:  
\[ \begin{split} Gain=Obj_{L+R}-(Obj_{L}+Obj_{R})=\\ \frac{1}{2}\left[\frac{\left(\sum\limits_{i\in I_{L}}g_{i} \right)^{2}}{\left(\sum\limits_{i\in I_{L}}h_{i}\right)+\lambda}+\frac{\left( \sum\limits_{i\in I_{R}}g_{i}\right)^{2}}{\left(\sum\limits_{i\in I_{R}}h_{i} \right)+\lambda}-\frac{\left(\sum\limits_{i\in I_{R}}g_{i}\right)^{2}}{\left( \sum\limits_{i\in I_{i}}h_{i}\right)+\lambda}\right]-\lambda\end{split}