# A Hybrid Intelligent Approach to Predict Discharge Diagnosis in Pediatric Surgical Patients
## 2 Materials and Methods
### Experimental Methodology
#### 2.2.2 Main Processing  
The central processing phase was designed to produce discharge diagnosis predictions from numerical samples prepared in the preprocessing stage.  
Data filtering is a two-step process that includes selecting relevant features for classification and removing noisy data. Feature selection techniques can be divided into three categories: filter, wrapper, and embedded methods. From these three categories, wrapper methods have the advantages of using feature dependencies and are developed with classifier performance [36]. Additionally, as a difference from embedded methods, the classifier can be replaced by any other available once the most relevant features are found.  
In the proposed system, the sequential forward selection (SFS) wrapper method was employed to select the most relevant features evaluated [37; 38]. In order to remove noisy records that are likely to affect classification performance negatively, medical records were filtered out using the Density-based spatial clustering of applications with noise (DBSCAN) clustering. The DBSCAN algorithm aimed to find the essential samples with higher density in order to expand clusters from them, finding clusters of similar density [26].  
XGBoost was employed to train the ensemble of decision trees for discharge diagnosis prediction. Ensembles of classifiers take advantage of the diversity of opinions between weak classifiers to produce accurate and commonly with more stable classification performance. A decision tree (DT) is a predictive model commonly used as a weak classifier in ensembles, in which the conjunctions of features are represented in the branches, and conclusions or decisions are represented in the leaves (class labels). XGBoost is commonly employed for training decision trees, where an objective function is defined for the supervised learning of a model [39]. Training consists of searching for the best parameters \(\theta\) that fit the training data \(x_{i}\) and the labels \(y_{i}\), see Equation (5):  
\[Obj(\theta)=L(\theta)+\Omega(\theta), \tag{5}\]  
where \(L=\) is the training loss; and \(\Omega\) is the regularization term. \(L\) measures how predictive the model is regarding the training data, and its equation is given by Equation (6):  
\[L(\theta)=\sum_{i}(y_{i}-\hat{y}_{i})^{2}. \tag{6}\]  
On the other hand, the regularization term controls the complexity of the model and is given by  
\[\gamma*T+\frac{1}{2}\lambda*\sum_{j=1}^{T}\omega_{j}^{2}, \tag{7}\]  
where \(T\) is the number of end nodes of the tree, and \(\omega\) is the vector of scores on end nodes.  
The solution is provided by trees that are constructed sequentially and learn from each tree's predecessors. The learning scheme is called additive training: the functions \(f_{i}\) contain the structure of the tree and the end nodes' scores. The learning structure of the tree is complex because it cannot be learned from all the trees at once, and a prediction value is obtained at each step \(t\) as \(\hat{y}^{t}\) according to Equation (8):  
\[\begin{array}{l}\tilde{y}^{0}=0,\\ \tilde{y}^{1}=f_{1}=\tilde{y}^{0}+f_{1}(X_{i}),\\ \tilde{y}^{t}=\sum_{k=1}^{t}f_{k}(x_{i})=\hat{y}_{i}^{t-1}+f_{1}(X_{i}).\end{array} \tag{8}\]  
Optimizing performance in decision trees includes finding the maximum depth to prune the trees backward, eliminating losses, and optimizing learning. Other parameters to be considered are the number of trees, the learning rate to prevent overfitting, the percentage of samples used per tree, and the percentage of features used per tree [6; 40]. The \(GridSearch\) algorithm was employed to automatically optimize each classifier's hyperparameters on a validation set [41]. The \(GridSearch\) algorithm starts by defining a limited number of valuesfor the hyperparameters, then the Cartesian product of these sets is evaluated through a sequential combination. Additionally, a 10-fold cross-validation process was applied for the statistical comparison of performance [42].