# Hybrid Techniques for the Diagnosis of Acute Lymphoblastic Leukemia Based on Fusion of CNN Features
## 3 Materials and Methods
### Strategy of Machine-Learning with Fusion Features CNN
#### 3.4.3 XGBoost Classifier  
XGBoost is a powerful machine-learning classifier that is based on gradient-boosted decision trees. The algorithm creates a decision tree that is sequentially based on assigning weights to independent variables and inserting them into the decision trees to predict results. The algorithm integrates weak learners into strong learners through sequential models to produce a strong model. This process is called Boosting. When training the network, the weights of the variables predicted by the method (decision tree) are incorrectly increased [51]. The variables are sequentially fed to the next decision tree. Boosting is an ensemble learning method for creating strong classifiers from serially weak classifiers. The first model is built to predict the training dataset, then, building the next predictive model to solve the errors in the previous model, and the process continues until the predicted training dataset is accurate [52].  
This hybrid strategy has two techniques. The first technique is shown in Figure 5, which goes through the following sequence: first, optimizing the images of the C-NMC 2019 and ALL-IDB2 datasets. Second, extracting WBC cells and isolating them from other major blood components by the active contour method and keeping them in a folder in a new dataset called ALL-IDB2-ROI. Third, the C-NMC 2019 and ALL-IDB2-ROI datasets are fed into DenseNet121, ResNet50, and MobileNet models to extract high-level feature maps and then represent them into feature vectors with sizes 10,661 \(\times\) 1024, 10,661 \(\times\) 2048, and 10,661 \(\times\) 1024 for DenseNet121, ResNet50, and MobileNet models, respectively, for C-NMC 2019 dataset. Whereas with the ALL-IDB2 dataset, features were saved in feature vectors with sizes 260 \(\times\) 1024, 260 \(\times\) 20,148, and 1024 \(\times\) 470 for DenseNet121, ResNet50, and MobileNet models, respectively.  
Fourth, dimensionality reduction of high feature vectors by PCA and retention of accurate representative features in feature vectors of sizes of 10,661 \(\times\) 440, 10,661 \(\times\) 630, and 10,661 \(\times\) 470 for DenseNet121, ResNet50, and MobileNet models, respectively, for C-NMC 2019 dataset. Whereas with the ALL-IDB2 dataset, features were saved in feature vectors with sizes 260 \(\times\) 440, 260 \(\times\) 630, and 260 \(\times\) 470 for DenseNet121, ResNet50, and MobileNet models, respectively. Fifth, the RF and XGBoost classifiers separately receive feature vectors of the DenseNet121, ResNet50, and MobileNet models.  
As shown in Figure 6, the second technique goes through the following sequence: the first four steps are the same as the steps of the first technique. Fifth, serially combining feature vectors into new hybrid vectors as follows: combine vectors of DenseNet121-ResNet50, ResNet50-MobileNet, DenseNet121-MobileNet, and DenseNet121-ResNet50-MobileNet into new feature vectors with sizes of 10,661 \(\times\) 1070, 10,661 \(\times\) 1100, 10,661 \(\times\) 910, and 10,661 \(\times\) 1540 for C-NMC 2019 dataset. Whereas with the ALL-IDB2 dataset, features were saved in feature vectors of sizes 260 \(\times\) 1070, 260 \(\times\) 1100, 260 \(\times\) 910, and 260 \(\times\) 1540. Sixth, the RF and XGBoost classifiers receive the CNN mixed feature vectors and classify them with high precision.  
Figure 5: Framework of a hybrid technique between CNN and deep learning for the diagnosis of ALL.