# Classification of Shoulder X-ray Images with Deep Learning Ensemble Models
## 3 Methods
### Proposed Classification Models Based on EL for Shoulder Bone X-ray Images
#### 3.3.1 EL1 (ResNext50 with Spinal FC, DenseNet169 with Standard FC, and DenseNet201 with Spinal FC)  
The EL1 (Ensemble learning-1) model was developed using a combination of ResNext50 with Spinal FC, DenseNet169 with Standard FC, and DenseNet201 with Spinal FC. A schematic diagram of the developed model is shown in Figure 14. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Models** & **Spinal FC Layer Width** & **Models** & **Spinal FC Layer Width** \\ \hline ResNet34 & 256 & ResNeXt50 & 20 \\ ResNet50 & 128 & ResNeXt101 & 128 \\ ResNet101,152 & 1024 & DenseNet169, 201 & 240 \\ VGG13 & 256 & MobileNetV2 & 320 \\ VGG16,19 & 512 & InceptionV3 & 20 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Layer with values of Spinal FCs used in classification models. Figure 14: The proposed EL1 classification model. It is observed upon examination of the figure above of the EL1 model proposed for classification of shoulder bone X-ray images as normal/abnormal (fracture) that three different CNN-based models are used. The \"data\" specified in this figure represent the entire dataset of shoulder bone X-ray images, \"Model I\" is the ResNeXt50 model with SpinalNet FC, \"Model II\" is the DenseNet169 model with a Standard FC layer, \"Model III\" is the DenseNet201 model with SpinalNet FC, \"Outputs 1-3\" are the outputs of these three models, and \"Prediction\" is the normal/abnormal (fracture) class types. In selection of the three models specified in this ensemble model, the parameters in the training outputs of the 26 CNN models presented in the previous title were taken into consideration. These parameters can be stated as [PERSON]'s kappa score, AUC, and test accuracy. The steps of the procedure and a block diagram in Figure 15 for the proposed EL1 ensemble model are provided as follows:  
* Step 1: The last layers of the three pre-trained sub-models in the EL1 ensemble model are adjusted as the Identity layer. * Step 2: A final layer with 80 outputs for ResNext50 with Spinal FC, 1664 for DenseNet169 with Standard FC, and 960 for DenseNet201 with Spinal FC is achieved. * Step 3: These outputs are combined to form a single linear layer and a hidden layer with 2704 outputs. * Step 4: This hidden layer is connected to the classifying layer connected to the sigmoid activation function, the output of which is 2. * Step 5: The network established as a result of these procedures is re-trained, providing results. Figure 15: A block diagram of the proposed EL1 ensemble model. 3.3.2. EL2 (ResNet34 with Spinal FC, DenseNet169 with Standard FC, DenseNet201 with Spinal FC, and ResNext50 with Spinal FC)  
The EL2 (Ensemble learning-2) model was developed using a combination of ResNet34 with Spinal FC, DenseNet169 with Standard FC, DenseNet201 with Spinal FC, and a sub-ensemble model. A schematic diagram of the developed model is provided in Figure 16. It is observed upon examination of the figure that the EL2 model, differently from the EL1 model, consists of three single models and one sub-ensemble model and has a different structure. The \"X-ray Images\" specified in the figure refers to the dataset containing shoulder bone images, \"Model I\" to the ResNet34 model with Spinal FC, \"Model II\" to the DenseNet201 model with Spinal FC, \"Model III\" to the sub-ensemble model, \"Model IV\" to the DenseNet169 model with Standard FC, and \"Predictions I-IV\" refer to the outputs of the classification performed with these four models. The parts of the procedure for the proposed EL2 ensemble model are as follows:  
* Part 1: In the \"Input\" section, there is a shoulder X-ray image dataset that has been subjected to certain image processing techniques. * Part 2: In the \"Classification Network\" section, the models that establish our ensemble model are defined. There are three single sub-models and a sub-ensemble model that constitute our ensemble model therein.