# Classification of Shoulder X-ray Images with Deep Learning Ensemble Models
## 5 Conclusions and Future Work  
The aim of this study was to find the most optimal model for classifying shoulder bone X-ray images as normal or abnormal (fracture). In order to achieve this aim, the first objective was to classify fracture images using 26 classification models, i.e., Standard FC and Spinal FC versions of 13 different CNN-based models. Based on the results obtained therein, two different ensemble models (EL1 and EL2) were developed and used for classification procedures in order to further increase classification accuracy and [PERSON]'s kappa score. Among the 28 different classifications models, the highest test accuracy and [PERSON]'s kappa score were achieved in the EL2 model, and the highest AUC score was achieved in the EL1 model. In the EL1 model, the fully connected layers of the models trained on the dataset were combined and turned into a single fully connected layer. This new layer was linked to the classification layer. In this way, in the last training part performed in the EL1 model, the weights in the new fully connected layer were updated, and the best configuration of the three models was obtained. For this reason, the EL1 model can be used on different datasets because it updates the weights of the new FC layer for the best configuration on any dataset. In the EL2 model, the sub-models consulted in the decision mechanism consist of models that show the best performance on the dataset. For this reason, if one is working on their own dataset, they should determine their sub-models \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Studies** & **Dataset** & **Best Method** & **Classification Type and Test Accuracy** \\ \hline
**Our study** & **MURA Shoulder, 563 validation/test** & **EL2** & **Fracture/normal: 0.8472** \\ [PERSON] and [PERSON] [5] & MURA dataset, 194 validation images & GCN & Fracture/normal: 0.9112 \\ [PERSON] et al. [6] & MURA dataset, various test images, \%50 & Capsule Network & Fracture/normal: 0.9208 \\ [PERSON] et al. [15] & 1376 CT images & ResNet152 & Normal/four different type: 0.96 \\ [PERSON] et al. [17] & 597 X-ray images & NASNet & Implant: 0.804 \\ Sezers [16] & 219 MR images & CNN & Normal/edematous/Hill-Sachs \\ [PERSON] [18] & 219 MR images & Extreme learning & Normal/edematous/[PERSON] \\ [PERSON] [19] & 1006 MR images & CapsNet & Normal/degenerated/torn: 0.9474 \\ \hline \hline \end{tabular}
\end{table} 
Table 16: Comparison with previous studies. Figure 30: ROC curves and AUC results of EL2. according to the performance results obtained with transfer learning. The reason for using transfer learning is the further improvement of classifications made with models that have not been pre-trained. With this approach, a serious increase in the classification problem was observed. Ensemble learning was mainly used to further improve the classification results obtained with 26 different deep learning models and to bring new approaches to the literature in this field. The contribution of the study to the literature is as follows. * In similar studies, binary classification is mostly performed. However, while there are mainly two classes (normal/abnormal) in this study, differently from the literature, multi-class classification was carried out, in order to determine the most compatible models to be used in ensemble models, developed by evaluating the outputs of each class of the 26 classification models initially used. This allowed the best results in this study to be achieved with ensemble models. * This is the first time that Spinal FC, which, compared to the Standard FC, has a lower number of weights in the hidden layer, was used in many models (Inception, ResNeXt, and MobileNet). Moreover, SpinalNet was used on medical images for the first time, and it had a positive effect on more than half of the classification results. * A unique structure is introduced, since the reliability of the detection of classes was used as a basis when designing the EL2 model, which further improves classification results.