# Detection of Pediatric Femur Configuration on X-ray Images
## 3 Results
### PS Estimation  
As a result of training over 200 networks with different architectures, the one ensuring the minimum loss function value (7) was chosen. The network architecture is presented in Figure 8. The optimal CNN architecture [26] consists of 15 layers, 10 of which are convolutional. The size of the last layer represents the number of network outputs, i.e., the coordinates of keypoints \(\mathbf{k}_{1}\), \(\mathbf{k}_{2}\), \(\mathbf{k}_{3}\). After 94 epochs of training, the early stopping rule was met and the learning process was terminated. The loss function of development set was equal to 8.4507 px\({}^{2}\). The results for all learning sets are gathered in Table 2.  
Loss function values for all learning sets are within acceptable range, given the overall complexity of the assigned task. The performance was slightly better for the train set in comparison to the development set. This feature usually correlates to overfitting of train data. Fortunately, low test set loss function value clarified that the network performance is accurate for previously unknown data. Interestingly, test set data achieved the lowest loss function value, which is not common for CNNs. There may be several reasons for that. First, X-ray images used during training were of slightly different distribution than those from the test set. The train set consisted of images of children varying in age and, consequently, of a different knee joint ossification level, whereas the test set included adult X-rays. Second, train and development sets were augmented using typical image transformations, to constitute a valid CNN learning set (as described in Table 1). The corresponding loss function values in Table 2 are calculated for augmented sets. Some of the image transformations (randomly selected) resulted in high contrast images, close to binary. Consequently, those images were validated with high loss function value, influencing the overall performance of the set. On the other hand, the test set was not augmented, i.e., X-ray images were not transformed before the validation. The optimization of the _hyperparameters_ of CNN, as described in Appendix A, improved the process of network architecture tuning, in terms of processing time as well as low loss function value (7). The optimal network architecture (optimal in the sense of minimizing the assumed criterion (7)) consists of convolution layers with different window sizes, for convolution and for pooling layers. It is not consistent with the widely popular heuristics of small window sizes [33]. In this particular scenario, small window sizes in \begin{table}
\begin{tabular}{c c c} \hline \hline
**Learning Set** & **Proposed** & **U-Net [23]** \\  & **Solution** & **(with Heatmaps)** \\ \hline Train & 7.92 px\({}^{2}\) & 9.04 px\({}^{2}\) \\ Development & 8.45 px\({}^{2}\) & 10.31 px\({}^{2}\) \\ Test & 6.57 px\({}^{2}\) & 6.43 px\({}^{2}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: CNN loss function (7) values for different learning sets. Figure 8: The optimal CNN architecture. Each rectangle represents one layer of CNN. The following colors are used to distinguish important elements of the network: blue (fully connected layer), green (activation functions, where HS stands for hard sigmoid, and LR denotes leaky ReLU), pink (convolution), purple (pooling), white (batch normalization), and yellow (dropout). CNN resulted in higher loss function or exceeded the maximum network size limited by the hardware restrictions. Several regularization techniques were implemented, enabling the long-term learning process and avoiding overfitting of the goal function. For instance, the probability of dropout was high, especially in the deep layers of the network. Additionally, the most effective activation function was leaky ReLU [34]. The other well-known and widely popular activation function ReLU was also considered, nevertheless, it was Leaky ReLU that was chosen in all network layers.