# Detection of Pediatric Femur Configuration on X-ray Images
## Appendix A  
In this work, contrary to frequently used hand engineering, we propose to optimize the structure of the estimator through a heuristic random search in a discrete space of _hyperparameters_. The _hyperparameters_ will be defined as all CNN features selected in the optimization process. The following features are considered as _hyperparameters_[26]: number of convolution layers, number of neurons in each layer, number of fully connected layers, number of filters in convolution layer and their size, batch normalization [29], activation function type, pooling type, pooling window size, and probability of dropout [28]. Additionally, the batch size \(\mathbf{X}\) as well as the learning parameters: learning factor, cooldown, and patience, are treated as _hyperparameters_, and their values were optimized simultaneously with the others. What is worth noticing--some of the _hyperparameters_ are numerical (e.g., number of layers), while the others are structural (e.g., type of activation function). This ambiguity is solved by assigning individual dimension to each _hyperparameter_ in the discrete search space. In this study, 17 different _hyperparameters_ were optimized [26]; therefore, a 17-th dimensional search space was created. A single architecture of CNN, denoted as \(\mathcal{M}\), is featured by a unique set of _hyperparameters_, and corresponds to one point in the search space. The optimization of the CNN architecture, due to the vast space of possible solutions, is achieved with the tree-structured Parzen estimator (TPE) proposed in [41]. The algorithm is initialized with \(n_{s}\) start-up iterations of random search. Secondly, in each \(k\)-th iteration the _hyperparameter_ set \(\mathcal{M}_{k}\) is chosen, using the information from previous iterations (from 0 to \(k-1\)). The goal of the optimization process is to find the CNN model \(\mathbf{\mathcal{M}}\), which minimizes the assumed optimization criterion (7). In the TPE search, the formerly evaluated models are divided into two groups: with low loss function (20%) and with high loss function value (80%). Two probability density functions are modeled: \(G\) for CNN models resulting with low loss function, and \(Z\) for high loss function. The next candidate \(\mathcal{M}_{k}\) model is chosen to maximize the Expected Improvement (\(EI\)) ratio, given by:  
\[EI(\mathcal{M}_{k})=\frac{P(\mathcal{M}_{k}\in G)}{P(\mathcal{M}_{k}\in Z)}. \tag{1}\]  
TPE search enables evaluation (training and validation) of \(\mathcal{M}_{k}\), which has the highest probability of low loss function, given the history of search. The algorithm stopsafter predefined \(n\) iterations. The whole optimization process can be characterized by Algorithm A1. ```
Result:\(\mathbf{\mathcal{M}}\), \(\mathbf{L}\)  Initialize empty sets: \(\mathbf{L}=\varnothing\), \(\mathbf{\mathcal{M}}=\varnothing\);  Set \(n\) and \(n_{s}<n\); for\(k=1\)to\(n\_startup\)do  Random search \(\mathcal{M}_{k}\);  Train \(\mathcal{M}_{k}\) and calculate \(L_{k}\) from (7); \(\mathbf{\mathcal{M}}\cup\mathcal{M}_{k}\); \(\mathbf{L}\cup L_{k}\);  end for for\(k=n_{s}\)to\(n\)do  Sort \(\mathbf{\mathcal{M}}\) is increasing order of \(L\);  Model density \(G(\mathcal{M}_{j})\), \(j=1,...,0.2k\) with TPE;  Model density \(Z(\mathcal{M}_{j})\), \(j=0.2k\), \(...,k\) with TPE;  Choose \(\mathcal{M}_{k}\) with maximum \(EI\) (A1); if\(\mathcal{M}_{k}\
otin\mathbf{\mathcal{M}}\)then  Train \(\mathcal{M}_{k}\) and calculate \(L_{k}\) from (7); \(\mathbf{\mathcal{M}}\cup\mathcal{M}_{k}\); \(\mathbf{L}\cup L_{k}\);  end for  end for
```  
**Algorithm A1**CNN structure optimization  
The presented optimization returns the set of the evaluated model architectures:  
\[\mathbf{\mathcal{M}}\triangleq\{\mathcal{M}_{k}\}_{k=1}^{n},\] (A2)  
together with corresponding loss function values  
\[\mathbf{L}\triangleq\{L_{k}\}_{k=1}^{n},\] (A3)  
in the increasing order.