## 2 Materials and Methods
### Experimental Setup Description
#### 2.2.1 K-Fold Cross Validation  
For the implementation of the architecture shown in Figure 2, the first step consists of defining the training dataset for the k-fold cross validation, comprising two classes: \"fracture\" and \"noFracture\". In particular, to keep the two classes balanced and reduce the computational times, we considered a reduced dataset, which is a subset of the total dataset described in Section 2.1. In particular, the training dataset used for the k-fold cross validation consists of 359 slices with fracture, belonging to 57 different patients, and 362 slices without fracture, belonging to 59 additional patients. In order to avoid class imbalance in patient-level, from some patients with fractures, we selected only a subset of the \"noFracture\" slices. Therefore, these patients will become patients with \"noFracture\" in this phase. In our case study, we adopted the transfer learning technique to reduce the development burden of the CNN. The pre-trained architecture we used was ResNet50. ResNet is the deep convolutional neural network that won the 2015 ImageNet Large Scale Visual Recognition Challenge (ILSVRC) [21]. ResNet architecture has many variants: the difference between them is not only a different number of layers, but also a novel architecture, such as ResNeXt [22], or densely connected CNN [23]. ResNet50 is trained on more than a million images from the ImageNet database [24]. The network is 50 layers deep and can classify images into 1000 object categories, such as pizza, umbrella, castle, and many animals (tiger, camel, frog, etc.). As a result, the network has learned rich feature representations for a wide range of images. The network has an image input size of 224-by-224. The architecture of ResNet50 has 4 stages:  
1. Initial convolution (kernel size of \(7\times 7\)) and max-pooling (kernel size of \(3\times 3\));
2. Nine convolutional layers: kernel size of \(1\times 1\) and 64 different kernels, followed by kernel size of \(3\times 3\) and 64 different kernels, followed by kernel size of \(1\times 1\) and 256 different kernels. These three layers are repeated 3 times;
3. Twelve convolutional layers: kernel size of \(1\times 1\) and 128 different kernels, followed by kernel size of \(3\times 3\) and 128 different kernels, followed by kernel size of \(1\times 1\) and 512 different kernels. These three layers are repeated 4 times;
4. Eighteen convolutional layers: kernel size of \(1\times 1\) and 256 different kernels, followed by kernel size of \(3\times 3\) and 256 different kernels, followed by kernel size of \(1\times 1\) and 1024 different kernels. These three layers are repeated 6 times;
5. Nine convolutional layers: kernel size of \(1\times 1\) and 512 different kernels, followed by kernel size of \(3\times 3\) and 512 different kernels, followed by kernel size of \(1\times 1\) and 2048 different kernels. These three layers are repeated 3 times;
6. Average pooling layer followed by a fully connected layer with 1000 neurons and a softmax function at the end. In order to choose the most suitable set of hyperparameters for our case, we used the stratified k-fold cross validation [25] with k = 5. The hyperparameters of interest were the following: learning rate, weight decay, and dropout; we chose them in the following ranges (0.000001; 0.005), (0.0001; 0.0005), (0.1; 0.5). We set the batch size at 50. Specifically, 20 combinations (N = 20 in Figure 2) of the hyperparameters were tested. We used a random search for hyperparameters' optimization. We also chose to adopt a random search compared to a grid search. When there are many hyperparameters, as in our case, the first is more effective from the computational time point of view, while maintaining good performance [26]. Figure 3 describes the procedure of the k-fold cross-validation. Early stopping criteria can be used during the training as a trade-off between generalization ability and computational costs.