## 2 Methodology
### CNN-Based Regressors
#### 2.3.2 Our CNN-Based Regressor  
In this paper, we propose use of the deep residual network architecture (ResNet) as introduced in [63]. In contrast to other CNN architectures, the ResNet consists of several residual blocks, which are composed of a convolutional layer, a batch normalization layer, and a shortcut that connects the original input to the output of the residual block. Figure 3a shows schematically how a Residual Block with Identity Shortcut (RB-IS) and a Residual Block with Projection Shortcut (RB-PS) operate. Mathematically, the residual block can be summarized as follows:  
\[h_{l}=F(h_{l-1},\{W_{l}\})+G(h_{l-1}), \tag{7}\]  
where and \(h_{l-1}\) is the output of the \(l-1\)-th layer, \(F\) is the residual function (e.g., a composition of convolutional layers, batch-norm layers, pooling-layers, and nonlinear activations such as ReLU), \(G\) corresponds to the shortcut connection, and \(G(h_{l-1})\) could be either the identity map, \(G(h_{l-1})=h_{l-1}\), or a projection, \(G(h_{l-1})=W_{p}h_{l-1}\), \(W_{l}\) denote weights of a residual block, and \(W_{p}\) stands for weights of the projection. If the dimensions of \(h_{l-1}\) and \(h_{l}\) are the same, the identity shortcuts are used; otherwise, a projection mapping is applied, meaning a linear projection \(W_{p}\) is performed on the shortcut connections to match the dimensions.  
The main idea of the ResNet is to learn the additive residual function \(F\) with respect to \(G(h_{l-1})\), with the choice to use an identity mapping and/or projection mapping. In our implementation, we changed the last fully connected layer of ResNet to output 1 scalar instead of the original 1000-\(D\) vector. In addition, the SoftMax loss function was replaced with MAE loss.  
In this paper, we used xResNet, which is a modified version of the original ResNet architecture, which has shown to increase accuracy, and provide improved performance for transfer learning tasks, compared to ResNet architectures [47]. xResNet applies a couple of small tweaks compared to the original architecture. The first one is shown in Figure 3b as  
Figure 3: The original ResNet architecture (**a**) and the adapted xResNet architecture (**b**).  
ResNet-B in which the A path is altered in the downsampling block by moving the stride of 2 to the second convolution and keeping a stride of 1 for the first layer.  
The second modification can be seen in ResNet-C (Figure 3b): the 7 \(\times\) 7 convolution is removed in the input stem of the network and replaced with three consecutive 3 \(\times\) 3 convolutions, whereby the first one has a stride of 2 and the other two a stride of 1. The third modification is present in ResNet-D (Figure 3b): the path B of the downsampling layer is modified by adding average pooling with a stride of 2, before a convolution with a stride of 1, and not 2.  
For our purposes, we propose use of xResNet50, which consists of an input stem, four stages with each varying amounts of xResNet blocks, and an output stem. In the input stem there are three convolutional layers, each followed by batch normalization, and a ReLU. In between the input stem and the first of the four stages there is a max pooling layer. The first stage consists of three xResNet blocks, the second stage of four xResNet blocks, the third stage of six xResNet blocks, and the fourth stage of three xResNet blocks again. In between the fourth stage and the output stem, there is an average pooling layer. In the output stem the input is first flatted and dropout is applied, before the final fully connected linear layer, which has 1 output. In total there are trainable 23,529,313 parameters.