## 3 Materials and Methods
### Preact ResNet Model's Description
This indicates a residual dependency between any units (\(I\)) and (\(i\)). Second, examining \(x_{I}=x_{0}+\sum_{j=0}^{l-1}F\big{(}x_{j},W_{j}\big{)}\), it becomes evident that the output feature of block \(I\) is the summation of \(x_{0}\) and residual functions, contrasting with the simple matrix-vector products found in a \"plain network\". \begin{table}
\begin{tabular}{c c} \hline
**ResNet-18’s Residual Block** & **ResNet-50’s Residual Block** \\ \hline
3 \(\times\) 3, 64 & 3 \(\times\) 3, 64 \\  & 1 \(\times\) 1, 256 \\ \hline \end{tabular}
\end{table} 
Table 2: Comparison of the residual blocks of RESNET-18 and RESNET-50. In larger ResNets, it is important to note that the final layer within a residual block has a feature map dimension four times larger than that of the initial layer. Despite the difference in the number of layers (two versus three), the overall computational load remains nearly equivalent. The principal advantage of Equation (4) is that it facilitates effective backpropagation. Denoting the loss function as \(L\), and applying the chain rule of backpropagation, we obtain the following [18]:  
\[\frac{\partial L}{\partial x_{i}}=\frac{\partial L}{\partial x_{j}}\frac{ \partial x_{I}}{\partial x_{i}}=\frac{\partial L}{\partial x_{I}}\bigg{(}1+ \frac{\partial}{\partial x_{i}}\sum_{j=i}^{I-1}F\big{(}x_{j},W_{j}\big{)}\bigg{)}. \tag{5}\]  
Equation (5) shows that the gradient \(\frac{\partial L}{\partial x_{i}}\) comprises two additive terms: one term (\(\frac{\partial L}{\partial x_{I}}\)) that propagates information directly without regard to weight layers and another term (\(\frac{\partial L}{\partial x_{I}}\Big{(}\frac{\partial}{\partial x_{i}}\sum_ {j=i}^{I-1}F\Big{)}\)) that propagates through the weight layers. The additive term of \(\frac{\partial L}{\partial x_{I}}\) ensures information is directly backpropagated to any shallower block (\(i\)). Assuming \(\frac{\partial}{\partial x_{i}}\sum_{j=i}^{I-1}F\) is not consistently \(-1\) across all the samples in a mini-batch, the probability of the gradient \(\frac{\partial}{\partial x_{i}}\) being canceled out is extremely low. This property ensures that the layer's gradient remains intact, even when the weights are very low. A crucial requirement for the validity of Equations (4) and (5) is the ability for signals to propagate directly between units, both forward and backward. This requirement is met through two conditions: \(h(x_{i})=x_{i}\), the identity skip connection, and \(f(y_{i})=y_{i}\), the identity mapping. To achieve these conditions, we employ the concept of preactivation (see Figure 1), which involves placing batch normalization and the activation function (ReLU [19]) inside the residual block. Batch normalization (BN) [20] stabilizes learning by normalizing each layer's input. BN adjusts each mini-batch's mean and variance, keeping input values within a specific range. This leads to faster convergence by reducing the number of epochs needed, allows for higher learning rates, and can sometimes act as regularization, thereby reducing overfitting. The primary benefit is improved accuracy, particularly in large datasets. However, BN has drawbacks, including noisy mean and variance estimates for small batches and added computational complexity. Figure 1: Schemes of the original residual block (**a**) and preactivated residual block (**b**). The ReLU activation function introduces nonlinearity and is defined by the following formula [19]:  
\[relu(x)=x,\;if\;x\geq 0,\;else\;0. \tag{6}\]