## 4 Dataset Description
### Norm-VGG16 Deep Learning Architecture  
One of the most prominent deep learning architectures is VGGs Network. VGG16 [32] architecture is utilized for its known superior performance, for example, compared to other models like AlexNet [33]. This may be attributed to its small kernel size (\(3\times 3\)) and more trainable parameters, which helps increase the depth of CNN without the problem of gradient loss in deep CNNs. A set of modifications to the standard VGG network architecture are proposed to create Norm-VGG16 Network variant architecture that enhances the performance of the basic architecture. First, the number of convolution layers is increased from 13 layers in standard VGG to 16 convolution layers. Each of the added layers has a kernel of size \(3\times 3\) and stride = 1. The pipeline of the convolution layers starts with first layer of size \(200\times 200\) and is convolved by 64 (\(3\times 3\)) kernels. The ending layer of the pipelined convolution layers has 512 (\(3\times 3\)) kernel with output of size \(3\times 3\). Between each group of convolution layers there is a max pooling layer of size (\(2\times 2\)) to down sample the feature tensor to reduce overfitting. The second applied modification is adding a global average pooling layer followed by a dense layer to reduce the overfitting of CNN. The dense layer has an output of four neurons representing the four classes of our dataset. Another enhancement to the standard architecture is adding L2 kernel regularization to limit overfitting. The added regularization is shown in Figure 5 at the last dense layer. Kernel regularization was first added in different layers, but the best performance is attained by the variant scenario implementing L2 kernel regularization at the dense layer (last layer). The value of the hyperparameter of L2 regularizer, which determines the relative importance of the regularization component compared to the loss component, is set to 0.01. Figure 4: The proposed HyCAD system architecture. L2 regularizer puts a constraint on the complexity of a network by giving a small value to its weights, which makes the distribution of weights regular. This is done by the increasing loss function of the network, which has large weights [34]. To explain how the regularizer works, a training function \(\hat{y}\): \(f(x)\) should be first defined as a function that maps an input vector \(x\) to output \(\hat{y}\) where \(\hat{y}\) is predicted value for actual value \(y\). The loss between \(\hat{y}\) (\(f(x)\)) and \(y\) can be defined for one sample \(x_{i}\) with corresponding target \(y_{i}\). Loss (\(L\)) can be computed as \(L(y_{i})\), \(\hat{y}_{i})=L (\(\hat{y}_{i}\)), \(y_{i})\)[35]. For all input samples \(x_{i}\)...... \(x_{n}\). The sum of all loss functions between each input \(x_{i}\) and its corresponding output \(\hat{y}\) is minimized as given in Equation (1). \[\min\ \sum_{i=1}^{n}L(f(x_{i}),y_{i}) \tag{1}\]  
The regularizer (\(R(f)\)) is added to the loss equation to increase the training loss to decrease overfitting as shown in Equation (2). \[L(f(x_{i}),\ y_{i})=\sum_{i=1}^{n}L_{\textit{losscomponent}}(f(x_{i}),y_{i})+ \lambda R(f) \tag{2}\]  
where \(\lambda\) is a hyperparameter that determines the relative importance of the regularization component compared to the loss component [35]. We choose our \(\lambda\) = 0.01. From Equation (1) and Equation (2), minimization after adding the regularization would occur to both the component of loss and the regularization component. Different values for \(\lambda\) are tested in the range between 0 and 1 (0.001, 0.01, 0.02, 0.1, etc.) according to the commonly used values in literature [34; 35]. The best results are obtained at \(\lambda\) = 0.01. The L2 regularization is a type of regularization that is proportional to the square of the value of the weight coefficients (the L2 norm of the weights) [36] as shown in Equation (3).