## 5 Experiments and Results
### Discriminative Fine Tuning  
So far, a comparison has been made with different ways of working the LR while keeping it constant for all layers of the network. In this section, a novel technique is proposed. [PERSON] et al. proposes a novel method for the classification of texts. Discriminative fine-tuning [29] allows us to tune each layer with different learning rates. Different layers capture different types of information [30], so a tuning according to the capacity of each layer gives good results. This method is based on the fact that the SGD update with discriminative fine-tuning is then the following:  
\[\theta_{t}^{l}=\theta_{t-1}^{l}-\eta^{l}\
abla_{\theta^{l}}J(\theta) \tag{7}\]  
where \(\eta\) is the learning rate and \(\
abla_{\theta}J(\theta)\) is the gradient with regard to the model's objective function. For discriminative fine-tuning, \(\theta_{l}\) is replaced by \(\{\theta^{1},\dots,\theta^{L}\}\), where \(\theta^{l}\) contains the parameters of the model at the l-th layer. Similarly \(\eta\) is replaced by \(\{\eta^{1},\dots,\eta^{L}\}\), where \(\eta^{l}\) is the learning rate of the l-th layer and L is the number of layers of the model. We decrease the learning rate with the increasing depth of the layers and are thus in the optimal range for convergence. The LR factors for the different layers assigned a name to \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{3}{*}{**Metrics**} & \multirow{3}{*}{**Class**} & \multicolumn{6}{c}{**VGG19**} \\ \cline{3-8}  & & \multicolumn{6}{c}{**Cyclical Learning Rate**} \\ \cline{3-8}  & & **Triangular** & **Triangular2** & **Exponential** & **Exponential2** & **Sine** \\ \hline \multirow{8}{*}{Precision} & ADI & 0.99 & 0.99 & 0.95 & 0.99 & 0.99 \\  & BACK & 1.00 & 1.00 & 1.00 & 1.00 & 1.00 \\  & DEB & 0.98 & 0.95 & 0.95 & 0.91 & 0.98 \\  & LYM & 0.99 & 0.99 & 0.99 & 0.99 & 0.99 \\  & MUC & 1.00 & 0.98 & 0.99 & 0.98 & 1.00 \\  & MUS & 0.84 & 0.87 & 0.87 & 0.83 & 0.59 \\  & NORM & 0.94 & 0.92 & 0.96 & 0.93 & 0.90 \\  & STR & 0.78 & 0.81 & 0.75 & 0.69 & 0.90 \\  & TUM & 0.97 & 0.97 & 0.95 & 0.97 & 0.99 \\ \hline \multirow{8}{*}{Recall} & ADI & 0.98 & 0.99 & 0.99 & 0.97 & 0.82 \\  & BACK & 1.00 & 1.00 & 1.00 & 1.00 & 1.00 \\  & DEB & 1.00 & 0.99 & 0.99 & 1.00 & 0.99 \\  & LYM & 1.00 & 1.00 & 1.00 & 0.96 & 1.00 \\  & MUC & 0.98 & 0.98 & 0.92 & 0.98 & 0.97 \\  & MUS & 0.85 & 0.84 & 0.82 & 0.72 & 0.95 \\  & NORM & 0.97 & 0.98 & 0.96 & 0.96 & 0.99 \\  & STR & 0.77 & 0.76 & 0.77 & 0.82 & 0.65 \\  & TUM & 0.97 & 0.96 & 0.98 & 0.96 & 0.94 \\ \hline \multirow{8}{*}{F1-Score} & ADI & 0.98 & 0.99 & 0.97 & 0.98 & 0.90 \\  & BACK & 1.00 & 1.00 & 1.00 & 1.00 & 1.00 \\ \cline{1-1}  & DEB & 0.99 & 0.97 & 0.97 & 0.95 & 0.99 \\ \cline{1-1}  & LYM & 0.99 & 1.00 & 0.99 & 0.97 & 0.99 \\ \cline{1-1}  & MUC & 0.99 & 0.98 & 0.96 & 0.98 & 0.99 \\ \cline{1-1}  & MUS & 0.84 & 0.86 & 0.84 & 0.77 & 0.73 \\ \cline{1-1}  & NORM & 0.96 & 0.95 & 0.96 & 0.95 & 0.94 \\ \cline{1-1}  & STR & 0.78 & 0.78 & 0.76 & 0.75 & 0.75 \\ \cline{1-1}  & TUM & 0.97 & 0.97 & 0.97 & 0.97 & 0.96 \\ \hline \multirow{2}{*}{Accuracy} & Train & 0.99 & 0.99 & 0.99 & 0.99 & 0.99 \\ \cline{1-1}  & Test & **0.96** & **0.96** & 0.95 & 0.94 & 0.92 \\ \hline \multicolumn{2}{c}{Training time} & 158 m 9 s & 155 m 7 s & 157 m 40 s & 156 m 59 s & 173 m 52 s \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The impact of using different methods of cyclical variation of the LR with respect to epochs on the training and validation of our model. specify this value were chosen per layer bundle, i.e. : The first ten layers have a factor of 0.01, the next thirteen layers have a factor of 0.001, the last four layers reduce their factor to 0.0001. In multiclass problems, it is important to obtain the so-called confusion table to analyze the behavior of the system. The Figure 10 shows the behavior of the model when using discriminative fine-tuning with a decreasing LR in epochs. Both the confusion matrix and the confidence matrix are shown.