## 3 Proposed TSR Methodology
### Traffic Sign Classification (TSC) Step
#### 3.2.2 TSC Models and Architectures
Classification metrics The possible values to get in a classification problem can be defined as follows:  
* True positives (TP): the number of positive instances that were classified as positive;
* True negatives (TN): the number of negative instances that were classified as negative;  
Figure 10: TSC model architecture with attention mechanism (TSC-MA). * False positives (FP): the number of negative instances that were classified as positive;
* False negatives (FN): the number of positive instances that were classified as negative;
* The used metrics are accuracy, precision, recall, and F1;
* Accuracy, used to evaluate the classification model, is the ration of the number of correct predictions to the total number of predictions: \[\text{Accuracy}=\frac{\text{TP }+\text{ TN}}{\text{TP}+\text{FP}+\text{TN}+\text{FN}};\] (1)
* Precision, often referred to as positive predictive value, is the ratio of correctly classified positive instances to the total number of instances classified as positive: \[\text{Precision}=\frac{\text{TP}}{\text{TP }+\text{ FP}};\] (2)
* Recall, also called sensitivity or true positive rate, is the ratio of correctly classified positive instances to the total number of positive instances: \[\text{Recall}=\frac{\text{TP}}{\text{TP }+\text{ FP}};\] (3)
* F1 combines precision and recall as a single value: \[\text{F1}=\frac{\text{Precision }*\text{ Recall}}{\text{Precision }+\text{ Recall}}.\] (4)  
In our work, 43 classes have unbalanced distributions in the dataset. For this, in addition to the accuracy metric, the weighted averaged (precision, recall, and F1) score is used to improve the lowest scores. The weighted average is calculated by taking the mean of all per-class (precision, recall, and F1) scores while considering each class's support (support refers to the number of actual testing occurrences of the class in the dataset);  
3. Obtained results using the TSC model without the attention mechanism (TSC_M)  
The TSC_M is tested on 12,630 images from the GTSRB dataset. Table 7 shows that after training the model for 70 epochs, 99.97% of training accuracy is achieved with a reduced loss of 0.00098. The achieved validation accuracy is 99.68% with a reduced loss of 0.0105. The testing accuracy reaches 98.56%, which is considered an interesting result compared to other small-scale models like the LeNet-5 model, which achieved 90%, and 97.71% for the small-scale CNN model developed by [48]. Moreover, accuracy and loss curves shown in Figure 11 confirm further the efficiency of our developed deep learning CNN model. Referring to Table 8, the deep learning CNN algorithm achieves a weighted average of 98%, which is better than the 97% obtained by the authors in [49]. The F1 score result is 100% when predicting ten classes, 99% for fourteen classes, 98% for seven classes, 97% for four classes, 96% for three classes, and 95% for five classes. Despite these good results, the test accuracy can be further improved by applying attention mechanisms in the developed neural network; \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Epoch**} & \multicolumn{3}{c}{**Accuracy (\%)**} & \multicolumn{3}{c}{**Loss (\%)**} \\ \cline{2-7}  & **Training** & **Validation** & **Testing** & **Training** & **Validation** & **Testing** \\ \hline
30 & 99.89 & 99.43 & 98.10 & 0.00339 & 0.02 & 0.070 \\ \hline
70 & 99.97 & 99.68 & 98.56 & 0.00098 & 0.01 & 0.060 \\ \hline
100 & 99.99 & 99.63 & 98.46 & 0.00050 & 0.01 & 0.067 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: TSC_M accuracy rates trained over 30, 70, and 100 epochs. 4. Obtained results using the TSC model with attention mechanism (TSC_MA)  
The TSR attention CNN model is trained for 30 and 70 epochs (Ep). The achieved results are summarised in Table 9. In fact, testing accuracy reaches 99.91% with a reduced loss of 0.062, which is considered an interesting accuracy rate compared to the previously achieved testing accuracy. The curves in Figure 12 describe a good fit of the model.