# Effective Detection of Malicious Uniform Resource Locator (URLs) Using Deep-Learning Techniques
## 4 Result and Discussion
### Experimental Results and Discussions
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Metrics** & **Loss** & **Accuracy** & **Precision** & **Recall (\%)** & **F1-Score** \\  & & **(\%)** & **(\%)** & **(\%)** & **(\%)** \\ \hline Training set & 0.0316 & 99.28 & 99.59 & 98.95 & 99.27 \\ Testing set & 0.0315 & 98.35 & 98.22 & 98.44 & 98.33 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Performance of the base model on the custom dataset. Figure 9: Confusion matrix of the BERT model evaluation result. The macro and weighted averages are also both 0.98, suggesting consistent performance across classes, even with slight class imbalance (8881 vs. 8563 samples). Therefore, the classification report shows high and consistent performance and the model performs reliably and robustly, with excellent predictive ability and no sign of bias toward either class. Figure 11 illustrates the classification report of our performance evaluation result. Out of 8881 benign test dataset samples, 98% of the dataset is correctly predicted as actually negative (benign) samples, and out of 8563 malicious test, dataset samples, 98% of samples are correctly predicted as actually positive (malicious) samples. The vertical axis displays the actual labels of the data, while the horizontal axis shows the model's predictions, which are either benign or malicious. The number of cases correctly predicted as benign (true negatives) is 8732, and these are located in the top left quadrant. This high number indicates that the model can accurately identify benign situations. There are 149 false positives in the upper right quadrant, which represent innocuous cases that were mistakenly labeled as malicious. Since it indicates fewer incorrect classifications of benign cases, a lower value is better in this case. The 112 false negatives that were mistakenly classed as benign are found in the bottom left quadrant. In this instance, a lower number is also preferable since it shows that harmful situations were successfully detected. The final quadrant, the bottom right one, displays 8451 true positives, or the cases that accurately classified as malicious. This high number indicates that the model can accurately detect harmful cases. We summarize the performance of the proposed model on the custom dataset using the classification report as shown in Table 5. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Class and Metrics** & **Precision** & **Recall** & **F1-score** & **Support** \\ \hline
0 & 0.98 & 0.98 & 0.98 & 8881 \\
1 & 0.98 & 0.98 & 0.98 & 8563 \\ Accuracy & & & 0.98 & 17,444 \\ Macro average & 0.98 & 0.98 & 0.98 & 17,444 \\ Weighted average & 0.98 & 0.98 & 0.98 & 17,444 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The classification report for the base model. Figure 11: Confusion matrix of the Char2B model evaluation result. _Experiment 2_: We obtained our proposed Char2B model using our training dataset as well as by testing the dataset and visualizing the confusion matrix result. The confusion matrix offers a comprehensive perspective on how well a classification model performs, particularly when it comes to differentiating between benign and malicious classifications. Next to the visualization of the performance metrics of the Char2B model using the classification report and confusion matrix, we used the ROC curve to illustrate how the model accurately distinguishes the positive (malicious) and negative (benign) samples using TPR and FPR. The ROC curve is a graphical tool used to evaluate the performance of a binary classifier by illustrating the trade-off between the TPR and the FPR at various threshold levels. On the graph shown in Figure 12, the x-axis represents the FPR of 0.017, measuring the proportion of actual negatives that incorrectly classified as positives, while the y-axis indicates the TPR, measuring the proportion of actual positives that were correctly identified. A diagonal dashed line represents a scenario where TPR equals FPR, indicating that a model performing no better than random guessing would lie along this line.