# Effective Detection of Malicious Uniform Resource Locator (URLs) Using Deep-Learning Techniques
## 4 Result and Discussion
### Experimental Results and Discussions  
Our proposed model used the training and testing datasets to assess its performance. The processed dataset split in an 80/20% splitting ratio for training and testing sets, respectively. We used the BERT model as the base model that used to drive the Char2B model. _Experiment 1_: We used the BERT model as the base model to construct the fusion of BERT embedding with CharBiGRU to obtain the Char2B model. The BERT model's performance was rigorously evaluated on the test dataset using multiple metrics: a confusion matrix, classification report, and ROC curve analysis. These evaluations were conducted using consistent epoch settings across both training and testing phases, while employing different batch sizes for optimization. Therefore, as shown in Figure 9, the number of correctly predicted negative (benign) samples out of all negative samples is 8728 with 153 samples misclassified as positive samples and the number of correctly predicted positive (malicious) samples out of the total positive samples is 8429 with 134 misclassified as negative samples. Table 3 illustrates the training and testing performance of the base model using our dataset. Figure 10 depicts the trade-off between the TPR and the FPR across different threshold settings, which is an essential tool for evaluating a binary classifier's performance. In Figure 10, the Y-axis shows the true positive rate, which is the percentage of actual positive instances that are correctly detected, while the X-axis shows the false positive rate, which is the percentage of actual negative cases that are mistakenly categorized as positive. The curve displays the classifier's performance at various thresholds; an ideal model is represented by a TPR of one and an FPR of zero in the upper left corner of the graph. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Hyperparameters** & **BERT Model** & **Char2B Model** \\ \hline Embedding\_Dim & 768 & 256 \\ \hline Max\_Position & 200 & 200 \\ \hline Num\_Layers & 12 & 12 \\ \hline Num\_Heads & 12 & 12 \\ \hline Dropout & 0.1 & 0.3 \\ \hline Hidden\_Size & 768 & 768 \\ \hline Learning Rate & 0.00002 & 0.00002 \\ \hline Weight\_Decay & 0.0001 & 0.0001 \\ \hline Optimizer & AdamW & AdamW \\ \hline Batch\_Size & 16 & 8 \\ \hline Epoch & 5 & 5 \\ \hline Scheduler & Reduce\_IROnPlateau & Reduce\_IROnPlateau \\ \hline Mode & Min & Min \\ \hline Factor & 0.1 & 0.1 \\ \hline Patience & 1 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Hyperparameters for BERT and Char2B models. The diagonal dashed line in the ROC graph represents a random classifier where the TPR and FPR are equal; any model along this line performs no better than chance. The model performs well in this particular curve, as seen by its placement close to the upper left corner. The classifier has a reported AUC of 0.9898, which indicates that it is quite good at differentiating between the two classes with little overlap. This high AUC value, which shows a high probability of ranking a randomly selected positive case higher than a randomly selected negative instance, supports the finding that the model is excellent at accurately recognizing positive examples while limiting false positives, with a result of 0.018. Overall, the ROC curve and its AUC offer a thorough assessment of the model's capacity to distinguish between benign and malicious events. Moreover, Table 4 shows the classification report precision, recall, and F1-score: All values are 0.98 for both classes (zero and one), indicating a strong balance between correctly identifying positives and negatives. The accuracy of the overall model accuracy is 98%, meaning it correctly classified 17,444 out of 17,444 samples. Figure 10: The ROC-AUC in the BERT model of TPR vs. FPR.