## 5 Comparative Analysis Results and Discussion
### Performance Comparison for All Models  
Figure 11 compared the validation and training and accuracy for all BERT models stacked with different network-based models. The highest validation accuracy was 97.3% attained by ClaimsBERT, which is a classifier we proposed based on BERT followed by optimal feature map obtained from CNN, fully connected NN and a classifier. Figure 11 compares the training and validation loss for our new model ClaimsBERT against our previous model CyBERT, BERT + MLP, BERT + BiLSTM, BERT + LSTM and BertForSequenceClassification. Based on the loss plots, we can see how well the learning rate function initialized both models. Furthermore, the loss plots illustrate how early-stopping avoids overfitting when the validation loss increases in the training phase after four epochs. We also evaluated and compared the classification performance of our ClaimsBERT and other network-based models, with the results shown in Table 9. To evaluate the time complexity for these models, we utilized the same machine when training both models and measured the time needed to complete the training process. We then also evaluated the classification time, again utilizing the same machine, with the results for training and classification time measurements shown in Table 9. All time measurements are shown in seconds. The classification time refers to the time required to complete the entire training process, from building the model architecture, loading the best trained model checkpoint, and training the model on the set of sequences. The classification time measurements we report in this table represent the time that the respective model required in order to test the classifier on each of the 6420 sequences in the dataset. The classification time indicates that for the exact same set of data on the same machine, the ClaimsBERT model is only 19 s, or 2.96%, slower than CyBERT, while the ClaimsBERT accuracy is two percent higher than CyBERT. We believe that the significant boost in accuracy 97% more than justifies the slight increase in classification time. Figure 10: Kernel density estimation (**a**) and box plot (**b**) for all sets accuracy; histogram only for validation set accuracy (**c**). Determining and reviewing the receiver operating characteristic (ROC) curves is a useful technique for evaluating binary classification algorithms. The ROC curve demonstrates a Dual dimensional presentation of the classifier efficiency. The False Positive Rates (FPR) versus the True Positive Rates (TPR) in classification are plotted in ROC. We would like to observe a high TPR and a low FPR in the ideal classifier scenario [69]. The True Positive Rate, which is also known as sensitivity is calculated as:  
\[TPR=\frac{TP}{(TP+FN)}, \tag{10}\]  
Figure 11: Comparing training accuracy (**left**), validation accuracy (**right**) for all tested models. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Training Time \(\lx@sectionsign\)** & **Classification Time \(\lx@sectionsign\)** & **Trainable Parameters** \\ \hline BERT + CNN & 12,873 & 727 & 110,769,474 \\ (ClaimsBERT) & & & \\ \hline BERT + NN & 32,970 & 708 & 108,647,026 \\ (CyBERT [8]) & & & \\ \hline BERT + BiLSTM & 51,211 & 1125 & 112,915,970 \\ \hline BERT + LSTM & 47,176 & 989 & 109,482,240 \\ \hline BERT + MLP & 10,478 & 821 & 109,380,482 \\ \hline BERTSequence & 7832 & 335 & 109,483,778 \\ Classifier & & & \\ \hline \hline \end{tabular} \(\lx@sectionsign\) Times are in seconds.  
\end{table} 
Table 9: Comparing ClaimsBERT and other tested models performance. where \(TP\) measures the probability of an actual positive instance being classified as positive, and where \(FN\) is a measure of the probability that an actual positive instance will be classified as negative. The False Positive Rate is calculated as:  
\[FPR=\frac{FP}{(TN+FP)}, \tag{11}\]  
where \(FP\) is a measure of the probability that an actual negative instance will be classified as positive. \(TN\) is the probability of an actual negative instance classified as negative.