## 4 Experiment and Result
### Evaluation Baseline
#### 4.4.2 Deep Learning Classifiers  
The DI-based models used for comparison with our proposed model are CNN, Bi-LSTM, and M-BERT. In a variety of problems, CNNs are beneficial in classifying short and long texts. We used kernels with lengths ranging from 1 to 4. As a pooling layer, we examined the global max pool and average pool. The outputs of the global max-pooling layer were concatenated and supplied as a pooled feature vector to the following layer. The hidden layer of the network was a dense fully-connected layer preceded by a dropout layer. Consequently, classification was performed using the output layer, which consisted of a dense layer with an activation function. Within the network, we employed ReLU as the activation function. Table (a)a displays the optimized hyperparameters of the CNN model for fake news classification. LSTM networks are the most extensively utilized models in text categorization and generation challenges thanks to their ability to capture sequential information in an effective manner. Bidirectional LSTM (Bi-LSTM) has displayed particularly outstanding ability in catching sequential information from both directions in texts [40, 47]. In this paper, we investigated LSTM and Bi-LSTM model with our dataset. We used a total of 256 LSTM units. Table (b)b displays the optimized hyperparameters of the LSTM and Bi-LSTM models for classifying fake news. In addition, a combination of CNN and LSTM was implemented to compare them with our model. The hyperparameters of CNN-LSTM were identical to those of the CNN and LSTM models. Figure 9: F1-score of various baseline methods on cross-validation datasets. Recently pre-trained multilingual language models have made significant progress in a variety of NLP tasks. BERT and its variant models, in particular, have outperformed the GLUE benchmark for Natural Language Understanding (NLU) [27]. To assess the scope of such a model in our research, we classified news articles using the multilingual BERT model. We used the pre-trained model from the HuggingFace Transformers libraries. A comparison of the above-mentioned models on the original dataset and augmented dataset is illustrated in Figure 10. The figure shows that handling imbalanced data improved the performance of these models. We experimented with undersampling the true news class as well. We trained our AugFake-BERT model with 4000 Fake news data items and 2000 true news data items. The model achieved an accuracy of 80.21% and a precision of 81.12%, whereas it achieved a low recall (56.35%) and f1-score (60.21%). Figure 11 illustrates the comparison of precision, f1-score, and recall on the imbalanced original dataset. Table 12 presents the testing result of the AugFake-BERT, M-BERT, CNN-LSTM, Bi-LSTM, LSTM, and CNN models on the test set of the augmented data, showing the highest performance obtained by our proposed AugFake-BERT model (accuracy = 92.45%, precision = 92.86%, recall = 91.23% and f1-score = 91.85%). The lowest performance is that of the CNN model, with an accuracy of 83.34%, precision of 80.23%, recall of 84.48%, and f1-score of 83.52%. The Bi-LSTM model has better results compared to CNN, with an accuracy of 85.10%, precision of 86.86%, recall of 85.43%, and f1-score of 86.85%. The combination of CNN-LSTM is the second highest in terms of performance (accuracy = 88.83%, \begin{table}
\begin{tabular}{c c} \hline \hline \multicolumn{2}{c}{**(a) Hyperparameters for CNN and CNN-LSTM.**} \\ \hline
**Parameters** & **Parameters Value** \\ \hline Kernel size & 3 \\ Filter Size & 64 \\ Activation & ReLU \\ Dropout Rate & 0.2 \\ Epochs & 4 \\ Batch Size & 32 \\ \hline
**Parameters** & **Parameters Value** \\ \hline Neurons & 100 \\ Activation & ReLU, Sigmoid \\ Recurrent & 0.2 \\ Dropout Rate & 4 \\ Batch Size & 32 \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Optimized hyperparameters adapted for CNN, LSTM, Bi-LSTM, and CNN-LSTM. Figure 10: The effect of accuracy of the CNN, LSTM, Bi-LSTM, CNN-LSTM, M-BERT, and AugFake-BERT models on the original and augmented dataset.