## 6 Conclusions  
This paper introduces the AugFake-BERT technique, which generates synthetic fake data to circumvent the problem of minority classes and performs classification using the most advanced language model, Bidirectional Encoder Representation of Transformers (BERT). This strategy consists of a pre-trained multilingual BERT for enhancing text data and feeding the augmented data to a fine-tuned BERT for generating embeddings. We generated an augmented dataset which escalates the minority class (Fake). Furthermore, we fed the augmented data to the embedding BERT model. We experimented with a frozen and unfrozen BERT base uncased architecture to perform the embedding generation. Finally, a feed-forward neural network with softmax activation was used to classify the data. The experimental results indicate that the AugFake-BERT technique is superior to twelve other state-of-the-art algorithms in identifying the veracity of a news article (LR, DT, MNB, SGD, SVM, KNN, RF, CNN, LSTM, Bi-LSTM, CNN-LSTM, and M-BERT). We utilized various performance evaluation parameters, such as F1-score, precision, recall, etc., to validate the results. The comparison table indicates that the proposed unfrozen BERT trained with an augmented dataset is competitive in terms of various performance metrics. The proposed model outperformed the current models utilized by other researchers, with a final accuracy score of 92.45%. The performance of the proposed method is prominent when considering the small increase in the number of the minority class. We demonstrated that applying text augmentation significantly affects the classification outcome of the existing classifiers. We strongly believe that this work can motivate future researchers to invest effort in a robust self-supervised language model-based fake news detection system. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Imbalanced Dataset**} & \multicolumn{4}{c}{**Balanced Dataset**} \\ \hline
**Model** & **A(\%)** & **P(\%)** & **R(\%)** & **F1-S(\%)** & **A(\%)** & **P(\%)** & **R(\%)** & **F1-S(\%)** \\ \hline LR & 50.27 & 51.63 & 24.31 & 37.52 & 88.66 & 85.11 & 87.58 & 86.16 \\ \hline MNB & 52.16 & 52.10 & 22.19 & 46.29 & 86.16 & 82.42 & 83.45 & 85.60 \\ \hline DT & 54.27 & 54.00 & 20.80 & 37.06 & 86.71 & 85.43 & 84.39 & 83.59 \\ \hline RF & 54.16 & 53.00 & 21.50 & 39.49 & 85.21 & 84.26 & 84.62 & 84.74 \\ \hline SGD & 55.49 & 55.21 & 22.42 & 40.90 & 88.41 & 87.63 & 87.62 & 88.67 \\ \hline SVM & 55.17 & 55.70 & 23.83 & 40.24 & 88.32 & 87.28 & 88.00 & 88.57 \\ \hline KNN & 53.71 & 51.09 & 20.77 & 30.29 & 87.60 & 87.54 & 87.70 & 87.56 \\ \hline CNN & 54.13 & 54.34 & 23.46 & 36.34 & 83.34 & 80.23 & 84.48 & 83.52 \\ \hline LSTM & 58.41 & 58.43 & 34.00 & 45.32 & 84.69 & 84.86 & 85.72 & 84.15 \\ \hline Bi-LSTM & 67.24 & 67.45 & 37.78 & 53.68 & 85.10 & 86.86 & 85.43 & 86.85 \\ \hline CNN-LSTM & 69.10 & 69.13 & 37.45 & 59.82 & 88.83 & 87.68 & 87.23 & 88.35 \\ \hline M-BERT & 68.54 & 68.21 & 33.67 & 51.23 & 88.69 & 86.86 & 87.33 & 89.55 \\ \hline
**AugFake-BERT** & **75.12** & **75.21** & **35.43** & **59.47** & **92.45** & **92.86** & **91.23** & **91.85** \\ \hline \hline \end{tabular}
\end{table} 
Table 13: Performance metrics of the classifiers on the balanced and imbalanced datasets. **Author Contributions:** Conceptualization, A.J.K., M.A.H.W. and M.F.M. ; methodology, A.J.K. ; data curation, M.F.M. and A.J.K. ; software, A.J.K., M.A.H.W. ; validation, M.F.M., M.A. and M.A.H. ; formal analysis, M.A. and M.F.M. ; investigation, A.J.K. ; resources, M.F.M. and M.A.H.W; writing--original draft preparation, A.J.K. ; writing--review and editing, M.A., M.A.H.W. and M.A.H. ; visualization, A.J.K. and M.A.H.W. ; supervision, M.F.M. ; project administration, M.F.M. All authors have read and agreed to the published version of the manuscript  
**Funding:** This research received no external funding. **Institutional Review Board Statement:** Not applicable. **Informed Consent Statement:** Not applicable. **Data Availability Statement:** Not applicable.