# Enhancing Fake News Detection in Romanian Using Transformer-Based Back Translation Augmentation
## 2 Related Work  
Fake news is intentionally created to mislead and distort information, making it difficult to identify using traditional methods. An obstacle to accurately determining fake news is the limited availability of labeling news. Extensive investigations have been conducted to detect false news in English, employing diverse data augmentation techniques. This section presents prior research on data augmentation using transformer-based techniques. In (18), [PERSON] et al. propose a solution that integrates data augmentation to detect fake news in Bengali. The researchers used the \"BanFakeNews\" data set, a highly imbalanced data set with 1300 instances of fake news data out of 50,000 news items. To augment these data efficiently to balance the data set, they employed transfer learning using the BERT multilingual uncased model. This model was trained on a vast multilingual data set without human labeling, using two specific goals: Masked Language Modeling and Next Sentence Prediction. For data augmentation, they performed prediction of the next word instead of prediction of the next sentence, allowing for insertion by prediction. This approach considers surrounding terms to predict the target, addressing issues with static vectors in traditional word embedding models. Using this technique, 2700 fake news content items were generated, and after under-sampling true news items, a balanced augmented data set was produced. Classical ML and DL algorithms were used to train the classifier, with the best performance of AugFake-BERT, the model proposed by the authors. The proposed model has a final accuracy score of 92.45% for the balanced data set and 75.12 for the original. The performance gain is significant, but part of it may be due to leakage, because no strategy was presented to counter this effect. Contrastingly, in this paper, stratified cross-validation is used to address this issue. Article (19) investigates the effectiveness of text augmentation in detecting position and fake news. The researchers evaluated the effects of multiple augmentation methods: masked words prediction with BERT, BT with Google Cloud Translation API, synonym technique, TF-IDF insertion and substitution. Two publicly available data sets in English were used, FNC-1 for the position on news articles and FakeNewsNet for false news items. The findings of the study suggest that no single augmentation technique works well with all classification algorithms and that text augmentation only occasionally improves prediction performance. Furthermore, combining all augmentation methods does not produce the best results. The best performance to detect fake news was achieved by Bagged Random Forest with BT, with an F1 score of 91.34 compared to 88.09 without data augmentation. [PERSON] et al. [20] used ChatGPT 4 to balance and enhance an existing data set to detect fake news. The starting data set was the English portion of the CheckThat-2022 Task 3 which contained four labels: true, partially false, false, and other. To overcome the limitations of CheckThat-2022, which is relatively small and imbalanced, the capabilities of ChatGPT 4 were used and the data set was expanded from 900 to 2400 items. A small improvement in the Macro F1 score was reported for the mBERT model, from 0.23 to 0.25. The authors suggest that this technique does not contribute to improving the classification quality of false news due to the ethical limitations of ChatGPT in generating this type of news. [PERSON] et al. [21] proposed an ML system for Romanian fake news detection. A large data set of 26,128 items was constructed by crawling a list of online publishers that are untrustworthy for fake news and official sources for real news. The authors compared the performance in predicting fake news from classical algorithms with DL and transformer models. The presented method gained an F1 score of 97.80% for the convolutional neural network model. Our approach differs by using a smaller data set labeled by experts and by using data augmentation to enhance performance.