shawon95/BengaliFakeReviewDetection

Bengali Fake Review Detection Model:

This model is taken from the paper 'Bengali Fake Reviews: A Benchmark Dataset and Detection System' which introduces the Bengali Fake Review Detection (BFRD) dataset, the first publicly available dataset for identifying fake reviews in Bengali. The dataset consists of 7710 non-fake and 1339 fake food-related reviews collected from social media posts. To convert non-Bengali words in a review a unique pipeline has been proposed that translates English words to their corresponding Bengali meaning and also back transliterates Romanized Bengali to Bengali. We have conducted rigorous experimentation using multiple deep learning and pre-trained transformer language models to develop a reliable detection system. Finally, we propose a weighted ensemble model that combines four pre-trained transformers: BanglaBERT, BanglaBERT Base, BanglaBERT Large and BanglaBERT Generator.

• Paper Link: https://www.sciencedirect.com/science/article/abs/pii/S0925231224005034

Fine tuned Bangla BERT Model

This model is basically a fine tuned Bangla BERT model on 13390 reviews, of which 6695 were fake (1339 were genuine fakes, while the remaining 5356 were augmented using nlpaug augmentation technique and 6695 were non-fake (randomly chosen from 7710 cases) from the BFRD dataset.

BFRD Dataset

• HuggingFace: https://huggingface.co/datasets/shawon95/Bengali-Fake-Review-Dataset

• Kaggle: https://www.kaggle.com/datasets/shawontanvir/bengali-fake-review-dataset

• paperswithcode: https://paperswithcode.com/dataset/bfrd

Using this model as a discriminator in transformers

# Load model directly
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("shawon95/BengaliFakeReviewDetection")
model = AutoModelForSequenceClassification.from_pretrained("shawon95/BengaliFakeReviewDetection")
review = "টাকা দিয়ে এই বিরিয়ানি কে খায় ভাই আর এটাকে বিরিয়ানি বলে? কার কাছে কেমন লাগে এই বোবার বিরিয়ানি?"

review_tokens = tokenizer.tokenize(review)
review_inputs = tokenizer.encode(review, return_tensors="pt")
discriminator_outputs = model(review_inputs).logits
predictions = torch.round((torch.sign(discriminator_outputs) + 1) / 2)

Performance of Bangla BERT model on Bengali Fake Review Dataset

Fake			Non-Fake
Precision	Recall	F1	Precision	Recall	F1	Weighted-F1	ROC-AUC	MCC
0.976	0.985	0.981	0.985	0.976	0.980	0.981	0.981	0.961

Citation

If you use this model, please cite the following paper:

@article{SHAHARIAR2024127732,
title = {Bengali fake reviews: A benchmark dataset and detection system},
journal = {Neurocomputing},
pages = {127732},
year = {2024},
issn = {0925-2312},
doi = {https://doi.org/10.1016/j.neucom.2024.127732},
url = {https://www.sciencedirect.com/science/article/pii/S0925231224005034},
author = {G.M. Shahariar and Md. Tanvir Rouf Shawon and Faisal Muhammad Shah and Mohammad Shafiul Alam and Md. Shahriar Mahbub},
keywords = {Bengali fake reviews detection, Ensemble learning, Transformers, Deep learning, Augmentation, Transliteration},
abstract = {The proliferation of fake reviews on various online platforms has created a major concern for both consumers and businesses. Such reviews can deceive customers and cause damage to the reputation of products or services, making it crucial to identify them. Although the detection of fake reviews has been extensively studied in English language, detecting fake reviews in non-English languages such as Bengali is still a relatively unexplored research area. The novelty of the study unfolds on three fronts: (i) a new publicly available dataset called Bengali Fake Review Detection (BFRD) dataset is introduced, (ii) a unique pipeline has been proposed that translates English words to their corresponding Bengali meaning and also back transliterates Romanized Bengali to Bengali, (iii) a weighted ensemble model that combines four pre-trained transformers model is proposed. The developed dataset consists of 7710 non-fake and 1339 fake food-related reviews collected from social media posts. Rigorous experiments have been conducted to compare multiple deep learning and pre-trained transformer language models and our proposed model to identify the best-performing model. According to the experimental results, the proposed ensemble model attained a weighted F1-score of 0.9843 on a dataset of 13,390 reviews, comprising 1339 actual fake reviews, 5,356 augmented fake reviews, and 6695 reviews randomly selected from the 7710 non-fake instances.}
}