Identification of Synthetic Sentence in B engali News using Hybrid Approach (2019)

Abstract: Often sentences of correct news are either made biased towards a particular person or a group of persons or parties or maybe distorted to add some sentiment or importance in it. Engaged readers often are not able to extract the inherent meaning of such synthetic sentences. In Bengali, the news contents of the synthetic sentences are presented in such a rich way that it usually becomes difficult to identify the synthetic part of it. We have used machine learning algorithms to classify Bengali news sentences into synthetic and legitimate and then used some rule-based postprocessing on each of these models. Finally, we have developed a voting based combination of these models to build a hybrid model for Bengali synthetic sentence identification. This is a new task and therefore we could not compare it with any existing work in the field. Identification of such types of sentences may be used to improve the performance of identifying fake news and satire news. Thus, identifying molecular level biasness in news articles.

## introduction the bengali language is rich in terms of the usage of its words. it is also a relatively free word order language. by changing the order of the same set of words, the author can add some emphasis to some part of the sentence. it is usually observed that the bengali sentences are frequently distorted like this way. the number of ways english sentences can be distorted is much less than the number of ways a bengali sentence can be. but all the distorted sentences not necessarily have added biasness or emphasis. some readers take the inherent meaning of the sentences without getting into involved in the biased part of it. they can take out an overview of the text. but often, an engaged reader gets engaged with the writer's views. sometimes it is not so harmful or it is preferred to be an engaged reader. for example, to get the full flavour of a literary work, the reader has to be engaged. but often it is not desirable. for example, in a piece of political news, it is not recommended to engage a reader without his concern. so, it is essential to notify the reader about synthetic sentences. often sentences of correct news are either made biased towards a particular person or a group of persons or parties or maybe distorted to add some sentiment or importance in it. we refer such types of sentences as synthetic sentences. engaged readers often are not able to extract the inherent meaning of synthetic sentences. in bengali, the news contents of the synthetic sentences are presented in such a rich way that it usually becomes difficult to identify the synthetic part of it. in this paper, we wish to identify the synthetic sentences in bengali news automatically. we use the linguistic features in multiple binary machine learning classifiers to decide whether it is synthetic or legitimate. then analyzing a confusion matrix, we apply a set of rules. finally, we combine these models using a voting-based approach. we test this hybrid technique in a bengali news corpus covering the news in politics, sports, entertainment, and social domains. the final hybrid technique is able to provide 86% accuracy. the rest of the paper is organized as follows. section 2 illustrates a background study related to synthetic news detection. section 3 and section 4 discuss how we have prepared the experimental dataset and model building part. section 5 shows the results of different steps. finally, section 6 presents concluding remarks of the task. ## related work there are some works in the detection of fake news. they detect fake news in a news corpus @xcite ; @xcite ; @xcite ; @xcite that is, misleading news stories which come from non-reputable sources. these papers mainly focus on fake news from four perspectives: the false knowledge, its writing style, its propagation patterns, and the credibility of its creators and spreaders. @xcite describes three types of fake news in contrast to reporting. these are -serious fabrications (uncovered in mainstream or participant media); large-scale hoaxes; humorous fakes (news satire, parody). @xcite discussed the threats posed by automatically generated propaganda articles that closely mimics the style of real news. they have designed a language model-based system called grover for the controllable generation of text from the title of the news. humans may find this generated text to be more trustworthy compared to the actual news article. such type of fake news called neural fake news is discriminated best using the generator system itself. bradshaw and howard (2017) compared the teams who spread manipulated information, also called disinformation through social media and news across 28 countries including india. these types of fake news are created manually to influence the voters and domestic audiences purposely. @xcite designs a global disinformation index (gdi) to combat the disinformation. another essential type of fake news is created by proliferating stylistic bias in the text. @xcite have used linguistic features in support vector machine (svm) to detect these fake news in some english newspapers. @xcite discriminated between synthetic and legitimate news using 5 features namely absurdity, humor, grammar, negative affect, and punctuation. ## dataset preparation we evaluate our proposed framework on two datasets, kaggle bengali news, and online bengali news. for the time being, we are not using the name of the newspapers to avoid the controversy. in total, we have 25k news covering seven different domains, namely kolkata, state, national, sports, entertainment, world, and travel. each news contains on an average of 15 sentences. the distribution of the bengali news dataset in the seven domains is shown in fig. 1 ## proposed system after preprocessing of the raw news, the news is tokenized and segmented into sentences level. in this paper, we create a hybrid approach by combining binary machine learning classifiers using the voting approach and then postprocessing with a rule-based approach. the proposed system is as follows: in this paper, we consider synthetic news classification for independent sentences as each sentence carries some synthetic or legitimate property. our approach is to use a committee of classifiers, each trained on a set of text features. the entire list of features is presented in this section. ## experimental result various evaluation metrics have been used to evaluate the performance of machine learning models. we want to test the performances of our models in terms of accuracy, precision, recall, and f1-score. these metrics are commonly used to evaluate the machine learning models and enable us to evaluate the performance of a classifier from different perspectives. the results of the k-fold cross-validations for each of our hybrid models are shown in section 5.1. then we have applied the voting approach to combine the models and then the rules. the final accuracy of this hybrid system is discussed in section 5.2. ## conclusion and future scope in this paper, we defined and compared synthetic and legitimate sentences and highlighted many interesting differences between these two categories. we then utilized these differences as features to detect synthetic sentences. we have proposed a hybrid approach that can detect synthetic news. to the best of our knowledge, our work is the first attempt to detect synthetic news at the bengali news sentence level. in the future, we want to extend it to use semantic features in the machine learning model and calculate the degree of synthetic property in the synthetic sentences. then we want to compare the renowned bengali newspapers in terms of the usage of different types of synthetic sentences.