Enhancing T elugu News Understanding: Comparative Study of ML Algorithms for Category Prediction (2023)

Abstract: As one of the most extensively used languages in India, Telugu has a sizable audience and a huge library of news articles. Predicting the categories of Telugu news items not only helps with efficient organization but also makes it possible to do trend research, advertise in a certain demographic, and provide individualized recommendations. In order to identify the most effective method for accurate Telugu news category prediction, this study compares and contrasts various machine learning (ML) techniques, including support vector machines (SVM), random forests, and naive Bayes. Accuracy, precision, recall, and F1-score will be utilized as performance indicators to gauge how well these algorithms perform. The outcomes of this comparative analysis will address the particular difficulties and complexities of the Telugu language and add to the body of knowledge on news category prediction. For Telugu-speaking consumers, the study intends to improve news organization and recommendation systems, giving them more relevant and customized news consumption experiences. Our result emphasize that, although other models can be taken into account for further research and comparison, W2Vec-skip gram with polynomial SVM is the best performing combination.

## introduction news is the latest information about recent developments and events that are relevant to the general audience @xcite ). it is distributed using a variety of media and covers a broad range of themes. news serves to inform the public, encourage openness, and facilitate informed decision making. the categorization and prediction of news articles have become crucial in the quickly changing environment of information transmission for effective organization and improved user experience. the challenge of predicting news category has significantly advanced with the introduction of machine learning (ml) techniques. we compare many machines learning (ml) techniques for predicting telugu news category in this research @xcite ). one of the most widely used languages in india is telugu @xcite ), which has a large audience and an extensive library of news stories. predicting the categories of telugu news articles ( boddupalli et al. (2019) ) not only allows for effective organization but also makes it possible for trend research, advertising that is specifically targeted, and personalized suggestions. we seek to determine the most efficient strategy for precise telugu news category prediction by examining and contrasting various ml algorithms. the comparative study will include numerous kinds of machine learning (ml) techniques, such as support vector machines (svm), random forests, and naive bayes @xcite ). the effectiveness of these algorithms in correctly classifying telugu news articles will be evaluated based on their performance indicators, such as accuracy, precision, recall, and f1-score. by performing this comparative analysis, we aim to add to the body of knowledge already available on news category prediction while taking into account the special difficulties and complexities of the telugu language. the results of this study could improve how telugu news articles are stored and found, giving users access to more relevant and individualized news consumption experiences @xcite ). with the ultimate goal of increasing the effectiveness and efficiency of news organization and recommendation systems for telugu-speaking users, this study intends to shed light on the comparative analysis of various ml algorithms for telugu news category prediction. ## related works in sheth et al. (2022) a thorough comparative analysis was conducted to assess the effectiveness of various data mining classification techniques. the major goal was to evaluate the performance of the naive bayes, support vector machines (svm), de-cision trees, and k-nearest neighbor classifiers. accuracy, recall, precision, and f1 score were the evaluation criteria, and several datasets were used for the evaluation. the study's results consistently showed that, in terms of these evaluation measures, the naive bayes method performed better than the other classifiers. in comparison to the other algorithms, it regularly shows greater accuracy, recall, precision, and f1 score values. in the comparison, svm took second place, k-nearest neighbor came in third, and decision trees came in fourth as the top classifier. these findings emphasize the importance of carefully choosing the right classifier based on the unique properties of the dataset and the surrounding circumstances. it highlights the significance of avoiding a one-size-fits-all strategy and instead taking into account the particular requirements and subtleties of the current challenge. in data mining jobs, selecting the most appropriate classifier based on the particular dataset and context can produce more accurate and dependable results. in @xcite ), the authors address challenges in mining large electronic data. they focus on accuracy and the zero frequency problem. they propose using multinomial naive bayes algorithm to forecast online story popularity. python is highlighted for ai due to adaptability. while naive bayes is used in various areas, its assumption of independence is a drawback. yet, it handles many features, is simple, and offers quick training. their aim is to enhance news popularity prediction using multinomial naive bayes and discuss its pros and cons in electronic data analysis, noting python's suitability for ai. in @xcite the goal of the study is to assess word2vec convolutional neural networks' (cnns') performance in classifying news articles and tweets as related or unrelated. in particular, the study looks into how well the word embedding techniques cbow (continuous bag-of-words) and skip-gram perform while creating cnn models for classification. the study's experimental analysis shows that the use of word2vec considerably improves the classification models' accuracy. interesting results are found when the performance of the cbow and skip-gram models are compared. when applied to tweets, the skip-gram model performs better, whereas the cbow model performs better and more consistently when applied to news items. this performance disparity shows that the word embedding approach selected should be adapted to the unique characteristics of the text under study. word2vec-enabled cnn models perform better than models without word embedding. these findings help us comprehend how the choice of word embedding models affects cnn-based classification for news articles and tweets. the study emphasizes the potential of utilizing cutting-edge neural network approaches for efficient text categorization in the context of news and social media analysis by highlighting the benefits of word2vec in enhancing classification accuracy. in @xcite , the authors explore telugu news sentiment categorization using machine learning. they classify news into categories and sentiment (positive, negative, neutral). various models are compared based on accuracy, precision, recall, and f1-score. sentiment analysis's importance for telugu news, addressing regional languages like telugu, is highlighted. techniques like naive bayes, random forest, svm, among others, are used. a framework with feature selection, training, testing, and performance evaluation is presented. passive aggressive classifier stands out with 80 in most of the researches, the focus is much tilted towards the various algorithms rather than the multiple features that are associated with natural language processing, this leaves a significant gap for us to make this research a vital part. ## dataset and preprocessing we used an in-house dataset for performing the experiments in this paper. the dataset was prepared by scrapping @xcite ) an online telugu news website, gulte.com. this dataset consists of 6 classes with a total of 38637 news articles (sachin @xcite ). figure 1 shows the category distribution in the dataset. figure 2 shows the samples for each category from the dataset. the data which has been scrapped from the websites consists of several unwanted characters, white figure 2: sample news for each category from dataset spaces, html tags or some numbers. these kinds of characters have been removed from the entire dataset (varshini et al. ( 2023 )), furthermore we have also removed stop words, belonging to telugu language and also perform stemming. ## experimental setup the pre-processed dataset has been used to perform all the experiments in this paper. we trained word2vec and fasttext on the pre-processed dataset to get the word embeddings instead of using the pre-trained models. linear svm, quadratic svm, polynomial svm, random forest, knn and multinomial naive bayes models are trained for each of the feature extraction methods. for feature extraction we have used w2vec-sg, w2vec-cbow, n-gram, fast text and tf-idf. the dimension of feature vectors for n-gram and tf-idf, is 10,000 whereas for other features w2vec-sg, w2vec-cbow, and fast text it is 100.the feature visualization for vectors of such large dimensions can be done through t-sne. t-sne is a technique for revealing patterns and correlations in word vectors by visualizing them in a lowerdimensional environment. it entails gathering word vectors, using t-sne to reduce dimensionality, then presenting the results on a scatter plot. figure 3 shows t-sne visualization for tf-idf word vectors taken for 500 samples where each color repre- all the models initially take a set of parameters and train on these set of parameters to find the best fit, then the model is evaluated on the best fit parameters. at n-fold cross validation has also been performed on each of the machine learning algorithms to evaluate the algorithm's performance for unseen data. ## results and discussion refer the tables 2 to 8 for the performance metrics and cross validation score for various algorithms against popular features. after studying the performance metrics of all the algorithms, we have observed that quadratic svm for n-gram is not performing up to the mark. figure 5 shows the confusion matrix accordingly. we can observe that there is a huge misclassification for category consisting of news articles that are contained in the 'nation' class. furthermore, the number of true classifications or correct figure 5: confusion matrix for quadratic svm using n-gram figure 6 confusion matrix for word2vec-cbow for same method, quadratic svm. it is clearly observed that the model performed much better and the number of misclassifications for the category nation are very less compared to that of n-gram, we can also see that the number of true classifications for the categories others, business, entertainment is relatively more compared to ngram. this can be because word2vec model was able to generate better word vectors for the given corpus. ## conclusion word-2-vec cbow and skip gram are the two models which showed constant and reasonable performance for all the algorithms except multinomial naive bayes. the performance of fast-text was also consistent except for multinomial naive bayes but it under performed when compared to word-2-vec. n-gram showed the third best performance among the feature extraction methods, its performance was better even for multinomial naive bayes, but it showed poor performance for knn. finally, tf-idf showed reasonable performance except for polynomial svm and knn. further it has been observed that changing the parameter value, 'c' in all three types of svm, resulted in minute increase of model's accuracy. on setting the 'c' value to 10,000 the models showed up to 5%-10% increase in accuracy. in general, a higher 'c' value in svm results in higher penalty and smaller margin. it also reduces the regularization strength which may lead to overfitting, these can be some of the reasons for improved performance. the primary goal of this research has been to choose the best classifier and feature extraction pair from the most popular techniques. we have observed that w2vec-skip gram and polynomial svm is the best pair for this task. however, other models may be considered in the future work for comparison and selection.