Automatic Authorship Classification for G erman Lyrics Using Naïve B ayes (2023)

Abstract: Text classification is a prevalent and essential machine-learning task. Machine learning classifiers have developed immensely since their inception. The naïve Bayes classifier is one of the most prominent supervised machine learning classifiers. In this experiment, we highlight the performance of Naïve Bayes for classifying of authors/artists on the German lyrics corpus (“Songkorpus”) and compare the classification results with other classifier algorithms. The corpus of investigation consists of six artists with 970 songs in total. Bayes model evaluation measures revealed a precision of 0.91, recall of 0.94, and F1-measure of 0.9. Furthermore, the classification performance with other classifier algorithms did not reveal any statistically significant difference in performance. The results of the study add to the high volume of reports on the classification accuracy of Naive Bayes for the task of lyrical classification.

## theoretical framework of naïve bayes a naive bayes classifier is a type of probabilistic classification mechanism based on the bayesian theorem, a posthumous theory by thomas bayes @xcite . derived from the concepts of inferential statistics, bayes' theorem serves as the basis for multiple machine learning models. the theorem is based on the logical probability of an event occurring concerning other events or features @xcite . equation ( 1 ## experiments we conduct a series of experiments focusing on authorship attribution for song lyrics of udo lindenberg, konstantin wecker, stoppok, ulla meinecke, hannes wader, and fettes brot. the results of naïve bayes and other classification approaches are contrasted. the first half of the experiment was focused on applying the naïve bayes classifier to the songkorpus dataset. the publicly available repository provides detailed corpus statistics, as well as visualizations on character, word, verse, song and corpus level. our first step was data gathering and pre-processing from the songkorpus website. the data came in xml format and was further transformed to plain text. the processed text included semantic and structural information related to the artist and the song verses. all additional information was omitted. the plain textual data was subject to further linguistic analysis, using the profiling ud tool @xcite all experiments were conducted on a system with an intel corei7 cpu at 2.4ghz, 8 gb of ram, and 1 tb of secondary storage, running windows 10 and matlab 2021b. the dataset was loaded onto the machine learning toolkit of matlab software for further processing. the utility of matlab's machine learning toolkit was due to the capability of comparing multiple classifiers in one place and also due to its ease of implementation. the classical naïve bayes classifier was designed with the preset features for classification. the developed dataset was split into a training and testing dataset. two-thirds of the dataset was used to train the classifier and the rest of the data was used for its testing and validation. the split of the dataset was randomized to eliminate any artist bias. the parameters were tweaked and multiple runs were carried out to find the best possible classification accuracy. in the second stage of the experiment, the classification accuracy of the naïve bayes classifier was compared with other commonly used classifiers, such as logistic regression (lr), support vector machines (svm), naive bayes (nb), decision tree classifier (dtc), k-nearest neighbor (knn), and neural networks (nns). to improve the classification accuracy, hyper-parameter tuning and dimension reduction were employed. additionally, multiple iterations of the classifier parameters were run by removing correlated features, using log probabilities in calculations, and parallelized calculations were performed. these additional steps were used in order to identify the best possible classification results. the performance of each classifier is compared in the next section. ## results and discussion when measuring performance of the naive bayes classifier, various iterations were carried out by implementing the laplace smoothing, with the classifier's accuracy being the best at the estimator's value of 0.06. figures 3 & 4 display the confusion matrix and roc, respectively, of the best naïve bayes classifier. in figure 4 , the roc curve of the classifier shows that the area under the curve (auc) was 0.89, which is a very good classifier performance. ## conclusion creating a meaningful lyrics dataset is a tedious and time-consuming task. for example, guest appearances by other artists or two versions of the same song (e.g. studio version and live version) must be handled with care. by using the precompiled songkorpus, we empirically tested the accuracy of different authorship classificators on a reliable dataset. the results of our best model seems promising and are in accordance with comparable research reports on naïve bayes classifiers @xcite @xcite @xcite @xcite . it can be concluded from our experiments that the naive bayes classifier seems to be a good choice for authorship attribution of song lyrics, at least for the investigated singer-songwriter dataset. since the used dataset is relatively small, it would be a reasonable choice to use our classifiers on bigger datasets in order to make better generalizations.