A Study on Using Semantic Word Associations to Predict the Success of a Novel (2021)

Abstract: Many new books get published every year, and only a fraction of them become popular among the readers. So the prediction of a book success can be a very useful parameter for publishers to make a reliable decision. This article presents the study of semantic word associations using the word embedding of book content for a set of Roget’s thesaurus concepts for book success prediction. In this work, we discuss the method to represent a book as a spectrum of concepts based on the association score between its content embedding and a global embedding (i.e. fastText) for a set of semantically linked word clusters. We show that the semantic word associations outperform the previous methods for book success prediction. In addition, we present that semantic word associations also provide better results than using features like the frequency of word groups in Roget’s thesaurus, LIWC (a popular tool for linguistic inquiry and word count), NRC (word association emotion lexicon), and part of speech (PoS). Our study reports that concept associations based on Roget’s Thesaurus using word embedding of individual novel resulted in the state-of-the-art performance of 0.89 average weighted F1-score for book success prediction. Finally, we present a set of dominant themes that contribute towards the popularity of a book for a specific genre.

## introduction every year a lot of literary fictions get published and only a few of them achieve the popularity. so it is very important to be able to predict the success of a book before the publisher commits a significant effort and resources for it. many factors contribute to the success of a book. the story, plot, and character development, all have specific role in the popularity of a book. there are some other factors like the time when the book has been published, the author's reputation, the marketing strategy, etc that may also influence a book's popularity. in this paper, we only focus on understanding a set of concepts' associations extracted from the content of the book to predict its success. according to the theory of word embedding, the vector representation of a word in the embedding space captures its semantic relationship with other words based on co-occurrence in the corpus. @xcite , and @xcite developed methods for detecting the statistically significant linguistic change using word embedding. in the meantime, @xcite developed the concepts of word embedding association test (weat) to uncover the gender bias and ethnicity bias. following these studies, @xcite , and @xcite used 100 years of text data and demonstrated that word embedding can be used as a powerful tool to quantify historical trends and social change. for every time period, they warp the vector spaces into one unified coordinate system and construct a distance-based distributional time series for each word to track its linguistic displacement over time. our idea is to use the associations of different semantically linked word groups or concepts in a book and investigate how its impact on book success prediction. in this article, we study the efficacy of word associations to represent literature as a spectrum of individually organized concepts as a set of connoted words in the popular roget's thesaurus @xcite . we represent word association as the euclidean distance between two words in the embedding space. to find the association of book content to a set of concepts, we compute the average euclidean distance for each set of semantically linked word vectors of a book's normalized embedding space to the respective word representation in the global embedding space. the concept of word embedding normalization and the word association score has been used successfully in many recent research works for computing the gender associations @xcite . in figure 1 , we show word associations of prominent themes for a successful book the prince and the pauper having goodreads-@xmath0.5, and an unsuccessful book the house of the seven gables with a goodreads-@xmath1.5. we observe that the average association score of each theme vary between these two books. we analyze the impact of these associations score for the success of each book, and obtain a set of dominant concepts that play an important role for a book success. in this paper, we include following research contributions: ## related work in the earlier work, ashok et al. (2013) used stylistic approaches, such as unigram, bigram, distribution of the part-of-speech, grammatical rules, constituents, sentiment, and connotation as features and used liblinear svm @xcite for the classification task. they used books from total 8 genres, and they were able to achieve an average accuracy of 73.50% for all the genres. van cranenburgh and koolen (2015) distinguished highly literary works from less literary works using textual features e.g. bigram. @xcite ; @xcite worked on emotion along with the book for success prediction. maharjan et al. ( 2017 ) used a set of hand-crafted features in combination with recurrent neural network and generated feature representation to predict the success, and obtained an average accuracy of 73.50% for the 8 genres. they also performed several experiments, including using all the features from @xcite , sentiment concept @xcite , different readability metrics, doc2vec @xcite representation of a book, and unaligned word2vec @xcite model of the book. in a more recent work by maharjan et al. (2018a), they used the flow of the emotions across the book for success prediction and obtained an f1-score of 69%. they divided the book into some chunks, counted the frequency of emotional associations for each word using the nrc emotion lexicon (mohammad and turney, 2013), and used a recurrent neural network with an attention mechanism to predict both the genre and the success. @xcite ; jarmasz (2012) showed that roget's has turned out to be an excellent resource for measuring semantic similarity and the words in roget's word clusters have higher correlation than many other prominent word groups e.g., @xcite . @xcite used svm weights for assigning ranks in the feature selection process. they verified that the topranked genes found by svm have biological relevance to cancer and the svm classifier with svm selected features worked better than other classifiers in determining the relevant features along with the classification task. ## dataset in this study, we use the dataset introduced by @xcite , a publicly available dataset genre unsuccessful successful total detective mystery 60 46 106 drama 29 70 99 fiction 30 81 111 historical fiction 16 65 81 love stories 20 60 80 poetry 23 158 181 science fiction 48 39 87 short stories 123 135 258 total 349 654 1,003 ## methodology in order to predict the success of a book, one of our major research questions was how we can represent a book properly. we explored a wide range of feature sets and performed multiple experiments in order to find the most suitable feature set that can represent the concept, emotion and writing style of a book. in this section, we discuss the relevant methods that we used for the study of book success prediction. ## conclusion and future work we present a novel study of word association of book content to predict the success of book and show that semantic word association features can be new vertical of the classification based task. our empirical results demonstrate that word association and different types of concepts can be very useful to capture the book's literary content and can predict the book success with better accuracy. rather than individual word frequency, the set of words with similar concepts has been proved to be more effective. we will continue our research work in this area and we intend to perform the experiments on a bigger data set in the future. we hypothesize that instead of preparing word embedding for individual book, we can retrain the global embedding using genre-wise data. this genre specialized embedding can help us to obtain a much better result for two reasons -as each embedding will be retrained on individual genre, the quality of generated embedding is expected to be better and it will represent the genre specific context for each word more explicitly. 2. train a l2-loss linear svm model using the best c.