Fractality of sentiment arcs for literary quality assessment: The case of Nobel laureates (2022)

Abstract: In the few works that have used NLP to study literary quality, sentiment and emotion analysis have often been considered valuable sources of information. At the same time, the idea that the nature and polarity of the sentiments expressed by a novel might have something to do with its perceived quality seems limited at best. In this paper, we argue that the fractality of narratives, specifically the long-term memory of their sentiment arcs, rather than their simple shape or average valence, might play an important role in the perception of literary quality by a human audience. In particular, we argue that such measure can help distinguish Nobel-winning writers from control groups in a recent corpus of English language novels. To test this hypothesis, we present the results from two studies: (i) a probability distribution test, where we compute the probability of seeing a title from a Nobel laureate at different levels of arc fractality; (ii) a classification test, where we use several machine learning algorithms to measure the predictive power of both sentiment arcs and their fractality measure. Our findings seem to indicate that despite the competitive and complex nature of the task, the populations of Nobel and non-Nobel laureates seem to behave differently and can to some extent be told apart by a classifier.

## introduction the question of what defines the perception of quality in literature is probably as old as narrative itself, but the ability to process and analyze large quantities of literary texts, and to perform complex statistical experiments on them @xcite , has recently made new ways of studying this question possible. this does not mean that the riddle has become easy at all: first of all, studying literary quality with methods from corpus linguistics means that one has to create a dataset of "high quality" texts, usually to contrast against "lower quality" texts; second, while it is possible to analyze a larger number of texts in a shorter time, we need to know where to look to find possible, nonobvious correlations with the perception of quality. recently, a series of studies have looked into the possibility of correlating some fractal properties of a text -the degree of fractality of its sentences' length, sentiment arc, or succession of topics -with its literary quality. these studies have been using as a proxy to define the quality of a text either canons defined by a single scholar, or majority-vote measures taken by large reader platforms, where the aggregated score given by a large number of readers is used as the value of the book, often with a threshold to transform it into a binary problem. other similar works have used the number of sales of a book to approximate its "quality". in this work, we try to use a perhaps more daring, less explored metric to define quality: we apply an already tested measure: the fractality of the sentiment arc of a text, which is the curve that represents the changes in sentiment throughout the text. we compute this metric for a group of texts written by authors who won the nobel prize for literature, and we ask whether this simple measure can help tell such texts from a highly competitive control group. despite the difficulty of the task -in the best cases, nobel prizes are assigned to only one among many valid competitors, which means that several high quality writers will fall in the negative class -our results seem to indicate that a weak but reliable signal is present, and that it can be exploited by classic machine learning algorithms to predict whether a narrative's arc belongs to a nobel laureate or not. the paper is organized as follows: in section 2, we describe some of the most relevant related works in sentiment analysis and fractal theory for studies in literary quality. in section 3 we present the corpus and discuss the idea of using nobel prize winners; section 4 gives a detailed overview of the concept of series fractality for sentiment arcs. finally, section 5 details the settings of our experiments and section 6 presents our main results. in section 7 we discuss our conclusions and possible future works. ## a dataset of nobel literature the first problem in determining the relationship between sentiment arcs and literary quality is finding a metric for literary quality itself; and it could be argued that the problem of finding a reliable source of quality judgments is the same that every individual reader has when faced with an amount of literature too large to read and evaluate alone @xcite ous works attempting to classify or measure some aspect of literary quality have relied on quantitative metrics such as number of copies sold or average reviews on large scale platforms -or even in newspapers -few attempts have been made, to the best of our knowledge, to use high prestige literary awards as a central metrics to approximate the quality of a work. in this paper we try to use arguably the most prestigious international literary award, the nobel prize for literature, as our sole guide to select literary quality. naturally, this setting is a deliberate extremization: no literary award can possibly be considered the unique indicator of what literary quality is, and questions on the sensibility of the nobel committee's choices, both in terms of who got the prize and in terms of who did not receive it, rise almost every year. at the same time, high level literary prizes can work as imperfect guidelines for one kind of quality, and it would be interesting to find out whether, on a larger scale than single books or authors, a "signal" telling nobel-winning texts from a control population can be found. unfortunately, no comprehensive corpus of nobel-winning authors exists to date. to carry out our experiment, we used a recent corpus of literary texts, the chicago corpus, compiled by hoyt long and richard jean so, composed of 9089 novels published in the us between 1880 and 2000. the corpus contains key works of us nobel laureates, seminal works from mainstream literature as well as relevant works in genres such as mystery and science fiction @xcite . 1 the us nobel laureates in the corpus make the relative majority of the group of nobel laureates, e.g. john galsworthy, sinclair lewis, william faulkner, ernest hemingway, john steinbeck, saul bellow and toni morrison. works by non-us writers like for example knut hamsun, samuel beckett and nadine gordimer are represented with a more 1 several quantitative literary studies have used the corpus @xcite , which can be found at https://textual-optics-lab. uchicago.edu/us_novel_corpus . as noted, the corpus is highly curated and contains high quality fiction from authors who have received other prizes, like the national book award, e.g. don delillo, joyce carol oates, and philip roth. our expectation is therefore not that nobel laureates will be completely different from the rest of the corpus, also in terms of literary quality. finally it is worth noting that the whole corpus is heavily skewed towards the anglosaxon literature, and that both the nobel laureates and their control group are mainly constituted by anglophone writers. this naturally moves the whole contest on the plain of a well refined "anglo-centric" canon. while it does not damage our experiments per se, given that the same imbalance happens among the nobel laureates as among the remaining writers, it is a distortion that we have to keep in mind. ## fractality of sentiment arcs to estimate the long-term memory of sentiment arcs we combine non-linear adaptive filtering with fractal analysis, specifically adaptive fractal analysis @xcite . nonlinear adaptive filtering is used because of the inherent noisiness of story arcs. first, the signal is partitioned into segments (or windows) of length @xmath0, where neighboring segments overlap by n + 1. the time scale is n + 1 points, which ensures symmetry. then, for each segment, a polynomial of order d is fitted. note that @xmath1, and @xmath2. the fitted polynomial for ith and (i + 1)th is denoted as y (i) (l 1 ), y (i+1) (l 2 ), where l 1 , l 2 = 1, 2, ..., 2n + 1. note the length of the last segment may be shorter than w. we use the following weights for the overlap of two segments. we use the hurst exponent to measure long-term memory. assuming that stochastic process @xmath3: @xmath4, 1, 2, ..., with stable covariance, mean µ and σ 2 , the process' autocorrelation function for r(k), @xmath5: detrended fluctuation analysis (dfa) is the most widely used method for estimating the hurst parameter, but dfa may involve discontinuities at the boundaries of adjacent segments. such discontinuities can be detrimental when the data contain trends @xcite , non-stationarity @xcite , or nonlinear oscillatory components @xcite . adaptive fractal analysis is a more robust alternative to dfa @xcite . afa consists of the following steps: first, the original process is transformed to a random walk process through firstorder integration u(n) = n @xmath6, @xmath7, 2, 3, ..., n , where x is the mean of x(k). second, we extract the global trend (v(i), @xmath8, 2, 3, ..., n ) through the nonlinear adaptive filtering. the residuals (u(i) -v(i)) reflect the fluctuations around a global trend. we obtain the hurst parameter by estimating the slope of the linear fit between the residuals' standard deviation f (2) (w) and w window size as follows: while vader is not the most recent sentiment analysis model, we chose it for its transparency, since it is possible to reconstruct the reasons of its judgments based on its systems of rules, as well as its popularity, as its underlying dictionary and set of rules has proven the weapon of choice for a large number of previous works. the sentiment arc is obtained by first computing the sentiment of each word in the text, and then by computing the average sentiment of each sentence. the sentiment of a word is in turn obtained using an ad-hoc lexicon, which links a sentiment score to each word and takes care of morphological variations. the sentiment of a sentence is then computed as the average of the sentiment scores of all the words in that sentence, by taking care of tricky structures like negations, intensifiers and so forth. ## experiments we present the results for two experiments: in both cases, we decided to design the non-nobelwinning class (or control group) in order to be as contextual to the nobel population as possible: for each book belonging to an author who won the nobel prize, we took all novels published between one year before and one year after its publication date, and we considered them as the "control group" for that book. all the control groups for all books of one author work as the control group for that author, and all control groups together combine into the overall control group for the nobel prize population. we did this also to mimic as much as possible the logic of the prize itself, that selects between contemporary candidates. a detailed summary of this selection process can be seen in ## discussion and conclusions in this paper we have tried to use a measure of fractality for sentiment arcs to distinguish nobelwinning writers in a corpus of selected literary texts in the english language, as a case for the relevance of this metric in literary quality evaluation. we are not interested in the overall valence of a literary work as such, but in its patterns of variation and repetition throughout the narrative arc, although the underlying argument for using sentiment analysis (and not just, for example, pos tagging) is that it can be linked to the evocation of emotions in the work. even if it is far from catching the expressions of emotions perfectly, as there are many way to express them, also through words with a neutral sentiment, we believe it remains a strong indicator of the work's rhetoric appeal structure. overall, the best attitude towards this kind of metric is probably similar to the attitude we can have towards the aesthetic properties of fractals in music or visual arts: it is never necessary for a work of art to contain anything fractal, but on the large scale we could expect fractal patterns to hold a correlation with the perception of beauty. in the same way, we should not imagine a systematic relationship between quality and a given range of hurst exponents: first of all because there is no single way to measure literary quality, and second because a "good" hurst exponent can hardly be the single factor in high quality textual narrative. nonetheless, we have found that the distribution of hurst exponents, as computed on the sentiment arcs of whole novels, for the titles of authors who won a literature nobel prize is different from the distribution of hurst exponents for the titles of the control group. this is particularly relevant considering that the control group still included several high-level writers, from nabokov to woolf, who can be said to rival the nobel population in terms of both fame and critical acclaim. what this difference in distribution seems to indicate is that there might be a "sweet spot" of self-similarity in sentiment arcs, roughly between 0.53 and 0.61, where the probability of seeing a text from a nobel laureate grows and the probability of seeing a title from a non-nobel laureate decreases. following on this finding, we tried to create a classifier that would tell whether a text came from a nobel laureate or not based on its hurst exponent and a representation of its sentiment arc only. what we found is that when we control for data imbalance by using near miss subsampling or smote upsampling, classi-fiers appear to perform well above chance, while if we subsample randomly their performance suffers considerably. we consider this a indication that a "signal" for nobel laureates exists, despite the highly competitive control group, and that it falls in line with previous studies on the hurst exponent for sentiment arcs.