Good Reads and Easy Novels: Readability and Literary Quality in a Corpus of US -published Fiction (2023)

Abstract: In this paper, we explore the extent to which readability contributes to the perception of literary quality as defined by two categories of variables: expert-based (e.g., Pulitzer Prize, National Book Award) and crowd-based (e.g., GoodReads, WorldCat). Based on a large corpus of modern and contemporary fiction in English, we examine the correlation of a text’s readability with its perceived literary quality, also assessing readability measures against simpler stylometric features. Our results show that readability generally correlates with popularity as measured through open platforms such as GoodReads and WorldCat but has an inverse relation with three prestigious literary awards. This points to a distinction between crowd- and expert-based judgments of literary style, as well as to a discrimination between fame and appreciation in the reception of a book.

## introduction and related works is it overall better for a novel to strive for an easy prose, or is there a link between difficulty and literary quality? the concept of readability has been studied for decades and is defined as the ease with which a text can be read and understood @xcite . several works have attempted to define an easy way to compute readability in order to make, for example, didactic books more accessible, reduce technical jargon in documents produced for the general public, and adjust text selections according to the intended audience @xcite . the result has been a series of popular and amply tested measures, each with a slight difference in their model of readability. @xcite , for example, referred to readability as the combination of elements in a text that impact important aspects of a reader's experience -including whether the reader can understand the text, finds it interesting, and can read with optimal speed @xcite . despite their shortcomings @xcite , readability measures have been broadly applied to a large number of different domains. measures of readability vary according to what aspect of a text they take into account, but they typically combine features such as sentence length, word length, and the presence of complex words. while the actual ease of a text depends on reader characteristics (background, situation, ability) it is widely accepted that simple textual features such as sentence length, syllables per word and lexical diversity impact the reading experience @xcite . the connection of readability to the quality of a text has often been implied when it comes to non-fiction, and early studies into readability attest to the educational and social importance of developing such measures to improve technical or expository documents @xcite , but its role in the quality of literary fiction is much more complex. an easy-to-read novel can be enjoyable to read, but may also apppear poor or uno-figure 1 : correlations between stylometrics and flavours of readability @xcite . all correlations between 0.09 and 0.99 are statistically significant. riginal. in literary studies, the idea that readability might be a precondition for literary success is debated, and literary texts have been assessed variously by readability measures and similar metrics. sherman (1893) was one of the first scholars to propose certain values of average sentencelength and reading ease as properties of "better" literary style. readability naturally varies across genre, but it is a widespread conception for readers and publishers alike that bestsellers (as defined by top book-sales) are easier to read @xcite . more recently, readability has gained traction in areas of (commercial) creative writing and publishing, especially where its measures are implemented in text-editing tools such as the hemingway or marlowe editors foot_0 . these applications tend to favour lower readability scores -which is, texts easier to read. yet, on the large scale, few studies have included readability as a measure that could help predicting literary quality. studying a small corpus of bestsellers and more literary, canonical works, @xcite found no significant difference in readability, using a modified flesch reading score, while @xcite found differences in readability between bestsellers and commercially endorsed book-list titles. relying on multiple measures of readability and one measure of literary quality (i.e., goodreads' average ratings), maharjan et al. @xcite found that readability was actually a weak measure for estimating popularity in comparison to, for example, character ngrams. still, many studies of literary success, popularity, or perceived literary quality have sought to approximate text complexity and have studied textual properties upon which formulae of readability are directly or indirectly based, such as sentencelength, vocabulary richness, or text compressibility @xcite @xcite . the question of the role of readability in literary quality is complicated by the practical and conceptual problem of defining literary quality itself, and consequently of quantifying it for large scale studies. studies that seek to predict perceived literary quality from textual features often rely on the provisional proxy of one single gold standard, such as book-ratings from large user-platforms like goodreads @xcite , personally or institutionally compiled canons @xcite or sales-numbers @xcite . however, it has been shown that readers may have different, distinct perceptions of quality that are not necessarily based on the same criteria or prompted by the same textual features @xcite . in this paper, we explore to what extent readability might contribute to the perception of literary quality -defined through several alternative measures -in a large fiction corpus of modern and contemporary novels in english, taking into account, instead of one golden standard, different contextual perspectives on literary quality, so as to cover both crowd-based and "expert"-based standards of judgment. ## data and methods the essence of our approach consists in examining whether readability, as measured through five different algorithms, and literary quality, as approximated through six different resources, show any correlation on a large corpus of english-language fiction. we use standard correlation measures (pearson and spearman product-moment correlation coefficients r p and r s , respectively). for inference on the correlation measures, simple student's t-tests are used. for robustness checks, correlation coefficients were also modelled using a bayesian ridge model of standardized the variables -al-though not reported due to limited space. foot_1 ## results pearson's and spearman's correlations between these five readability metrics and commonly used stylometric features show -as a sanity check -that readability measures capture aspects of novels' overall style. all measures are similarly correlated to sentence-length (naturally, being a base for all measures) but also to lexical diversity and compressibility, which measure, respectively, complexity at the word-and sequence-level. moreover, the correlations with our "quality scores" show that readability is linked with the ones closer to popularity than to appreciation. pearson's r, specifically in its significance testing, relies on the assumption of normally distributed data and it assumes that the two variables have a linear relationship, while spearman's r correlation coefficient is non-parametric, meaning that, while it still assumes a monotonic relation between the two variables, it does not make strong assumptions on the shape of the data. for this reason, spearman is probably the best overall measure for this study, as we have no reason to assume that all our measures are normally distributed (and some are evidently not, as can be seen in figure 2 ). for these reasons, we will mainly credit the correlations observed through spearman's r, although we report both in 2. ## conclusions and future works readability measures proved significantly consistent, both between each other and with other relevant stylometric features, when applied on mod-ern and contemporary fiction. their relation with different proxies of literary quality is intriguing: more popular works, in terms of number of ratings on goodreads and in terms of libraries willing to hold a copy of the book, appear to have a correlation with readability, while the appreciation of readers alone (independently from their number) seems to hold almost no link with it, and longlisted titles have an inverse relation with readability, tending to prefer slightly more difficult prose on the readability metrics' scale. it can be argued that we are seeing the divide between high-brow and "popular" literature, but the lack of correlation with goodreads average rating might point to a slightly more nuanced conclusion. it is worth noting that the only measure showing a meaningful correlation with all of the crowd-based quality metrics was the new dale-chall measure of readability, also the only one explicitly focusing on the presence of widely understood lexicon in a text, but it was also the only one showing no significant difference between long-listed and non long-listed titles. the only other measure having a correlation higher than 0.1 with average goodreads' ratings was smog, which, while not using a list of hard words, considers "difficult words" in its own way in its computation, using the number of polysyllable words as a central element. if we were to draw rough conclusions from these observations, it would seem that surface-level simplicity of style in terms of words per sentence, characters per words, and similar metrics "helps" a text's popularity, but has nothing to do with its likelihood of being highly liked by its readers -and it even slightly hinders its possibilities of receiving a prestigious awards. in other words, surface-level simplicity improves a text's quality only if we equate it with popularity or fame. similarly, looking at threshold-based probability distributions showed that indeed increasing the difficulty of the novels' style might hinder its diffusion across libraries and goodreads' users. using a more common vocabulary might also increase readers' appreciation of the text, but only when it comes to crowd-based measures. on the other hand, the correlations of average number of ratings and library holdings with readability measures do not appear linear or monotonic, meaning that there might also be a "point of balance" between too easy and too difficult, that maximizes the correlation with a novel's fame. the same might be true for the likelihood of a novel being long-listed for one of the three awards we took into consideration. overall, readability seems to have an impact on different perceptions of literary quality, although its role and interaction with other features of the text remains to be defined. another overarching point to observe from these findings is that there is a difference between crowd-based (goodreads) and expert-based (awards) standards of literary quality in readability-level preference, which indicates that the criteria change across different quality-judgements, which suggetss that "literary quality" cannot be quantified reliably if it is reduced to a single golden standard. further research points towards extending the set of correlations to more proxies of quality as well as more sophisticated stylometric measures to see whether interactions can provide a clearer picture of what we perceive as literary quality. other further work could be to check the correlations of our measures with publication date: readability might depend on time, either in the sense of the evolution of the average novelistic style, overall language change, or even cultural selection, which would make the passage of time a particular form of "quality test" of its own accord.