Stylometric Literariness Classification: the Case of Stephen King (2021)

Abstract: This paper applies stylometry to quantify the literariness of 73 novels and novellas by American author Stephen King, chosen as an extraordinary case of a writer who has been dubbed both “high” and “low” in literariness in critical reception. We operationalize literariness using a measure of stylistic distance (Cosine Delta) based on the 1000 most frequent words in two bespoke comparison corpora used as proxies for literariness: one of popular genre fiction, another of National Book Award-winning authors. We report that a supervised model is highly effective in distinguishing the two categories, with 94.6% macro average in a binary classification. We define two subsets of texts by King—“high” and “low” literariness works as suggested by critics and ourselves—and find that a predictive model does identify King’s Dark Tower series and novels such as Dolores Claiborne as among his most “literary” texts, consistent with critical reception, which has also ascribed postmodern qualities to the Dark Tower novels. Our results demonstrate the efficacy of Cosine Delta-based stylometry in quantifying the literariness of texts, while also highlighting the methodological challenges of literariness, especially in the case of Stephen King. The code and data to reproduce our results are available at https://github.com/andreasvc/kinglit

## introduction stephen king, as both a highly successful and critically acclaimed author, has been referred to as "a contradiction in the literary world, a genre novelist who has achieved both popular and critical success," @xcite . in this paper we investigate the literariness of king's corpus using data-driven stylometric methods. stylometry is often operationalized using distances in vector space based on the most frequent words (mfw) in texts. stylometry was canonically developed for questions of authorship attribution, but it has increasingly been applied to other research questions, including author profiling (e.g. age and gender), and stylochronometry, or the order in which an author wrote their works @xcite . in this paper we propose to further the quantification of literariness as an additional application of stylometry. the term literariness was introduced by the russian formalist @xcite , although we follow a more general definition: "the sum of special linguistic and formal properties that distinguish literary texts from non-literary texts" @xcite . recent work has approached literariness with data-driven methods. the riddle of literary quality project @xcite @xcite @xcite @xcite focuses on predicting reader judgments of literary quality @xcite using textual features. van cranenburgh and @xcite report that machine learning solely using bigrams "can successfully separate novels that are seen as highly literary from less literary ones." moreover, content and style bigrams achieve comparable accuracy, indicating that literariness is reflected in both thematics and stylistics of texts. @xcite report that an ensemble of lexical, syntactic, and text complexity features "contribute to explaining judgments of literature," while individual syntactic constructions and individual bigrams are weak predictors of literariness, as "the top 100 features contribute only 34% of the total [literariness] prediction." @xcite report that even with short fragments of several pages, literariness can be predicted effectively; a keyword analysis reveals that literariness is associated with particular stylistic differences, such as strong and weak pronouns. @xcite present another study of contemporary fiction and focus on the distinctions between prize-winning, bestselling, and genre fiction in american novels, employing liwc fea- the studies above demonstrate that literariness is readily reflected in textual features, but these studies worked with sizeable datasets and reliable indicators of literariness. but what if literariness itself is in question in the texts? what if there is only limited data available, as in the case of a single author? moreover, in the absence of consensus on literariness based on reader surveys or literary award juries, we must contend with a "ground truth" that is potentially more subjective and unreliable. we propose to build upon these studies by exploring stylometric methods for quantifying the literariness of works by a single author, and, again, not just any author, but an author known for occupying an exceptional position across high and low literariness in fiction. since our data and task are challenging, we opt for a proven stylometric method: cosine delta @xcite , a variant of burrow's delta @xcite . while this method has been especially successful for authorship attribution, we expect it to be effective for stylistic aspects in general, of which literariness is an instance. ## materials and methods king novels and novellas (73 texts). the main corpus contains all fiction solely authored by stephen king published through 2020. of these texts, 17 are categorized as high literary ( we contrast the king corpus with two comparison corpora: literary fiction (38 novels). inclusion criteria include: national book award winning authors published from 1974-2020, and, to attempt to control for diachronic change, american authors roughly the same age (within 10 years) as @xcite . for each author, we select the novel which won the award, and an arbitrarily chosen second novel by the same author, to increase the corpus size. no effort was made to control for genre in this corpus, but this could be considered in future work. 1 popular fiction (36 novels). inclusion criteria include: new york times bestsellers (top 10 only) published from 1974-2020, american authors near the same age (within 10 years) as stephen king, in genres king himself has written in: thriller and/or horror. we select two novels for each author. 2 1 literary fiction corpus: thomas pynchon, gravity's @xcite . tim @xcite . john irving, the world according to @xcite . alice walker, the color purple @xcite . larry heinemann, paco's @xcite . pete dexter, paris trout (1988). john casey, @xcite the fundamental issue with using this corpus as a proxy for low literariness is whether the identification of "popular" "genre" fiction with "low literariness" is valid, and whether the distinctions between high and low fiction as described by e.g., @xcite as "the great divide" are as great in recent fictional texts. we acknowledge this issue but, in the absence of e.g. large datasets of reader surveys on the literariness of english-language novels, choose this as the best option. ## conclusion and future work our results confirm that popular fiction and high literary novels can be distinguished remarkably effectively, as shown by the classification f-scores. applying the same model to the king corpus gives promising results, but the classification f-scores are not as impressive. on the one hand, this is entirely expected: the model is evaluated on a different domain than what it was trained on, and some of the patterns it exploits in the training data will not apply. on the other hand, there may be a deeper problem; as many of king's works display aspects of high and low literariness, in both thematics and perhaps style, king may be an exceptionally problematic case in any quantitative measurement of literariness. moreover, the lower performance on the set of labeled king texts may indicate that more training data and more sophisticated models are needed, but it may also be a sign that the categorizations of the king novels are not consistent with the textual features or the categorizations of the popular and high literary novels in the comparison corpora. what is taken as a misclassification by the model in our evaluation could also point to a misclassification by critics; moreover, since our labels reflect the perspective of different individuals, they are not necessarily consistent with each other; these critics may have applied different standards. these are methodological challenges that should be addressed in future work. robust evidence emerging from the experiments on king's work is the consistent ranking by the models of novels in king's dark tower series and other texts as being among his more literary works, consistent with critical assessments. this shows promise for cosine delta and other stylometric methods in literariness quantification. while the question of king's literariness is not settled, our results confirm the notion that king's work spans the spectrum of low and high literariness. we also leave for future work the case of literariness and richard bachman, a pen name used by king in early novels and two later works. despite the nom de plume's practical origins (king and his publishers were wary of publishing two novels in one year by the same author), king later described bachman as a "persona" and "alter ego," stating that the early novels had been written "in a bachman state of mind: low rage, sexual frustration, crazy good humor, and simmering despair" @xcite . we suggest that an exploration of king/bachman would merit a dedicated, mixed-method study, particularly as there is far from critical consensus on the literariness of the various bachman books, individually or collectively, let alone whether the bachman novels (some separated by decades) can convincingly be argued to share distinctive features. the running man, a bachman book, was scored by our model as the lowest king novel in literariness, and while a hypothetical critic might dismiss that novel as a low-literary dystopian thriller, the book has many literary aspects, as discussed by @xcite . meanwhile, the long walk, the second bachman book, scored relatively highly in literariness in our model, consistent with our own critical assessment of the novel as a fine dystopian allegory of the vietnam war. is bachman a signal, or merely noise?