Idiosyncratic but not Arbitrary: Learning Idiolects in Online Registers Reveals Distinctive yet Consistent Individual Styles (2021)

Abstract: An individual’s variation in writing style is often a function of both social and personal attributes. While structured social variation has been extensively studied, e.g., gender based variation, far less is known about how to characterize individual styles due to their idiosyncratic nature. We introduce a new approach to studying idiolects through a massive cross-author comparison to identify and encode stylistic features. The neural model achieves strong performance at authorship identification on short texts and through an analogy-based probing task, showing that the learned representations exhibit surprising regularities that encode qualitative and quantitative shifts of idiolectal styles. Through text perturbation, we quantify the relative contributions of different linguistic elements to idiolectal variation. Furthermore, we provide a description of idiolects through measuring inter- and intra-author variation, showing that variation in idiolects is often distinctive yet consistent.

## introduction linguistic identities manifest through ubiquitous language variation. the notion that language functions as stylistic resources for the construction and performance of social identity rests upon two theoretical constructs: sociolect and idiolect @xcite . the term 'sociolect' refers to the socially structured variation at the group level, whereas 'idiolect' denotes language variation associated with individuals @xcite . variationist sociolinguistics emphasizes the systematic variation of sociolect such as gender, ethnicity, and socioeconomic stratification @xcite . while a central concept in sociolinguistics, idiolect has received far more research attention in forensic linguistics @xcite . although idiolects have played a central role in stylometry and forensic linguistics, which seek to quantify and characterize individual textual features to separate authors @xcite @xcite , the theory of idiolect remains comparatively underdeveloped @xcite ). an in-depth understanding of the nature and the variation of idiolect not only sheds light on the theoretical discussion of language variation but also aids practical and forensic applications of linguistic science. here, we characterize the idiolectal variation of linguistic styles through a computational analysis of large-scale short online texts. specifically, we ask the following questions: 1) to what extent can we extract distinct styles from short texts, even for unseen authors; 2) what are the core stylistic dimensions along which individuals vary; and 3) to what extent are idiolects consistent and distinctive? by using deep metric learning, we show that idiolect is, in fact, systematic and can be quantified separately from sociolect. and we introduce a new set of probing tasks for testing the relative contributions of different linguistic variations to idiolectal styles. secondly, we found that the learned representations for idiolect also encode some stylistic dimensions with surprising regularity (see figure 1 ), analogous to linguistic regularity found in the word embeddings. thirdly, with our proposed metrics for style distinctiveness and consistency, we show that individuals vary considerably in their internal consistency and distinctiveness in idiolect, which has implications for the limits of authorship recognition and the practice of forensic linguistics. for replication, we make our code available at https: //github.com/lingjzhu/idiolect . ## idiolectal variation figure 1 : compositionality in stylistic embeddings encoded by sroberta, projected into the first two principal components. the other theoretical question relevant to cognitive science as well as forensic linguistics is to what extent an individual's idiolect is distinctive and consistent against a background population @xcite . studies on forensic linguistics @xcite @xcite provide confirmatory answers to both questions-yet these studies only focus on a specific set of features for a small group of authors. at the other extreme, the high performance of applying machine learning on authorship verification and attribution @xcite @xcite stems from placing more emphasis on separating authors (distinctiveness) than consistency. as an empirical investigation of these two concepts in a relatively large population remains to be conducted. here we aim to quantify these two linguistic constructs in large-scale textual datasets. ## learning representations of idiolects in forensic linguistics, textual similarity was traditionally quantified as the proportion of shared vocabulary and the number and length of shared phrases or characters (n-grams) @xcite . more sophisticated statistical methods to perform textual comparison have been developed over the years @xcite . to learn representations of idiolectal style, we propose using a proxy task of authorship verification, where, given two input texts, a model must determine if they were written by the same author or not @xcite . the identification is performed by scoring the two texts under comparison with a linguistic similarity measure and, if their linguistic similarity measure exceeds a certain threshold, the two texts are judged to be written by the same author. ## results on proxy task to test whether our model does recover stylistic features, we first test its performance on the proxy task of author verification, contextualizing the performance with other models specifically designed for that task. the performance of authorship verification is evaluated by accuracy, f1 and the area-under-the-curve score (auc) @xcite . ## linguistic analysis in this section, we seek to quantify the idiolectal variation at different linguistic levels. ## characterizing idiolectal styles in this section, we turn our attention to distinctiveness and consistency in writing styles, both of which are key theoretical assumptions in forensic linguistics @xcite . distinctiveness we examine inter-author variation through inter-author distinctiveness by constructing a graph that connects users with similar style embeddings, described next. for each user in the test set, we randomly sampled one text sample and extracted its embedding through the siamese models. then we created the pairwise similarity matrix m by computing the pairwise similarity between each text pair. then m is pruned by re ## compositionality of styles finally, we sought to understand how stylistic variations are encoded. at least for certain stylistic features, there is additive stylistic compositionality in the latent space onto which the texts are projected (figure 1 ). ## conclusions the relatively unconstrained nature of the online genres tolerates a much wider range of stylistic variation than conventional genres @xcite . online genres are often marked by unconventional spellings, heavy use of colloquial language, extensive deviations in formatting, and the relaxation of grammatical rules, providing rich linguistic resources to construct and perform one's identity. our analysis of idiolects in online registers has highlighted that idiolectal variations permeate all linguistic levels, present in both surface lexico-syntactic features and high-level discourse organization. traditional sociolinguistic research often regards idiolects as idiosyncratic and unstable and not as regular as sociolects @xcite ; here, we show that idiolectal variation is not only highly distinctive but also consistent, even in a relatively large population. our findings suggest that individuals may differ considerably by degrees of consistency and distinctiveness across multiple text samples, which sheds light on the theoretical discussions and practical applications in forensic linguistics. our findings also have implications for sociolinguistics, as we have shown an effective method to discover, understand and exploit sociolinguistic variation.