Style is NOT a single variable: Case Studies for Cross-Stylistic Language Understanding (2021)

Abstract: Every natural text is written in some style. Style is formed by a complex combination of different stylistic factors, including formality markers, emotions, metaphors, etc. One cannot form a complete understanding of a text without considering these factors. The factors combine and co-vary in complex ways to form styles. Studying the nature of the covarying combinations sheds light on stylistic language in general, sometimes called cross-style language understanding. This paper provides the benchmark corpus (XSLUE) that combines existing datasets and collects a new one for sentence-level cross-style language understanding and evaluation. The benchmark contains text in 15 different styles under the proposed four theoretical groupings: figurative, personal, affective, and interpersonal groups. For valid evaluation, we collect an additional diagnostic set by annotating all 15 styles on the same text. Using XSLUE, we propose three interesting cross-style applications in classification, correlation, and generation. First, our proposed cross-style classifier trained with multiple styles together helps improve overall classification performance against individually-trained style classifiers. Second, our study shows that some styles are highly dependent on each other in human-written text. Finally, we find that combinations of some contradictive styles likely generate stylistically less appropriate text. We believe our benchmark and case studies help explore interesting future directions for cross-style research. The preprocessed datasets and code are publicly available.

## introduction people often use style as a strategic choice for their personal or social goals in communication @xcite @xcite . some stylistic choices implicitly reflect the author's characteristics, like personality, demographic traits @xcite , and emotions @xcite , whereas others are explicitly controlled by the author's choices for their social goals like using polite language, for better relationship with the elder @xcite . in this work, we broadly call each individual linguistic phenomena as one specific type of style. style is not a single variable, but multiple variables have their own degrees of freedom and they co-vary together. imagine an orchestra, as a metaphor of style. what we hear from the orchestra is the harmonized sound of complex combinations of individual instruments played. a conductor, on top of it, controls their combinatory choices among them, such as tempo or score. some instruments under the same category, such as violin and cello for bowed string type, make a similar pattern of sound. similarly, text reflects complex combination of multiple styles. each has its own lexical and syntactic features and some are dependent on each other. consistent combination of them by the author will produce stylistically appropriate text. to the best of our knowledge, only a few recent works have studied style inter-dependencies in a very limited range such across demographical traits @xcite , across emotions @xcite , across lexical styles @xcite , across genres @xcite , or between metaphor and emotion @xcite . unlike the prior works, this work proposes the first comprehensive understanding of cross-stylistic language variation, particularly focusing on how different styles co-vary together in written text, which styles are dependent on each other, and how they are systematically composed to generate text. our work has following contributions: • aggregate 15 different styles and 23 sentencelevel classification tasks ( §3). based on their social goals, the styles are categorized into four groups ( definition of style. people may have different definitions in what they call 'style'. several sociolinguistic theories on styles have been developed focusing on their inter-personal perspectives, such as halliday's systemic functional linguistics (halliday, 2006) or biber's theory on register, genre, and style @xcite . in sociolinguistics, indexicality @xcite @xcite is the phenomenon where a sign points to some object, but only in the context in which it occurs. nonreferential indexicalities include the speaker's gender, affect @xcite , power, solidarity @xcite , social class, and identity @xcite . building on silverstein's notion of indexical order, @xcite built the notion that linguistic variables index a social group, which leads to the indexing of certain traits stereotypically associated with members of that group. @xcite argued that style change creates a new persona, impacting a social landscape and presented the expression of social meaning as a continuum of decreasing reference and increasing performativity. despite the extensive theories, very little is known on extra-dependency across multiple styles. in this work, we empirically show evidence of extra-linguistic variations of styles, like a formality, politeness, etc, but limited to styles only if we can obtain publicly available resources for computing. we call the individual phenomena a specific type of "style" in this work. we admit that there are many other kinds of styles not covered in this work, such as inter-linguistic variables in grammars and phonology, or high-level style variations like individual's writing style or genres. cross-style analysis. some recent works have provided empirical evidence of style interdependencies but in a very limited range: warriner et al. (2013) analyzed emotional norms and their correlation in lexical features of text. chhaya et al. (2018) studied a correlation of formality, frustration, and politeness but on small samples (i.e., 960 emails). nguyen et al. (2014) focused on correlation across demographic information (e.g., gender, age) and with some other factors such as emotions (preoţiuc-pietro and ungar, 2018). dankers et al. (2019); mohammad et al. (2016) studied the interplay of metaphor and emotion in text. liu et al. (2010) studied sarcasm detection using sentiment as a sub-problem. brooke and hirst (2013) conducted a topical analysis of six styles: literary, abstract, objective, colloquial, concrete, and subjective, on different genres of text. passonneau et al. (2014) conducted a detailed analysis of biber's genres and relationship between genres. ## case #2: style dependencies in addition to the theoretical style grouping in §3.1, we empirically find how two styles are correlated in human-written text using silver predictions from the classifiers. ## case #3: cross-style generation we study the effect of combination of some styles in the context of generation. we first describe our figure 3 : cross-style correlation. correlations with @xmath0.05 (confidence interval: 0.95) are only considered as statistically significant. the degree of correlation gradually increases from red (negative) to blue (positive), where the color intensity is proportional to the correlation coefficients. we partition the correlation matrix into three pieces: across interpersonal, figurative and affective styles (upper left), between persona and a group of interpersonal, figurative, and affective styles (upper right), and across persona styles (lower right). important note: please be very careful not to make any unethical or misleading interpretations from these modelpredicted artificial correlations. best viewed in color. ## conclusion and discussion we introduce a benchmark xslue of mostly existing datasets for studying cross-style language understanding and evaluation. using xslue, we found interesting cross-style observations in classification, correlation, and generation case studies. we believe xslue helps other researchers develop more solid methods on various cross-style applications. we summarize other concerns we found from our case studies: style drift. the biggest challenge in collecting style datasets is to diversify the style of text but preserve the meaning, to avoid semantic drift. in the cross-style setting, we also faced a new challenge; style drift, where different styles are coupled so changing one style might affect the others. ethical consideration. some styles, particularly on styles related to personal traits, are ethically sensitive, so require more careful interpretation of the results not to make any misleading points. any follow-up research needs to consider such ethical issues as well as provides potential weaknesses of their proposed methods. from correlation to causality. our analysis is based on correlation, not causality. in order to find causal relation between styles, more sophisticated causal analyses, such as propensity score @xcite , need to be considered for controlling the confounding variables. by doing so, we may resolve the biases driven from the specific domain of training data. for example, generated text with the politeness classifier @xcite ) contains many technical terms (e.g., 3d, opencv, bugs) because its training data is collected from stackexchange.