Analysis of Style-Shifting on Social Media: Using Neural Language Model Conditioned by Social Meanings (2023)

Abstract: In this paper, we propose a novel framework for evaluating style-shifting in social media conversations. Our proposed framework captures changes in an individual’s conversational style based on surprisals predicted by a personalized neural language model for individuals. Our personalized language model integrates not only the linguistic contents of conversations but also non-linguistic factors, such as social meanings, including group membership, personal attributes, and individual beliefs. We incorporate these factors directly or implicitly into our model, leveraging large, pre-trained language models and feature vectors derived from a relationship graph on social media. Compared to existing models, our personalized language model demonstrated superior performance in predicting an individual’s language in a test set. Furthermore, an analysis of style-shifting utilizing our proposed metric based on our personalized neural language model reveals a correlation between our metric and various conversation factors as well as human evaluation of style-shifting.

## introduction style-shifting refers to such changes in a speaker's language style in a conversation as speech acts @xcite , vocabulary @xcite , syntax @xcite , prosody @xcite according to the context of the conversation, the speaker's background, and the relationship with the interlocutor. for example, formal language is used in business situations; informal language is used in private situations. style-shifting can also happen within a conversation to show respect or intimacy to fit the situation @xcite . such changes in speech style also depend on the presence of people who may be directly or indirectly listening to a conversation, such as on social media @xcite @xcite . style-shifting is often associated with speech accommodation theory, which describes how speakers adjust their language use to converge or diverge from that of their interlocutors @xcite . convergence occurs when speakers adopt a similar style to their partner, aiming to foster understanding, build rapport, and establish a sense of affiliation. divergence occurs when deviating from the interlocutor's language style to accentuate differences, to assert individuality, or emphasize social identity. effective style-shifting helps speakers adjust their psychological distance and improve their communication intentions, leading to smoother conversations @xcite . measuring style-shifting phenomena is important to advance our understanding of conversation dynamics. research on stylistic variation has underscored the intrinsic properties of conversational styles and their interplay with various non-linguistic factors @xcite @xcite . furthermore, many studies have quantified the changes in a speaker's language style in a conversation based on the similarity in word choice between utterances @xcite @xcite @xcite . however, despite these insights, such word-based methods still face challenges in capturing nuanced changes in style that reflect various social meanings. the term "social meaning" represents a constellation of traits that linguistic forms convey about the social identity of their users, such as group membership, personal attributes, and individual beliefs @xcite . to evaluate style-shifting effectively, it is essential to go beyond merely observing words used in the figure 1 : the architecture of our personalized neural language model based on the transformer language model: the green blocks denote linguistic factors, an orange block denotes implicit social meanings and their relationships associated with each user, and yellow blocks reflect the potential social impacts of users and their utterances. to tackle these problems, we propose focusing on the surprisal experienced by a neural language model personalized for an individual. the magnitude of surprisal in the personalized language model is inversely proportional to the consistency between the utterances they are confronted with and the standard language style they support. in other words, the smaller surprisal the more consistent the utterance is with the style of their language; and a larger surprisal indicates that the utterance deviates from their standard language style. the magnitude of surprisal also depends on the incorporated factors of individuals. comparing surprisal under various conditions could not only assess the level of style-shifting but could also reveal valuable insights into the complex interplay of various factors that influence that change. furthermore, neural language models are capable of taking into account long-term dependencies between words, deeper meanings, and structures in natural language @xcite @xcite . it can also incorporate non-linguistic factors, such as social meanings, in addition to linguistic factors as conditions for personalizing neural language models for individuals @xcite . our key contributions are as follows: 1) we introduce a novel metric for evaluating styleshifting, based on surprisal predicted by a personalized neural language model. 2) we develop a unique personalized language model that incorporates both linguistic and non-linguistic factors. 3) our personalized language model outperforms existing models in predicting an individual's language use in new conversations, highlighting the effectiveness of incorporating non-linguistic factors into language modeling. 4) we conduct a comprehensive analysis of style-shifting, revealing correlations between our metric and various linguistic and non-linguistic conversation factors as well as human evaluation of style-shifting. ## neural language modeling conditioned by social meanings in this section, we describe our approach to developing a personalized neural language model integrating both linguistic and non-linguistic factors. ## evaluation metric of style-shifting in this section, we describe the metrics used to evaluate style-shifting in our personalized neural language model. our style-shifting evaluation metrics are based on the surprisal, which is the negative log-likelihood of target utterance calculated from our personalized neural language model conditioned by social meanings. in general, the magnitude of surprisal in an individual's personalized language model is inversely proportional to the consistency between the utterances they are confronted with and the standard language style they support. in other words, the smaller the surprisal, the more consistent the utterance is with their language style, and a larger surprisal indicates that the utterance deviates from their standard language style. comparing surprisal under various conditions could not only assess the level of style-shifting but could also reveal valu-able insights into the complex interplay of various factors that influence that change. based on the idea of surprisal, we consider two perspectives that are not typically considered in conventional metrics. the first perspective is the similarity of language models among individuals during a conversation. the second perspective is the relative change in language style compared with their usual conversation. these aspects are often under-emphasized in existing studies that rely on vocabulary comparisons between speakers within the conversation @xcite @xcite . however, our approach takes both of these perspectives into consideration, enabling us to gain a deeper understanding of style-shifting by assessing stylistic alterations in relation to our own usual conversation. ## experimental settings this section describes dataset construction, training, evaluation criteria, and analysis framework. ## related work there are existing methods for analyzing styleshifting that utilize language models, similar to our proposed method. some studies construct unigram language models for each speaker in a conversation and compare differences in word likelihoods @xcite . another study evaluates whether one speaker's use of a marker increases the likelihood of its use by another @xcite . however, these methods often encounter issues with sparse training data due to the limited number of conversations involving the same speaker @xcite . as a result, they carefully select a small number of marker words that capture the syntactic features of conversation for training the language model. moreover, they do not consider non-linguistic factors that could influence language style beyond speaker ids. our method addresses these issues by extending the existing methods that use unigram language models. ## conclusion in this paper, we proposed a novel framework for analyzing style-shifting in social media conversations. our framework captures changes in an individual's conversational style, leveraging surprisals predicted by a personalized neural language model. this model integrates not only the linguistic content of conversations but also nonlinguistic factors such as social meanings. compared to existing models, our personalized language model demonstrated superior performance in predicting an individual's language in a test set. furthermore, our proposed metric for style--2 -2 -2 -3 -2 -4 -2 -5 0 2 -5 2 -4 2 -3 2 -2 2 -1 2 0 2 1