Linguistic Elements of Engaging Customer Service Discourse on Social Media (2022)

Abstract: Customers are rapidly turning to social media for customer support. While brand agents on these platforms are motivated and well-intentioned to help and engage with customers, their efforts are often ignored if their initial response to the customer does not match a specific tone, style, or topic the customer is aiming to receive. The length of a conversation can reflect the effort and quality of the initial response made by a brand toward collaborating and helping consumers, even when the overall sentiment of the conversation might not be very positive. Thus, through this study, we aim to bridge this critical gap in the existing literature by analyzing language’s content and stylistic aspects such as expressed empathy, psycho-linguistic features, dialogue tags, and metrics for quantifying personalization of the utterances that can influence the engagement of an interaction. This paper demonstrates that we can predict engagement using initial customer and brand posts.

## introduction providing quality customer service on social media has become a priority for most companies (brands) today. a simple customer-brand interaction started by a moment of annoyance can be relieved when the brand resolves the issue in a public display of exceptional customer service. according to forbes, companies that use twitter as a social care channel have seen a 19 percent increase in customer satisfaction @xcite . furthermore, customers are rapidly turning to social media for customer support; research from jd power finds that approximately 67 percent of consumers now tap networks like twitter and facebook for customer service @xcite . while providing timely and stellar service has its advantages, engaging in a collaborative dialogue with its customers also leads to mutual trust and transparency according to social customer relations management (scrm) the-ories @xcite . scrm has been documented as a core business strategy @xcite . scrm refers to "a philosophy and a business strategy, supported by a technology platform, business rules, processes, and social characteristics, designed to engage the customer in a collaborative conversation to provide mutually beneficial value in a trusted and transparent business environment" @xcite . while brand agents on social platforms are typically motivated and wellintentioned to help engage with customers, their efforts are often ignored if their initial response to the customer does not match a specific tone, style, or topic the customer is aiming to receive. hence, we explore what textual elements of a brand's response are predictive of customer engagement. while there has been substantial research on customer service on social platforms, a majority has predominantly addressed issues such as timely response @xcite , timely transfer from a bot to human @xcite , improving bot performance @xcite @xcite , improving dialogue act prediction @xcite , and managing customer sentiment of users on the platform @xcite . a few studies have also examined the tone and emotional content of the customer service chats @xcite @xcite . however, more subtle and integral stylistic aspects of language, such as expressed empathy, psycho-linguistic features (e.g., time orientation), and the level of personalization of the responses, have received little attention, particularly when analyzed for engagement metrics. for the few studies that have looked at subtle stylistic aspects of language in customer service settings @xcite , the studies were more lab-based in synchronous, face-to-face, or call settings and may not translate to asynchronous, text-based contexts. additionally, only emotional aspects of empathy were considered. yet, similar to face-to-face communication, text-only communication also contains many subtle, and not so subtle, social cues. @xcite @xcite . broadly, there are two dimensions of language that can provide information about a conversation: language content and style. the content of a conversation indicates the general topics being discussed, along with the relevant actors, objects, and actions mentioned in the text. conversational style reflects how it is said @xcite . thus, a text-based response can be examined for its content and its style. language can be viewed as a fingerprint or signature @xcite . besides reflecting information about the people, organizations, or the society that created it, the text also impacts the attitudes, behavior, and choices of the audience that consume it @xcite . for example, the language of the response from a brand agent can assure and calm a consumer, infuriate them, or make a customer anxious. while language certainly reflects something about that writer (e.g., their personality, how they felt that day, and how they feel towards someone or something), the language also impacts the people who receive it @xcite . it can influence customer attitudes toward the brand, influence future purchases, or affect what customers share about the interaction @xcite . overall, in this paper, we aim to examine how the initial query from a customer and the initial response from a brand's agent impact the engagement of interaction on social media. however, rather than focusing just on the textual content of a conversation alone, we also examine the language style through the use of cognitive and emotional expressed empathy (e.g., emotional reactions, interpretations, explorations) @xcite , psycho-linguistic (liwc) language features (e.g., time orientation, tone) @xcite ,dialogue-tags, and novel use of perplexity as a metric of personalization of the responses @xcite . overall, to the best of our knowledge, we make a first attempt at examining a comprehensive set of content and style features from customer service conversations to understand their impact on engaging conversations between brand agents and customers. in addition, this is the first study to analyze customer engage-ment as a measure of the effort of brand agents to provide customer support beyond positive sentiment. moreover, we also build a prediction model to demonstrate the predictive capability of these stylistic features. we show it is possible to predict the likelihood of its engagement from the first response from a brand agent. ## related work given the importance of customer service there exists a substantial body of research addressing different challenges in this area. a body of researchers focuses on improving chatbots. @xcite build upon social response theory and anthropomorphic design cues. they find artificial agents can be the source of persuasive messages. however, the degree to which humans comply with artificial social agents depends on the techniques applied during human-chatbot communication. in contrast, @xcite designed a new customer service chatbot system that outperformed traditional information retrieval system approaches based on both human judgments and an automatic evaluation metric. @xcite examine the role of tone and find that tone-aware chatbot generates as appropriate responses to customer requests as human agents. more importantly, the tone-aware chatbot is perceived to be even more empathetic than human agents. @xcite emphasize the repair mechanisms (methods to fix bad chatbot responses) and find that chat-bots expressing uncertainty are bad in the customer service setting. thus, @xcite develop a method to suggest likely alternatives in cases where confidence falls below a certain threshold. another stream of research examines the role of emotions and sentiment in service responses. for example, @xcite find that emotional text positively impacts customers' perceptions of service agents. service agents who use emotional text during an online service encounter were perceived to be more social. @xcite identify elements of customer service-related discussions that provide positive experiences to customers in the context of the airline industry. they find that positive sentiments were linked mostly to online and mobile check-in services, favorable prices, and flight experiences. negative sentiments revealed problems with the usability of companies' websites, flight delays, and lost luggage. evidence of delightful experiences was recorded among ser-vices provided in airport lounges. @xcite explores the factors and external events that can influence the effectiveness of customer care. they focus on the telecom industry and find that the quality of digital customer care that customers expect varies across brands. for instance, customers of higher priced firms (e.g., verizon and at&t) expect better customer care. different firms provide different levels of customer service and seemingly unrelated events (e.g., signing an exclusive contract with a celebrity) can impact digital customer care. there is also research that focuses on "dialogue acts"-identifying utterances in a dialogue that perform a customer service-specific function. for instance, @xcite study how agent responses should be tailored to the detected emotional response in customers, in order to improve the quality of service agents can provide. they demonstrate that dialogue features (e.g., dialogue acts/topics) can significantly improve the detection of emotions in social media customer service dialogues and help predict emotional techniques used by customer service agents. similarly, @xcite develop a novel method for dialogue act identification in customer service conversations, finding that handling negation leads to better performance. @xcite use 800 twitter conversations and develop a taxonomy of fine-grained "dialogue acts" frequently observed in customer service. example dialogue acts include, but are not limited to, complaints, requests for information, and apologies. moreover, @xcite show that dialogue act patterns are predictive of customer service interaction outcomes. while their outcome analysis is similar to our study (i.e., predicting successful conversations), it differs in the ultimate analysis and goals. specifically, they focus on function rather than language style. for example, their work can guide service representatives to ask a yes-no question, provide a statement, or ask for information. in contrast, in this paper, we focus on looking at specific language features. our work can guide how the customer representative should respond (e.g., provide a more specific response), which is different than describing what they should do (e.g., ask for more information). finally, closely related to our study, there are a range of studies examining the different aspects of language and its impact on customer service. for example, @xcite study linguistic correctness-the tangibility, specificity, or imag- @xcite find that customer empathy strengthens the positive effect of employee empathy on customer satisfaction, leading to more "symbiotic interactions." the major difference between the prior studies and ours is that they study empathy in lab-based settings (i.e., real-world interactions) and not text conversations on social media. furthermore, we correlate language to consumer engagement, which is missing in prior work. ## data our data set consists of customer service-related queries and brand responses from their twitter service handles. we start with two million tweets (2,013,577) spanning over 789k conversations between 108 brands and 667,738 customers. some brands have as few tweets as 107 conversations (e.g., officesupport), while other brands such as amazon and apple have 100k and 36k conversations, respectively. figure 1 plots the brands that have more than 10k tweets. the average number of tweets per conversation is 4.15 and the average number of words per tweet is 18.52. the length of these conversations varies substantially, while some are as short as one round (user @xmath0. we measure engagement by counting the number of @xmath1. for instance, if the customer writes a question, a brand responds, and then the customer never responds again, then that would have an engagement count of zero. if the customer responds once, then the engagement count is one. to understand engagement better, we manually analyzed 500 conversations with more than 1 round (@xmath2...) to understand the nature of these conversations. we found that 85% of these conversations were putting substantial personalized effort towards trying to resolve customer issues, 1% were appreciations from customers, and the remaining 14% represented customers' frustrations/anger with a poor experience. even among the 14% of the conversations that expressed anger initially, we found that the brand agent showed effort towards helping the customer when the customer actively engaged with them. thus, we find that the length of a conversation can reflect the effort and quality of the initial response made by a brand toward collaborating and helping consumers. while many of these conversations might get resolved offline and reflect neutral sentiment, or limited engagement, on social platforms, the conversation size can provide a helpful signal in finding conversation-related characteristics that signal quality brand responses. hence, we are able to use this metric of engagement to better understand which language factors lead to it. we operationalize the engagement in two forms. first, we consider the length of the conversation, for example, figure 2b has a length of three. the length of the conversation represents the effort brand agents put in to resolve customer query. second, we also construct a binary "engagement indicator" outcome variable which is "engaging" when the discourse has more than one round (@xmath3..), and "not-engaging" when it ends in one round (@xmath4. figure 2b and figure 2a provide a sample example of a engaging vs. not engaging conversation. ## method our main research question is to understand how the content and stylistic features of the text influence the engagement of interaction in social media discourse. thus, our methodology involves five major steps. in step 1, we collect and understand data to identify engagement through the length. for step 2, we generate the stylistic metrics to cover empathy, personalization/novelty of a response, dialogue tags, and general psycho-linguistic features. we also include content-based features. for step 3, we operationalize engagement as an engagement indicator ("engaged", if the length is greater than one or "not engaged" otherwise). step 4 involves machine learning experiments. we start with all two million tweets (2,013,577) spanning over 789k conversations between 108 brands and 667,738 customers. we then randomly split (60:40) the entire data set into training and test sets (train size-472,512; test size-317,348). for step 5 we report the most predictive items for all features. the details of the feature generation process are described in the following subsections. ## experiments this section evaluates how style and content features impact the general predictive performance of the machine learning models. ## conclusion our study demonstrates that even though some customer support requests on social media might reflect anger, there are some key indicators that can engage customers in a positive direction. most extant research has not paid any significant attention to this aspect of the discourse, mostly simply focusing on sentiment or timely response. hence, we examined text based, asynchronous social media discourses for both what is written and how it is written, to examine how these features influence customer engagement. our study effectively identifies multiple such stylistic features that can influence the engagement of these social discourses between customers and brands.