Towards Personality-Aware Chatbots (2022)

Abstract: Chatbots are increasingly used to automate operational processes in customer service. However, most chatbots lack adaptation towards their users which may results in an unsatisfactory experience. Since knowing and meeting personal preferences is a key factor for enhancing usability in conversational agents, in this study we analyze an adaptive conversational agent that can automatically adjust according to a user’s personality type carefully excerpted from the Myers-Briggs type indicators. An experiment including 300 crowd workers examined how typifications like extroversion/introversion and thinking/feeling can be assessed and designed for a conversational agent in a job recommender domain. Our results validate the proposed design choices, and experiments on a user-matched personality typification, following the so-called law of attraction rule, show a significant positive influence on a range of selected usability criteria such as overall satisfaction, naturalness, promoter score, trust and appropriateness of the conversation.

## introduction in today's rapidly emerging technology-driven world, chatbots are becoming a more significant factor in customer interaction. next to voice-driven assistants, text-based conversational agents-commonly known as chatbots-have attracted significant attention in recent years. chatbots are designed to interact with humans using natural language and are commonly used on messaging platforms and websites @xcite . with recent advancements in the field of artificial intelligence (ai), organizations are starting to realize the potential of chatbots to automate their customer service operations and hence reduce costs @xcite . furthermore, it was predicted that 80% of organizations ⋆ corresponding authors would have deployed a chatbot by 2020 @xcite . however, the quality of today's systems does not seem to meet customer expectations @xcite . a key obstacle preventing most chatbots from being successful is that the interaction lacks humanness and naturalness @xcite . several studies have investigated social cues and their positive effect on users' perceived social presence, trust, enjoyment, and usage intentions @xcite . however, it has also been shown that social cues may have a negative effect that ends up irritating the user @xcite . studies about the nature and quality of humanmachine interactions have identified personality as an essential factor for this issue @xcite . personality is a stable pattern that provides a measure for a person's behavior (und gregory @xcite . traditionally, personality is assessed by questionnaires; current approaches, however, make it possible to use human-generated data from social media or online forums @xcite . a person's language can provide information about the user's personality @xcite @xcite . to address these challenges and leverage modern technologies, the development of a personality type-indicator adaptive chatbot that automatically adapts to a user's presumed personality type is proposed in this work. the studies analyzes the impact of the so-called "law of attraction," according to which users reported higher communication interaction, human-likeliness, preference, and friendliness when interacting with a chatbot that has equal personality traits @xcite . however, the studies introduced did not produce statistically significant results except for @xcite work @xcite . their study did not require full interaction with an applied chatbot, but rather examined the perception of different personalities in a chatbot by showing their participants screenshots of the interactions. in our empirical quantitative user study, we therefore evaluate how adapted personality types are perceived by chatbot users for the domain of a job recommender chatbot and whether or not personality type-based adaptation can lead to higher overall satisfaction, usability, trust, and appropriateness. furthermore, there exist very few works about design criteria for how to realize personality in terms of chatbot design. this paper seeks to contribute to this area by giving design implementation details. ## chatbot design our personality-adaptive chatbot prototype is based on the microsoft azure bot framework and is built in the browser, allowing it to be embedded into various channels. depending on the input personality, the respective conversation tree is activated for the task of job recommendation divided into two subdialogs. the first sub-dialog generally greets the user, while the second one asks job-related questions to give a personality-based recommendation. to design the chatbot's personality type, the previously introduced linguistic cues were used. overall, due to the short nature of the interactions, the metrics concerning word counts, sentence length, and word length were hard to manipulate when designing the messages, as there were too many dependencies on other metrics such as references to facts. ## user study the experiment consists of five steps. (1) users fill out a short 12-item personality self-report according to oejts. (2) participants interact with our chatbot, with random assignment of matched or mismatched personality type. (3) users assess first interaction by nine usability items. ( 5 the first part of the study is a survey is a 12item personality self-report based on the oejts. for this study, each of the nine highest scoring items on the i/e and the t/f scales are used in this experiment. additionally, each dichotomy has further been divided into six items of the e/i types and six items of the t/f types. the selected items were assessed by using a five-point likert scale in between"strongly agree," "agree," "neither agree nor disagree," "disagree," and "strongly disagree." depending on the users personality type, two chatbots were automatically selected to be tested in step 2 and step 5, of which one is designed to be perceived the same personality type as the user (matched), whereas the other one represents the opposite option settings (mismatched).for example, if a user is classified as ef (extroverted-feeling), they interacted with both an ef and an it designed chatbot, in random order. the extroverted chatbot was named carla and the introverted one was named sophia to achieve the effect that users are more likely to share personal information if the chatbot appears to be female @xcite . the topic of the interactions in step 2 and 5 is to chat about personal and job-related preferences to recommend a suitable job. the job recommendations given by the chatbot in the end of the conversation are hand crafted and based on the personality of the user. note that we do not analyze the performance of any recommendation accuracy, nor the users' acceptance towards it. in this work, we focus on the impact of personality on the usability of the interaction explicitly. in more detail, the conversation starts with some general questions regarding the name, origin, and personal preferences. afterwards, the chatbot commences asking about job-related preferences. three questions are asked that are based on additional items of the oejts. for example, one item of the oejts to measure extroversion assesses whether the user "works best in groups" or "works best alone." further, the chatbot is designed to be between the edges of an intra-and an interpersonal chatbot within a closed domain, offering limited functionality @xcite . hence, the chatbot only allows the user to answer the questions instead of providing functionality that answers custom questions of the user. this limitation was explicitly clarified at the beginning of the survey to avoid false expectations. moreover, the users have also been instructed of another limitation of the current state of chatbot prototype implementation, namely that writing multiple messages is not supported. this means that all information has to be put into a single message. the usability questionnaire applied consists of nine items that are asked after each chatbot interaction: two items that compare both chatbots with each other and five general items about the participants. first, the nine items that are asked directly after each conversation with the chatbot are introduced. these items are split into four items derived from itu telecommunication standardization sector (itu-t) recommendation p.851 @xcite , while the other five items are custom-designed. adapted to the personality domain, four items were selected that are related to the following factors: acceptability, naturalness, and promoter score. for these items (among others), it was demonstrated that acceptability and naturalness are well generalized @xcite . the personality factor from itu-t was not suitable for the experiments at hand due to the strong focus on personality type differentiation of this study. hence, five custom items were designed to measure whether the design choices applied could be perceived by the participants when interacting with the different chatbots. these nine usability items were assessed using the same five-point likert scale from above. in addition, two items were designed to directly compare carla (extroverted) and sophia (introverted) headto-head. the first item assesses which chatbot is being perceived as more adapted toward the users' preferences, while the second asks for the general preference when comparing both directly. for both items, users had the option to choose carla, sophia, both, or none. eventually, five profiling questions were asked at the end of the survey regarding gender, age range, experience with chatbots, native language, and their current profession. all items are shown in appendix a, also including the items used for comparison and general profile data. ## discussion in general, our results and expectations are in line with the law of attraction within a text-based conversational agent @xcite domain such that overall satisfaction, trustworthiness and appropriateness are significantly higher for the matched personality-based chatbot. also, the difference between the combination of et and if is much smaller compared to a scenario in which the user interacts with the bots ef and it. for the first scenario, the messages of the bots only differ by 59 words; however, the second scenario offers 87 words in message length through the overall course of the dialogue. a set of preliminary results may shed some light on the unexpected results. when looking at a onesided t test within the sub-sample of ef and it classified participants, the effect of perceived message length is also greater compared to the whole sample (t(144) = 3,863, @xmath0.001, @xmath1.321). however, it is natural that the et and if types are more similar to each other compared to the ef and it. surprisingly, the differently designed emotionality of the messages did not yield significant results in terms of distinction. a possible explanation for this could be that the perception of emotionality is biased by the use of emojis, which are perceived as an emotional variable. the difference in the usage of emoticons between if and et is in favor of the et type. hence, the et type could be perceived as more emotional given the higher number of emojis, which is also related to feeling. therefore, similar to the design aspect of message length, the other combinations of it and ef should show clearer results as ef is designed to be feeling and uses numerous emojis. a paired t test also supports this assumption where the ef type is per-ceived as significantly more emotional than the it, t(144) = 1,967, @xmath2.026, @xmath3.163. hence, there might be an interference with the usage of emojis and the relationship towards feeling that was not designed clearly enough for those participants that were interacting with et carla and if sophia. another possible reason for the lack of perceived emotionality, in general, could be that this study designed the t/f dichotomy under the assumption that there is a correlation with big 5's agreeableness. due to the lack of research modelling thinking and feeling linguistically, the linguistic cues of agreeableness were used to design t/f. the two traits correlate with each other (0.47) according to a study by @xcite . nevertheless, they are not equal, which might result in an information loss or false interpretation other than what was intended. further, the separation of the extroverted and introverted bot is also dependent on whether they were rated as the matched or the mismatched interaction, respectively. our study shows, the law of attraction has an impact on the perception of the two chatbots. however, a subliminal study showed that there are no major differences when analyzing the scores within the samples of only matched interactions, the samples of only mismatched interaction, and the whole sample. when investigating the results, regardless of the matched or mismatched personality, the introverted and formal-designed chatbots (introverted sophia) were rated higher than the more informal ones (extroverted carla). this also fits into the domain of job recommendation which is usually associated with professionalism where formality is required. the more formal bot also scores better on appropriateness and overall satisfaction. for the evaluation, 266 people have interacted with it in a realistic scenario, and have rated the interaction by means of mos. similar studies either did not provide a direct interaction with the chatbot @xcite (users only rated screenshots) or could only show tendencies with small sample sizes @xcite . hence, to the best of our knowledge, this is the first study to show a statistically significant positive effect, though small, of automatically adapted matched personality of a chatbot (@xmath4, trust, and appropriateness for the task of job recommendation. in addition, linguistic cues that correlate with cer-tain personality traits were introduced @xcite @xcite and the results presented in this paper further contribute to this body of research. they indicate that personality differences embodied in language were significantly perceived in two out of three design choices. these findings further validate that matched personality results in significantly higher usability scores (in all but one of the items used in our study) of a chatbot. apart from that, trustworthiness and appropriateness (for the task of job recommendation) were also shown to be significantly better when matching the personality type compared to mismatching it. our results are in line with previous research @xcite @xcite @xcite , while at the same time quantitatively demonstrating the effect of the law of attraction for a high number of participants @xcite . in contrast to other studies, our study enhances the transparency of the degree of linguistic cues applied by precisely stating the numbers of linguistic cues; related work on chatbots with personality only described their exact measures briefly.