Listener’s Social Identity Matters in Personalised Response Generation (2020)

Abstract: Personalised response generation enables generating human-like responses by means of assigning the generator a social identity. However, pragmatics theory suggests that human beings adjust the way of speaking based on not only who they are but also whom they are talking to. In other words, when modelling personalised dialogues, it might be favourable if we also take the listener’s social identity into consideration. To validate this idea, we use gender as a typical example of a social variable to investigate how the listener’s identity influences the language used in Chinese dialogues on social media. Also, we build personalised generators. The experiment results demonstrate that the listener’s identity indeed matters in the language use of responses and that the response generator can capture such differences in language use. More interestingly, by additionally modelling the listener’s identity, the personalised response generator performs better in its own identity.

## introduction persona plays an important role in our daily communication since it affects the way we render our dialogues. social variables, such as gender, age, place of birth or even wealth and social status, account for a large proportion in each individual's persona. numerous previous studies have suggested that these variables strongly affect each speaker's word preference in dialogues. a growing body of works has been carried out to implicitly or explicitly model these variables in dialogues @xcite @xcite @xcite . despite the reported success, most previous studies for personalised dialogue modelling consider only the persona of speakers. 1 nevertheless, the 1 for using the terminology consistently, we use "speaker" pragmatics theory suggests that the speaking style will be adjusted not only by who the speaker is, but also whom the speaker is talking to @xcite . in the computational linguistics community, @xcite investigates this issue by measuring and mitigating gender bias in dialogue dataset utilising a gender classifier. from the aspect of personalised dialogue generation, @xcite and @xcite tried to attach the listener persona to the encoder of their generator, but interestingly, they obtained very different results, namely, the performance of @xcite went down while that of @xcite went up. nonetheless, no systematic studies have been conducted to investigate what role does the listener's identity play in personalised response generation. research questions that we wish to answer by the proposal put forward in this paper are: to this end, we apply analysis and build a response generator on a chinese personalised dialog dataset: personaldialog, a corpus extracted from weibo 2 . there are two reasons to use this dataset: one is that the personaldialog dataset origins from the real conversations on social media weibo, in which speakers' social variables play an important role; the other is that this dataset provides a massive amount of dialogue data (over 20m sessions) between a large population of speakers (over 8m speakers). it is of sufficient size to capture a variety of linguistic phenomena that are associated with social variables. each speaker/listener in per-sonaldialog comes up with 4 social variables: gender, age, location, and interests. for simplicity and for conducting controlled analysis and experiments, we only focus on gender in this paper. as for the first research question, we postulate that a speaker behave differently when s/he speaks to people with different gender stylistically. this yields four possible speaking styles: ff, mf, fm, and mm 3 . we, therefore, build a classifier to separate these styles defining on "gender-pairs". previous analysis on blogging data @xcite @xcite has identified that one of the key features for distinguishing contents produced by a female from those by a male is the sentence length, i.e., females tend to utter longer sentences. as shown in figure 1 , the same phenomenon is found in personaldialog: females' responses are generally longer than males'. further statistics on the response length falling the above four styles suggest that gender-pairs are also separable, perhaps excepting mf and fm at first glance. to validate this and understand why, we build a gender-pair classifier and conduct so-called pivot word analysis. we find out which word contributes the most for helping the classifier make decisions. experiment results show that these styles are separable, but mf and fm are often confused with each other. as for the second research question, we build a personalised response generator conditioning on these styles. the outcomes suggest that the generator could capture the difference between those styles and, in addition, modelling the listener's 2 weibo is the largest chinese social media. 3 we use fm to represent the style used by a male speaker when talking to a female listener. similar definition applies to mf, mm, and ff. identity helps the generator to express its own identity. moreover, based on previous analyses, we have also tried to merge the style of mf and fm into a single integrated style mf/fm. however, the final results of the response generator suggests that it is hard to model utterances with this integrated style. ## gender-pair classification to approach the first research question, we build a gender-pair classifier to simultaneously recognise the speaker's and listener's social identity based on the dialogue utterances. concretely, as aforementioned in section 1, we assume the present task as a style classification task and design four labels for each input dialogue utterance: mm (male talking to male), mf (female talking to male), fm (male talking to female), and ff (female talking to female). however, in light of the linguistic style matching theory @xcite , speakers will imitate the linguistic style of their conversation companion to pursue higher engagement. in other words, when two different gendered speakers communicate with each other, their speaking style may assimilate to each other as the conversation proceed. on the top of this observation, one may say that dissociating fm and mf is hard, and, therefore, it would be favourable if we merge fm and mf into a single category, namely mf/fm. ## personalised response generation for exploring the second research question, that is, can a personalised response generator capture the differences of language use when imitating a speaker talking to listeners with different social identities? we train multiple response generators conditioning on the three style categorising schemes mentioned in section 2. we start by introducing the basic architecture of our generator and the experimental settings. we then describe the evaluation metrics we use, with which we evaluate and analyse the generators. ## discussion we investigated the language use on chinese social media regarding to the social identities of speakers and listeners. specifically, we aim to explore whether the listener's social identities impact the responder's language use and whether such differences are separable. the primary answers to both of these questions are "yes" on the basis of our experiments and, additionally, by conducting pivot word analysis, we also found that mf and fm are less separable owing to the linguistic matching phenomenon. this raises as open question of which style categorisation scheme (i.e., whether to distinguishing mf and fm or not) is better for modelling personalised dialogues. we then trained personalised response generators which take the social identities of listeners into account. to conduct insightful analysis, we design a number of new metrics with the help of the speaking style classifiers and the extracted pivot words. the outcomes show that modelling listener's identity assists the dialogue system to express more of its own identity. however, our system failed to model the style of mf/fm, which suggests the necessity of disassociating the style between mf and fm. note that our work focus mainly on the gender, which from our perspective, underlies further studies on investigating the influence of other listener's social variables, such as age or location, or even of listener's persona as a whole. likewise, since we study only on data from chinese social media, it is also worth to validate whether our findings still hold in multilingual platforms like twitter. as for the designing of dialogue systems, we highlighted the importance of modelling listener's persona for the chatbot to express its own personality, it is also worthwhile to evaluate the built system in other angles, such as relevance and fluency, or to validate whether the resulting chat machine is empathetic @xcite or not. our decision on using single turn dialogue also limits the generalisability of our conclusion to real conversations since the assimilation of each others style may progress in the course of a dialogue. this may result in under-estimating the effect of the linguistic matching between speakers and listeners. in future, we will extend our work into multi-turn dialogue modelling. ## ethical statement in this paper, we use the gender as an example of social identity to understand how the speaking style of a speaker is influenced. to this end, we build gender classifiers and stylised dialogue systems. in light of the discussion in @xcite , gender is notoriously difficult to detect @xcite , and mis-gendering individuals is harmful to users @xcite . therefore, we are not and will not apply or extend the built classifiers and dialogue systems into real applications. we hope our findings could help with further works on mitigating gender bias @xcite or improving fairness @xcite in dialogue systems.