Reducing Sensitivity on Speaker Names for Text Generation from Dialogues (2023)

Abstract: Changing speaker names consistently throughout a dialogue should not affect its meaning and corresponding outputs for text generation from dialogues. However, pre-trained language models, serving as the backbone for dialogue-processing tasks, have shown to be sensitive to nuances. This may result in unfairness in real-world applications. No comprehensive analysis of this problem has been done in the past. In this work, we propose to quantitatively measure a model’s sensitivity on speaker names, and comprehensively evaluate a number of known methods for reducing speaker name sensitivity, including a novel approach of our own. Extensive experiments on multiple datasets provide a benchmark for this problem and show the favorable performance of our approach in sensitivity reduction and quality of generation.

## introduction the safety and fairness issue of generations from dialogue models is a crucial concern in real applications. previous work focuses on response generation from open-ended dialogue systems @xcite , such as offensive contents @xcite , gender bias @xcite and other discriminated behavior @xcite . for other text generation tasks where the whole dialogue is provided and the output shouldn't go beyond the dialogue, such as dialogue summarization @xcite and dialogue reading comprehension @xcite , the fairness issue is still unexplored. in these tasks, the input dialogues are selfcontained, and the names of the speakers do not carry any connotation from outside of the dialogue. therefore, changing the speaker names consistently in a dialogue should not affect the meanings of the dialogue and the desired outputs. this contrasts with response generation, where the dialogue is in progress and the output is expected to be different in styles or contents for various speakers. taking dialogue summarization @xcite as an example for text generation from dialogues, it focuses on generating concise "who-did-what" summaries in the third person. in fig. 1 , the two dialogues are identical except for the speaker names. the two summaries are expected to be the same modulo the speaker names. unfortunately, models nowadays, following the pretrain-finetune paradigm, are sensitive to trivial changes, which has been verified in other tasks. in relation extraction, spurious correlations between entity mentions and relations lead to entity bias @xcite @xcite . other similar work includes the analysis of robustness by entity renaming for machine reading comprehension models on narrative texts @xcite and name biases in machine transla-tion with inflected languages @xcite , like german. besides, @xcite claims that pre-trained language models do not treat given names as interchangeable or anonymous, showing unfairness in reading comprehension. obviously, dialogue understanding models are sensitive to speaker names according to fig. 1 as well. the model tends to generate different information given different speaker names, such as "don't want to go" and "doesn't like them". incorrect content, "... betsy don't want to go", is generated with the first group of speakers, while not with the other group. according to our pilot experiment with the vanilla bart fine-tuned on samsum, around 74.00% of generations are changed by switching speaker names and 69.82% among them are due to distinct contents. such uneven performances create unfairness among different speakers, especially in the aspect of information allocation. the model may also catch latent properties in names @xcite and lead to discrimination, raising the importance of research on the sensitivity on speaker names. previous work has also mentioned this problem. different data pre-processing approaches are adopted during the construction of datasets to avoid using speaker names, such as "a" or "b" in @xcite . @xcite replace speaker names with more common and frequent names that the model may have seen during pre-training. data augmentation by changing speaker names is adopted by @xcite . however, all of them only attempted to attack this problem subjectively, without quantitive analysis and fair comparisons. in this work, we systematically analyze speaker name sensitivity in text generation from dialogues. we define the speaker name sensitivity and divide the approaches into offline and online ones. then, we propose two novel insensitivity losses, helping to reduce attention and hidden state distances of the same dialogue with different speaker names for transformer-based models during finetuning. these losses can be used in both kinds of approaches. results on several tasks show that our losses reduce the sensitivity and get better generations. in summary, our contributions are: ## proposed approach we focus on the widely-accepted encoder-decoder architecture for pre-trained generation models and design two auxiliary insensitivity losses to take full advantage of augmented data on top of aug. given the dialogue sample with different speaker names, a model outputs distinct generations due to its different internal behaviors. therefore, penalizing unexpected internal differences should help the model behave consistently and reduce the sensitivity. with this intuition, we propose the crossattention loss and the decoder-hidden-state loss. an illustration for them is in appendix a. the former corresponds to cross-attention distributions that help the decoder make a soft information selection among encoder hidden states at each step and should be similar with different speaker names. the latter is based on the final decoder hidden states which are expected to be the same under the default teacher-forcing training strategy except for the speaker name tokens. we didn't consider the encoder attentions since according to our pilot analysis of the vanilla bart, the cross attentions distance of the different predictions is around 1.5 times of the same ones. however, there are no differences in the encoder attentions. other intermediate hidden states are excluded since they are all affected by different input embeddings of speaker names, except that the final decoder hidden states are sure to be the same. ## experimental setup we define the evaluation metrics for sensitivity, introduce multiple text generation tasks with dialogue data and present implementation details. ## results we show performances of approaches first, followed by ablation studies and evaluations. then, we take a closer look at offline approaches, which show the inherent capability of models, with multifaceted analysis. hyper-parameter search and case studies are in appendix c and e. ## conclusion this paper focuses on the speaker name sensitivity in the text generation from dialogues. we provide a classification for previous approaches, and propose the insensitivity losses to reduce the sensitivity while achieving favorable generation quality. fair comparisons and comprehensive analysis are done among different approaches for evaluating the sensitivity quantitatively. more approaches targeting dialogue sensitivity issues are expected.