Self- and Pseudo-self-supervised Prediction of Speaker and Key-utterance for Multi-party Dialogue Reading Comprehension (2021)

Abstract: Multi-party dialogue machine reading comprehension (MRC) brings tremendous challenge since it involves multiple speakers at one dialogue, resulting in intricate speaker information flows and noisy dialogue contexts. To alleviate such difficulties, previous models focus on how to incorporate these information using complex graph-based modules and additional manually labeled data, which is usually rare in real scenarios. In this paper, we design two labour-free self- and pseudo-self-supervised prediction tasks on speaker and key-utterance to implicitly model the speaker information flows, and capture salient clues in a long dialogue. Experimental results on two benchmark datasets have justified the effectiveness of our method over competitive baselines and current state-of-the-art models.

## introduction dialogue machine reading comprehension @xcite aims to teach machines to understand dialogue contexts so that solves multiple downstream tasks @xcite @xcite @xcite . in this paper, we focus on question answering (qa) over dialogue, which tests the capability of a model to understand a dialogue by asking it questions with respect to the dialogue context. qa over dialogue is of more challenge than qa over plain text @xcite @xcite owing to the fact that conversations are full of informal, colloquial expressions and discontinuous semantics. among this, multi-party dialogue brings even more tremendous challenge compared to two-party dialogue @xcite since it involves *corresponding author. this paper was partially supported by key projects of national natural science foundation of china (u1836222 and 61733011). u 4 u 1 u 6 u 8 u 11 u 2 u 5 u 9 u 10 u 3 u 7 u 1 : [monica geller: tell him.] u 2 : [rachel green: no.] u 3 : [phoebe buffay: tell him, tell him!] u 4 : [monica geller: just please tell him.] u 5 : [rachel green: shut up!] u 6 : [chandler bing: tell me what?] u 7 : [monica geller: look at you, you even look at him.] u 8 : [chandler bing: oh, come on tell me. i could use another reason why women look at me.] u 9 : [rachel green: all right, all right. last night, i had dream that, uh, you and i, were ] u 10 : [phoebe buffay: dating on this table.] u 11 : [chandler bing: wow!] q 1 : who was with rachel in her dream? a 1 : chandler bing q 2 : where did rachel and chandler date? a 2 : on this table multiple speakers at one dialogue, resulting in complicated discourse structure @xcite and intricate speaker information flows. besides this, @xcite also pointed that for long dialogue contexts, not all utterances contribute to the final answer prediction since a lot of them are noisy and carry no useful information. to illustrate the challenge of multi-party dialogue mrc, we extract a dialogue example from friendsqa dataset @xcite which is shown in figure 1 . this single dialogue involves four different speakers with intricate speaker information flows. the arrows here represent the direction of information flows, from senders to receivers. let us consider the reasoning process of q 1 : a model should first notice that it is rachel who had a dream and locate u 9 , then solve the coreference resolution problem that i refers to rachel and you refers to chandler. this coreference knowledge must be obtained by considering the information flow from u 9 to u 8 , which means rachel speaks to chandler. q 2 follows a similar process, a model should be aware of that u 10 is a continuation of u 9 and solves the above coreference resolution problem as well. to tackle the aforementioned obstacles, we design a self-supervised speaker prediction task to implicitly model the speaker information flows, and a pseudo-self-supervised key-utterance prediction task to capture salient utterances in a long and noisy dialogue. in detail, the self-supervised speaker prediction task guides a carefully designed speaker information decoupling block (sidb, introduced in section 3.4) to decouple speaker-aware information, and the key-utterance prediction task guides a key-utterance information decoupling block (kidb, introduced in section 3.3) to decouple key-utterance-aware information. we finally fuse these two kinds of information and make final span prediction to get the answer of a question. to sum up, the main contributions of our method are three folds: ## conclusion in this paper, for multi-party dialogue mrc, we propose two novel self-and pseudo-self-supervised prediction tasks on speaker and key-utterance to capture salient clues in a long and noisy dialogue. experimental results on two multi-party dialogue mrc benchmarks, friendsqa and molweni, have justified the effectiveness of our model.