Zero-shot Cross-lingual Conversational Semantic Role Labeling (2022)

Abstract: While conversational semantic role labeling (CSRL) has shown its usefulness on Chinese conversational tasks, it is still under-explored in non-Chinese languages due to the lack of multilingual CSRL annotations for the parser training. To avoid expensive data collection and error-propagation of translation-based methods, we present a simple but effective approach to perform zero-shot cross-lingual CSRL.Our model implicitly learns language-agnostic, conversational structure-aware and semantically rich representations with the hierarchical encoders and elaborately designed pre-training objectives. Experimental results show that our model outperforms all baselines by large margins on two newly collected English CSRL test sets. More importantly, we confirm the usefulness of CSRL to non-Chinese conversational tasks such as the question-in-context rewriting task in English and the multi-turn dialogue response generation tasks in English, German and Japanese by incorporating the CSRL information into the downstream conversation-based models. We believe this finding is significant and will facilitate the research of non-Chinese dialogue tasks which suffer the problems of ellipsis and anaphora.

## introduction conversational semantic role labeling (csrl) @xcite is a recently proposed dialogue understanding task, which aims to extract predicateargument pairs from the entire conversation. figure 1 illustrates a csrl example where a csrl parser is required to identify "《泰坦尼克号》(titanic)" as the arg1 argument of the predicate "看 过 (watched)" and the arg0 argument of the predicate "是 (is)". we can see that in the original conversation, "《泰坦尼克号》(titanic)" is omitted in the second turn and referred as "这 (this)" in the last turn. by recovering the dropped and referred components in conversation, csrl has shown its usefulness to a set of chinese dialogue tasks, including multi-turn dialogue rewriting @xcite and response generation @xcite . however, there remains a paucity of evidence on its effectiveness towards non-chinese languages owing to the lack of multilingual csrl models. to adapt a model into new languages, previous solutions can be divided into three categories: 1) manually annotating a new dataset in the target language @xcite ) 2) borrowing machine translation and word alignment techniques to transfer the dataset from the source language into the target language @xcite 3) zero-shot transfer learning with multilingual pre-trained language model @xcite . due to the fact that manually collecting annotations is costly and translation-based methods might introduce translation or word alignment errors, zeroshot cross-lingual transfer learning is more practical to the nlp community. recent works have witnessed prominent performances of multilingual pre-trained language models (prlms) @xcite on cross-lingual tasks, including machine translation @xcite , semantic role labeling (srl) @xcite and semantic parsing @xcite . how-ever, cross-lingual csrl, as a combination of three challenging tasks (i.e., cross-lingual task, dialogue task and srl task), suffers three outstanding difficulties: 1) latent space alignment -how to map word representations of different languages into an overlapping space; 2) conversation structure encoding -how to capture high-level dialogue features such as speaker dependency and temporal dependency; and 3) semantic arguments identification -how to highlight the relations between the predicate and its arguments, wherein prlms can only partially encode multilingual inputs to an overlapping vector space. although there are some success that can separately achieve structural conversation encoding @xcite and semantic arguments identification @xcite , a unified method for jointly solving these problems is still under-explored, especially in a cross-lingual scenario. in this work, we summarize our contributions as follows: (1) we propose a simple but effective model which consists of three modules, namely cross-lingual language model (clm), structureaware conversation encoder (sa-encoder) and predicate-argument encoder (pa-encoder), and five well-designed pre-training objectives. our model implicitly learns language-agnostic, conversational structure-aware and semantically rich representations to perform zero-shot cross-lingual csrl. (2) experiments show that our method achieves impressive cross-lingual performance on the language pair (@xmath0, and outperforms all baselines on the two newly collected english csrl test sets. (3) we confirm the usefulness of csrl to the questionin-context rewriting task in english and multi-turn response generation tasks in english, german and japanese. we believe this finding is important and will facilitate the research of non-chinese dialogue tasks that suffer from ellipsis and anaphora. (4) we release our code, the new annotated english csrl test sets and checkpoints of our best models to facilitate the further research at https://github. com/hahahawu/zero-shot-xcsrl . conversational semantic role labeling. while ellipsis and anaphora frequently occur in dialogues, @xcite observed that most of the dropped or referred components can be found in dialogue histories. following this observation, they proposed conversational semantic role labeling (csrl) which required the model to find predicateargument structures over the entire conversation instead of a single sentence. in this way, when analyzing a predicate in the latest utterance, a csrl model needs to consider both the current turn and previous turns to search potential arguments, and thus might recover the omitted components. furthermore, @xcite also confirmed the usefulness of csrl to chinese dialogue tasks by applying csrl information into downstream dialogue tasks. however, there are still two main problems to be solved for csrl task: (1) the performance of current state-of-the-art csrl model @xcite is still far from satisfactory due to the lack of high-level conversational and semantic features modeling; (2) the usefulness of csrl to conversational tasks in non-chinese languages has not been confirmed yet due to the lack of crosslingual csrl models. in this work, we primarily focus on the latter problem and propose a simple but effective model to perform cross-lingual csrl. we would like to distinguish our work from the work @xcite which purely focuses on improving the monolingual csrl performance where they try to model predicate-aware representations. this solution could benefit to monolingual csrl task, but hurt the cross-lingual performance, because the relative positions of the predicates may differ from language to language. ## methodology following @xcite , we solve the csrl task as a sequence labeling problem. formally, given a dialogue @xmath1, u 2 , ..., u n } of n utterances, where u @xmath2, w i 2 , ..., w i |u i | } consisting of a sequence of words, and a predicate indicator @xmath3, ..., p i k , ..., p n |u n | ## conclusion in this work, we propose a simple but effective model with five pre-training objectives to perform zero-shot cross-lingual csrl, and also confirm the usefulness of csrl to non-chinese dialogue tasks.