A Pilot Study on Dialogue-Level Dependency Parsing for C hinese (2023)

Abstract: Dialogue-level dependency parsing has received insufficient attention, especially for Chinese. To this end, we draw on ideas from syntactic dependency and rhetorical structure theory (RST), developing a high-quality human-annotated corpus, which contains 850 dialogues and 199,803 dependencies. Considering that such tasks suffer from high annotation costs, we investigate zero-shot and few-shot scenarios. Based on an existing syntactic treebank, we adopt a signal-based method to transform seen syntactic dependencies into unseen ones between elementary discourse units (EDUs), where the signals are detected by masked language modeling. Besides, we apply single-view and multi-view data selection to access reliable pseudo-labeled instances. Experimental results show the effectiveness of these baselines. Moreover, we discuss several crucial points about our dataset and approach.

## introduction as a fundamental topic in the natural language processing (nlp) community, dependency parsing has drawn a great deal of research interest for decades @xcite @xcite . the goal of dependency parsing is to find the head for each word and the corresponding relation @xcite . most of previous works have focused on the sentence level, while the dialogue-level dependency parsing still stands with the paucity of investigation. prior studies build dialogue-level discourse parsing datasets @xcite with reference to the text-level discourse dependency @xcite . the discourse structures in these data are constructed by elementary discourse units (edus) and the relationships between them, without regard to the inner structure of edus. it is of interest to incorporate both inner-edu and the scarcity of research on dialogue-level dependency parsing might be caused by the prohibitive cost of annotation. thus, we focus on low-resource settings, aiming to construct a sufficient test set to support reliable evaluation and label a small training set. we craft an annotation guideline and develop a platform for human labeling. after that, we perform manual annotation and obtain 50 training instances and 800 test samples. figure 1 illustrates a fragment in a labeled dialogue, which contains three utterances with dependency links. learning a parser with scant to no supervised signals can be challenging. fortunately, it is straight-forward to use an existing syntactic treebank to attain inner-edu dependencies. meanwhile, we find some overlap between syntactic dependency and inter-edu dependency. as shown in figure 1 , "offer" is dependent on "see" with the "dfsubj" (different subject) relationship in syntactic dependency, which matches the "attr" (attribution) of inter-edu dependency. furthermore, inter-edu arcs between utterances are often emitted from the root node above to the root below. hence, we can naturally obtain inter-edu arcs from partial syntactic dependencies. we find that certain words can reflect the discourse role of edus, helping to assign labels to inter-edu arcs. for instance, "see" and "if" reflect the "attribution" and "condition" signals respectively, illustrated in figure 1 . thus, we propose a method to discover and leverage these signals for inter-edu label assignment. inspired by promptbased learning @xcite , we adopt masked language modeling (mlm) to recover signal words, the set of which is carefully defined in advance. predicted words are then mapped to corresponding signals. based on these signals, a handful of simple well-defined rules can infer credible dependencies. the gap between the syntactic treebank and dialogue-level dependencies goes beyond labels; differences in text style expose a trained parser to the out-of-distribution (ood) effect @xcite . to alleviate that, we utilize unlabeled dialogue data to bridge the cross-domain gap. we borrow ideas from self-training @xcite and co-training @xcite , applying single-view and multi-view data filtering augmentation. given pseudo-labeled samples predicted by parsers, we calculate their confidence scores and set a threshold. instances above the threshold are provided as additional training samples. we carry out experiments in zero-shot and fewshot scenarios. the empirical results suggest that our signal-based baseline can achieve reasonable performance in the zero-shot condition and provide significant improvement in the few-shot setting. incorporating filtered pseudo-labeled data further advances performance. in addition, we present several discussion topics, which can affirm directive intuitions and crucial practices in this work. our contributions are mainly in three respects: ## dataset construction to facilitate our research on dialogue-level dependency parsing, here we construct a high-quality corpus manually, named the chinese dialogue-level dependency treebank (cddt). the treebank borrows the sentence-level dependency @xcite and the discourse-level dependency @xcite foot_0 , extending both to the dialogue texts. below, we present the annotation details. ## methodology dialogue-level dependency parsing involves two folds, namely inner-edu dependency parsing and inter-edu dependency parsing. we leverage an existing syntactic treebank s @xcite to learn a parser, which can analyze inner-edu dependencies well. meanwhile, it can be observed that syntactic dependencies have the potential to be converted into inter-edu ones (figure 1 ). thus, we propose a signal-based method to perform dependency transformation for parser training and inference. moreover, we apply pseudo-labeled data filtering to leverage the large-scale unlabeled dialogue data d. ## discussion what are the advantages of introducing discourse dependency compared to only syntax? figure 1 has illustrated the superiority of introducing discourse dependency in two aspects. relationships between utterances can be expressed by discourse dependency. also, it represents a hierarchical structure and enables a more nuanced connection between edus. figure 2 gives another sample, the syntactic dependency suffers from the problem of somorphism. those two sentences contain different semantic information, while their two syntactic subtrees are simply connected by "sasubj", resulting in their highly analogous architecture. according to rst dependency, the two subtrees of the first sentence are linked by "cond" (condition) from right to left, and the two below are connected by "elbr" (elaboration) from left to right. the inclusion of discourse dependency can alleviate the dilemma of somorphism by expressing high-level hierarchies. how much overlap between partial syntactic dependencies and inter-edu ones? figure 1 pro- we present a matching score to measure the extent of overlap. we first obtain syntactic predictions by parser-s. then we replace those dependencies that bear a specific syntactic label with an inter-edu class. next, we compute the las for that class, as the matching score between the syntactic and inter-edu labels. figure 3 illustrates the top-5 matching scores of syntactic labels "root", "sasubj", and "dfsubj". it can be seen that these syntactic labels have a respectable consistency with certain inter-edu classes, revealing the potential for dependency conversion. interestingly, different syntactic labels are matched with different inter-edu classes, indicating the overlap of specific syntactic structures with certain discourse structures. how broadly can inter-edu signals be reflected by pre-defined signal words? as mentioned above, certain words can reflect the semantic role of edu. we calculate the consistency of signal words and the relationship labels on edus to quantify the extent. given an edu, we directly assign its label by the signal corresponding to the pre-defined signal word it contains. then, we compute the accuracy of each inter-edu label as the matching score of the relevant signals. it can be observed in figure 4 that some labels such as "elbr" ( 27 ), "qst-ans" (37), and "stm-rsp" ( 38 ) can be strongly reflected by the pre-defined signal words. some labels that do not contain significant signals in edus are maintained at low scores, especially "bckg" (22) and "topic-chg" (35), indicating that there is much room for improvement. is the mlm-based signal detection method able to detect implicit signal? our inter-edu signal detection method applies an mlm paradigm, which tries to recover the dropped signal word during the training stage. intuitively, this method should have the capacity to predict those implicit signals. figure 5 gives two examples to prove that. it can be seen that even when "if" is removed, the approach of mlm-based signal detection can still predict the "cond" signal, which is denoted as a 25 number. based on this, the parser can correctly predict the "cond" relations between two edus. how does the reserved size of pseudo-labeled data change by confidence thresholds? we vary the threshold ϵ from 0.5 to 0.98 to investigate the changes in the amount of data selected. figure 6 illustrates the trend. interestingly, the size of selected data is still large even though the threshold is greater than 0.9, illustrating the high confidence in dependency parsing. it can also be observed that the increase in merged data becomes more pronounced as the threshold increases. intuitively, as the amount of data selected decreases, fewer data will overlap. why not leverage chinese rst treebanks to train a parser? to date, there exist two chinese rst treebanks, the rst spanish-chinese treebank @xcite and gcdt @xcite . it is natural to use them to provide supervised signals for inter-edu parsing. in our prior experiments, we find that directly using these treebanks for training a parser leads to unsatisfactory performance. one possible reason is that their annotation style is not aligned with ours. for instance, gcdt splits sentences very finely and uses lots of "same-unit" relations to align to the english style, fitting their multilingual tasks. this occurs not in our data as it tends to cause an incomplete structure within the edu. furthermore, gcdt lacks some of the label categories in our data, especially some common relations such as "statement-response". in addition, inconsistencies in data distribution can also contribute to underperformance. figure 3 : top-5 matching labels and scores of "root", "sasubj", and "dfsubj". ## related work dependency parsing. to date, there are several chinese dependency paradigms and the corresponding treebanks @xcite @xcite . these works mostly focus on sentence-level dependency parsing, while the document-level one is conspicuous by its paucity. @xcite adopt a dependency parsing paradigm for discourse parsing, while its edu-wise pattern neglects to parse inside edus. we propose a unified schema, which includes both word-wise dependencies within and between edus that organize whole dialogues into tree structures. dialogue parsing. discourse structures can be expressed by several theories, e.g., @xcite , sdrt @xcite , and ptdb @xcite . the investigation of dialogue-level discourse parsing is still in its early stage. @xcite ; @xcite build an annotated corpus stac based on sdrt @xcite , using a dependency paradigm for annotation of relationships between edus. @xcite present the molweni dataset, contributing discourse dependency annotations. these two data are annotated at the granularity of edus and, to accommodate multi-party dialogue scenarios, their annotations are based on sdrt that fits the graph structure. differently, in line with common syntactic dependencies, the inter-edu part of our dialogue-level dependency is word-wise and references rst @xcite , which organizes a text into the tree structure. furthermore, the prevalent corpus for dialogue dependency parsing is in english, and our dataset fills a lacuna in the chinese corpus. weakly supervised learning. it is challenging to predict new classes that are unobserved beforehand @xcite . plms @xcite @xcite can address this challenge through language model-ing combined with label mapping, and promptbased learning can stimulate this ability @xcite @xcite . inspired by that, we adopt an mlm-based method to sense inter-edu signals and map them to unseen dependencies. furthermore, our approach employs single-view and multi-view data selection, borrowing from self-training @xcite and co-training @xcite , which are used for growing a confidently-labeled training set. ## conclusion we presented the first study of chinese dialoguelevel dependency parsing. first, we built a highquality treebank named cddt, adapting syntactic @xcite and rst @xcite dependencies into inner-edu and inter-edu ones. then, we conducted manual annotation and reached 850 labeled dialogues, 50 for training and 800 for testing. to study low-resource regimes, we leverage a syntactic treebank to get inner-edu dependencies and induce inter-edu ones. we employed an mlm method to detect the inter-edu signals of each edu and then assign the detected signals to the arcs between edus. furthermore, we exploited single-view and multi-view approaches for pseudo-labeled sample filtering. empirical results suggest that our signal-based method can achieve respectable performance in zero-shot and few-shot settings, and pseudo-labeled data utilization can provide further improvement.