Enhancing Discourse Dependency Parsing with Sentence Dependency Parsing: A Unified Generative Method Based on Code Representation (2024)

Abstract: Due to the high complexity of Discourse Dependency Parsing (DDP) tasks, their existing annotation resources are relatively scarce compared to other NLP tasks, and different DDP tasks also have significant differences in annotation schema. These issues have led to the dilemma of low resources for DDP tasks. Thanks to the powerful capabilities of Large Language Models (LLMs) in cross-task learning, we can use LLMs to model dependency parsing under different annotation schema in an unified manner, in order to alleviate the dilemma of low resources for DDP tasks. However, enabling LLMs to deeply comprehend dependency parsing tasks is a challenge that remains underexplored. Inspired by the application of code-based methods in complex tasks, we propose a code-based unified dependency parsing method. We treat the process of dependency parsing as a search process of dependency paths and use code to represent this search process. Furthermore, we use a curriculum-learning based instruction tuning strategy for joint training of multiple dependency parsing tasks. The experimental results show that our proposed code-based DDP system has achieved good performance on two Chinese DDP tasks (especially significant improvement on the DDP task with relatively less training data).

## introduction discourse dependency parsing (ddp) is a structured prediction task. the input of this task is a sequence of elementary discourse units (edus), and the output of this task is a dependency structure (a dependency tree labeled with discourse relation labels). as one of the fundamental tasks in the field of natural language processing (nlp), ddp can assist downstream tasks such as emotion recognition @xcite , pronoun resolution (yang * corresponding author et al., 2021), and event relation extraction @xcite . although the neural network-based nlp technology has greatly promoted the development of ddp tasks @xcite . however, there are still some unresolved issues with ddp tasks. firstly, the cost of annotating ddp data is relatively high, compared to sentencelevel dependency parsing (sdp) data, the existing scale of ddp data is smaller. secondly, there are large differences in the annotation schema of ddp tasks, and heterogeneous data under different annotation schema are difficult to be effectively utilized @xcite . these problems have led to the dilemma of low resources in ddp tasks, limiting the application and development of ddp tasks. recently, large language models (llms) has shown significant advantages in few-shot learning @xcite and cross-task learning scenarios @xcite due to its powerful semantic understanding ability and transfer learning ability. therefore, using llm to design an unified dependency parsing method helps to break through the gap between dependency parsing tasks under different annotation granularity and different annotation schema, thereby overcoming the the dilemma of low resources in ddp tasks. however, how to more effectively use llms for dependency parsing tasks remains an issue that requires more attention. specifically, we need to face challenges from the following two aspects. 1. how to linearize dependency structures to suit the modeling method of language models? 2. how to design unified and llm-friendly prompts for dependency parsing tasks? for the first challenge, we propose a path decomposition-based strategy for linearizing dependency structures. inspired by related work on chain of thought technology @xcite in complex tasks, we treat the dependency parsing task as a step-by-step search process of dependency paths. the model needs to gradually search out the entire dependency structure from the root node according to some search order, such as depth-first search (dfs) or breadthfirst search (bfs). this strategy not only adapts to the modeling method of language models but also retains the structural information in the dependency structure. for the second challenge, inspired by llmrelated works in the field of information extraction @xcite . we proposed a code-based representation method for dependency parsing tasks. firstly, we use code language to represent the basic elements in dependency parsing tasks. specifically, we use variables to represent the sequence of semantic units and semantic relation labels; classes to represent dependency arcs and their constituent head nodes, dependent nodes, and dependency relations; functions to represent some specific operations in the process of dependency analysis. then, based on these basic elements, we can transform the process of dependency parsing into the process of code execution. this method can not only use code language to describe the process of dependency parsing clearly and concisely but also leverage the powerful underlying code generation capability of llms. we attempt to model the sdp task and ddp task in an unified way, in order to achieve the goal of enhancing the ddp task using the sdp task. specifically, we set the sdp task (we chose a syntactic dependency parsing task and a semantic dependency parsing task) as the source task, and set the ddp task as the target task. leveraging our proposed dependency structure linearization strategy and the code-based dependency parsing task representation method, we are able to craft unified dependency parsing prompts for these tasks, and employ instruction tuning to equip llms with dependency parsing capabilities. to achieve better performance on ddp tasks, we further adopt a curriculum learning-based (clbased) instruction tuning strategy. specifically, we first calculate a difficulty value for each sample in the dependency parsing task based on prior features such as the type of dependency parsing task, the number of dependency paths, and the average depth of dependency paths. then, we dynamically adjust the loss weight of each sample during the entire training process according to the difficulty value of the sample and the current training steps, so as to achieve a training mode that learns from simple samples first, and then learns from difficult samples. our contributions can be summarized in the following three aspects: 1) we first propose to treat the dependency parsing process as a search process of dependency paths and design a code-based unified dependency parsing task prompt to represent this process. 2) we use a curriculum learning-based instruction tuning strategy to jointly train sdp tasks and ddp tasks, aiming to enhance the performance of ddp tasks with sdp tasks. 3) our experimental results show that our code-based ddp system has achieved good performance in two chinese ddp tasks, confirming the effectiveness of our proposed method. ## conclusion in order to better utilize llms to alleviate the low-resource dilemma of ddp tasks, we propose a code-based unifed dependency parsing method. specifically, we regard the process of dependency parsing as a search process of dependency paths and design a code-based unifed dependency parsing task prompt based on this search process. furthermore, we use a curriculum learning-based instruction tuning strategy to jointly train sdp tasks and ddp tasks under different annotation shema, achieving the goal of using sdp tasks to enhance ddp tasks. experimental results show that our code-based ddp system can take advantage of llm's strengths in code understanding, enabling it to achieve good performance on ddp tasks.