GRASP : Guiding Model with R el A tional Semantics Using Prompt for Dialogue Relation Extraction (2022)

Abstract: The dialogue-based relation extraction (DialogRE) task aims to predict the relations between argument pairs that appear in dialogue. Most previous studies utilize fine-tuning pre-trained language models (PLMs) only with extensive features to supplement the low information density of the dialogue by multiple speakers. To effectively exploit inherent knowledge of PLMs without extra layers and consider scattered semantic cues on the relation between the arguments, we propose a Guiding model with RelAtional Semantics using Prompt (GRASP). We adopt a prompt-based fine-tuning approach and capture relational semantic clues of a given dialogue with 1) an argument-aware prompt marker strategy and 2) the relational clue detection task. In the experiments, GRASP achieves state-of-the-art performance in terms of both F1 and F1c scores on a DialogRE dataset even though our method only leverages PLMs without adding any extra layers.

## introduction the relation extraction (re) task aims to extract semantic relations from unstructured text such as a sentence, a document, or even a dialogue. re plays a critical role in information extraction and knowledge base construction as it can extract structured relational information @xcite . however, the utilization of sentence-level re in a conversational setting is limited because numerous relational facts appear across multiple sentences with more than one speaker in a dialogue @xcite . thus, the dialogue-based relation extraction (dialogre) task, which includes argument pairs and their corresponding relations, has been proposed to encourage building a model that captures the underlying se- in previous studies where state-of-the-art (sota) performance is achieved on dialogre benchmarks, fine-tuning is employed on pre-trained language models (plms) @xcite , such as bert @xcite and roberta @xcite . as fine-tuning requires the addition of extra layers on top of the plms and the training objectives are different from those used in the pre-training phase, plms cannot effectively exploit their learned knowledge in the downstream task, resulting in less generalized capability @xcite . to effectively utilize knowledge from plms, several studies involving prompt-based fine-tuning have been conducted. they employ the plm directly as a predictor and completing a cloze task @xcite jeff bezos is founder of amazon @xcite per:founded_by per:children per:friends cls head class set … we propose a method named guiding model with relational semantics using prompt (grasp) for dialogre. to maximize the advantages of the prompt-based fine-tuning approach for the dialogre task, we suggest an argument-aware prompt marking (apm) strategy and a relational clue detection (rcd) task. the apm strategy guides the model to the significant arguments scattered in the dialogue by carefully considering arguments. for our apm strategy, we conduct empirical study based on the diverse marker types to validate our apm strategy. along with the apm strategy, the suggested rcd task with a training objective leads the model to pay 1 in the dialogre dataset, 65.9% of relational triples involve arguments that never appear in the same utterance, demonstrating that multi-turn reasoning plays an important role. our contributions are as follows: the remainder of this paper is organized as follows. in section 3, we present the entire process of our method in detail. the experimental setup and the results are explained in section 4. the further analyses is provided in section 5, and section 6 presents the conclusions. appendix 2 provides related works including the prompt-based learning, and the dialogre. ## methodology an overview of grasp is illustrated in figure 2 . first, an input with a prompt template is constructed by using the apm strategy. then, the plm receives the constructed input for prompt-based fine-tuning and estimates probability distributions of the model's vocabulary by using contextualized representations of the plm. the rcd task lets the model predict the relational clue type of each token, and the model takes the [mask] representation to predict a final relation in the mlm task. in other words, our model is trained through multitask learning to encourage mutual communication between relational clues and the final relation for the argument pair. ## conclusion in this paper, we proposed grasp, which is a method for guiding plms to relational semantics using prompt-based fine-tuning for the dialogre task. we focused on alleviating the critical challenge in dialogues, that is, low information density, by effectively capturing relational clues. in grasp, we constructed prompts with deliberate initialization and suggested 1) an apm strategy considering both speaker and non-speaker arguments and 2) the rcd task, which guides the model to determine which token belongs to the relational clues. experimental results on the dialogre dataset revealed that grasp achieved sota performance in terms of the f1 and f1 c scores, even though our method only leveraged a plm without adding any extra layers.