Generative Prompt Tuning for Relation Classification (2022)

Abstract: Using prompts to explore the knowledge contained within pre-trained language models for downstream tasks has now become an active topic. Current prompt tuning methods mostly convert the downstream tasks to masked language modeling problems by adding cloze-style phrases and mapping all labels to verbalizations with fixed length, which has proven effective for tasks with simple label spaces. However, when applied to relation classification exhibiting complex label spaces, vanilla prompt tuning methods may struggle with label verbalizations with arbitrary lengths due to rigid prompt restrictions. Inspired by the text infilling task for pre-training generative models that can flexibly predict missing spans, we propose a novel generative prompt tuning method to reformulate relation classification as an infilling problem, which frees our approach from limitations of current prompt based approaches and thus fully exploits rich semantics of entity and relation types. In addition, we design entity-guided decoding and discriminative relation scoring to generate and align relations effectively and efficiently during inference. Extensive experiments under fully supervised settings and low-resource settings demonstrate the effectiveness of our approach.

## introduction relation classification (rc) is a fundamental task in natural language processing (nlp), aiming to detect the relations between the entities contained in a sentence. with the rise of a series of pre-trained language models (plms) @xcite @xcite , fine-tuning plms has become a dominating approach to rc @xcite @xcite . however, the significant objective gap between pre-training and fine-tuning may hinder the full potential of pre-trained knowledge for such a downstream task. ## approach as presented in figure 2 (c ), this paper considers relation classification as a text infilling style task, which takes the sequence t (x) processed by the template as source input and outputs a target sequence y to predict relations. the problem definition is formally given in section 3.1. we first introduce how to construct entity-oriented prompts in section 3.2, and then present the model and training objective in section 3.3. the inference details including entity-guided decoding and relation scoring are discussed in section 3.4. ## results and discussion in this section, we conduct extensive experiments and answer the following questions: 1) how does our proposed method perform? we demonstrate the superiority of our approach under extensive datasets and settings. 2) what is the impact of prompt design? we conduct extensive experiments to investigate the influence of different prompts. 3) how much benefit can label semantics bring to the rc task? we carry out experiments to illustrate the impact of label semantics on the performance of re. 4) what is the effect of the proposed decoding strategy? we perform ablation and case studies to prove the effectiveness and efficiency of decoding strategies. ## conclusion this paper presents a novel generative prompt tuning method for rc. unlike vanilla prompt tuning that converts a specific task into an mlm problem, we reformulate rc as a text infilling task, which can predict label verbalizations with varying lengths at multiple predicted positions and thus better exploit the semantics of entity and relation types. in addition, we design a simple yet effective entity-guided decoding and discriminative scoring strategy to efficiently decode predicted relations, making our generative model more practical. qualitative and quantitative experiments on four benchmarks prove the effectiveness of our approach. future work includes applying the proposed method to other related tasks where the label space is large with rich semantic information. in addition, we believe genpt can be adapted to support multiple labels for the same class. this is analogous to the typical nmt model involving manyto-many mapping (where the target may not be unique). how to exactly introduce multiple valid, semantically equivalent label descriptions (including design of label verbalizations, learning process, etc.) is a research topic deserving a separate study.