DSP : Discriminative Soft Prompts for Zero-Shot Entity and Relation Extraction (2023)

Abstract: Prompt-based methods have shown their efficacy in transferring general knowledge within pre-trained language models (PLMs) for low-resource scenarios. Typically, prompt-based methods convert downstream tasks to cloze-style problems and map all labels to verbalizers.However, when applied to zero-shot entity and relation extraction, vanilla prompt-based methods may struggle with the limited coverage of verbalizers to labels and the slow inference speed. In this work, we propose a novel Discriminate Soft Prompts (DSP) approach to take advantage of the prompt-based methods to strengthen the transmission of general knowledge. Specifically, we develop a discriminative prompt method, which reformulates zero-shot tasks into token discrimination tasks without having to construct verbalizers.Furthermore, to improve the inference speed of the prompt-based methods, we design a soft prompt co-reference strategy, which leverages soft prompts to approximately refer to the vector representation of text tokens. The experimental results show that, our model outperforms baselines on two zero-shot entity recognition datasets with higher inference speed, and obtains a 7.5% average relation F1-score improvement over previous state-of-the-art models on Wiki-ZSL and FewRel.

## introduction zero-shot entity and relation extraction @xcite aim to extract novel entities and their relations by transferring semantic knowledge from seen classes to unseen ones. it is a fundamental problem in information extraction, which can be decomposed into two subtasks: zero-shot named entity recognition (zsner) @xcite and zero-shot relation extraction (zsre) @xcite . recent works @xcite focus on fine-tuning * corresponding author. ## method in this section, we first formally define the problem of zero-shot entity and relation extraction. then we introduce our discriminative soft prompts (dsp) method. the following is a detailed description. ## conclusion this paper presents a novel discriminative soft prompts method for zero-shot entity and relation extraction. unlike the cloze-style prompt method that converts a specific task into an mlm problem, we reformulate zsner and zsrc as a discriminative language modeling problem, which takes advantage of the prompt learning to strengthen the transmission of general knowledge without having to construct a verbalizer. furthermore, we propose a soft prompt co-reference strategy, which significantly improves the inference efficiency of the discriminative prompt method. experiments on four datasets demonstrate the effectiveness of our model in both zsner and zsre tasks. also, the inference speed of the dsp is up to 120 times faster than the cloze-style prompt method on zsner.