Keyphrase Generation with GAN s in Low-Resources Scenarios (2020)

Abstract: Keyphrase Generation is the task of predicting Keyphrases (KPs), short phrases that summarize the semantic meaning of a given document. Several past studies provided diverse approaches to generate Keyphrases for an input document. However, all of these approaches still need to be trained on very large datasets. In this paper, we introduce BeGanKP, a new conditional GAN model to address the problem of Keyphrase Generation in a low-resource scenario. Our main contribution relies in the Discriminator’s architecture: a new BERT-based module which is able to distinguish between the generated and humancurated KPs reliably. Its characteristics allow us to use it in a low-resource scenario, where only a small amount of training data are available, obtaining an efficient Generator. The resulting architecture achieves, on five public datasets, competitive results with respect to the state-of-the-art approaches, using less than 1% of the training data.

## introduction a keyphrase (kp) is a piece of text that conveys the main semantic meaning of a document. kps can be either present (or extractive) or absent (or abstractive): present kps are exact substrings of the document while absent kps are not. their automatic prediction is an important challenge for the community research as kps are a key component for a wide range of applications such as text summarization @xcite , opinion mining @xcite , document clustering @xcite , information retrieval @xcite ) and text categorization @xcite . historically, the first approaches focused on simply extracting substrings of the text to be used as keyphrases candidates @xcite @xcite . recently, the research community has focused on the broader task of keyphrase generation @xcite @xcite . keyphrase generation aims to produce a set of phrases that summarize the essential information in a given text, as opposed to simply look for them in the text. this allows for greater flexibility. several approaches introduced generative models based on the encoder-decoder architecture @xcite . this architecture works by compressing the contents of the input (e.g. the text document) into a hidden representation using an encoder module. the same representation is then decompressed using the decoder module, which returns the desired output (e.g. a sequence of kps). the modules are trained jointly to learn the best intermediate representation to perform this mapping. more recently, an approach based on gan (generative adversarial networks @xcite ) architecture has been proposed to address the task @xcite . although all these solutions achieved interesting results, they require a very large amount of data in order to be trained. our aim is to improve training efficiency, so that a model can be trained using only small subsets of the data. we focus our research in the generation of present kps and we propose a new conditional gan architecture for keyphrase generation that can be trained with a relatively small set of samples. the key component of our solution is the discriminator: a model based on bert that is able to distinguish between human and machinegenerated keyphrases leveraging on the language modelling information obtained from finetuning in a low-resource scenario. a reinforcement learning (rl) strategy is then used to train the generator, with rewards evaluated by the discriminator. this encourages the model to generate more accurate and relevant kps. thanks to the characteristics of our architecture, we are able to use only a small subset of the available data, using less than 1% of them to train our system. compared to all the previous approaches that needed to be fully trained on large set of training samples, our architecture greatly reduces required resources, while still providing competitive results in the generation of present kps. ## the proposed approach to generate present kps in a low-resource scenario we propose an approach based on the gan framework that we call began-kp. it mainly consists of three components: (1) a conditioned generator model that produces a set of kps, (2) a novel bert discriminator model that checks if the kps are fake (generated) or real (human-curated), and (3) the reinforcement learning (rl) module that is involved in the training process of the system as a whole (see figure 1 ). ## conclusion in this paper we introduced an approach to the task of present keyphrase generation in a low-resources scenario, began-kp. it is based on the gan framework with a novel bert based discriminator model, trained by mean of the reinforcement learning paradigm. it has been tested on five public datasets showing performances competitive with state-ofthe-art approaches while using less than 1% of the