A Preliminary Exploration of GAN s for Keyphrase Generation (2020)

Abstract: We introduce a new keyphrase generation approach using Generative Adversarial Networks (GANs). For a given document, the generator produces a sequence of keyphrases, and the discriminator distinguishes between human-curated and machine-generated keyphrases. We evaluated this approach on standard benchmark datasets. We observed that our model achieves state-of-the-art performance in the generation of abstractive keyphrases and is comparable to the best performing extractive techniques. Although we achieve promising results using GANs, they are not significantly better than the state-of-the-art generative models. To our knowledge, this is one of the first works that use GANs for keyphrase generation. We present a detailed analysis of our observations and expect that these findings would help other researchers to further study the use of GANs for the task of keyphrase generation.

## introduction keyphrases capture the most salient topics of a document and are often indexed in databases to help with search and information retrieval techniques. researchers tag their scientific publications with high-quality keyphrases to ensure discoverability in scientific repositories. automatic identification of keyphrases is of great interest to the scientific community as it helps to recommend relevant articles, suggest missing citations to authors, identify potential peer reviewers, and analyze research trends @xcite . keyphrases could either be extractive (part of the document) or abstractive (not part of the document). some prior works have referred to them as present and absent keyphrases, respectively. keyphrase generation is the process of predicting both extractive and abstractive keyphrases from a given document. most of the previous works in keyphrase domain, including both supervised and unsupervised techniques, primarily focus on extractive keyphrases @xcite @xcite . recent studies @xcite ; @xcite ; @xcite have started to develop generative approaches that produce both abstractive and extractive keyphrases from documents. though these studies have shown some promise, the results suggest that there is great room for improvement. most of the supervised natural text generation approaches use maximum likelihood estimation (mle) objective @xcite . however, mle techniques have often been observed not to be generating satisfactory text because of exposure bias @xcite . generative approaches based on reinforcement learning (rl) or adversarial training have been shown to address some of the challenges of exposure bias. one such example is the success in the field of summary generation using gans @xcite . driven by these developments, we posit that, as with summarization, keyphrase generation can also benefit from the use of gans. to pursue this hypothesis, in this paper, we propose a new gan architecture for keyphrase generation where the generator produces a sequence of keyphrases from a given document, and the discriminator distinguishes between human-curated and machine-generated keyphrases (section 2). we introduce an adversarial training setup for the state-of-the-art generative approaches used in @xcite (section 3). we also propose a hierarchical attention-based discriminator architecture. our empirical analysis shows that the generative models, in gan setup, improves generation of abstractive keyphrases but do not show any significant improvements for extractive keyphrases. as concluded by (c ¸ano and bojar, 2019), we think keyphrase generation is still a hard text summarization challenge. we share our observations as well as our implementations with the scientific community. we expect these findings would help other researchers further explore the application of gans for keyphrase generation. following are our main contributions: ## methodology given a document @xmath0, x 2 , ..., x n }, where x i is the i th token, the problem of keyphrase generation is to generate a set of keyphrases @xmath1, y 2 , ..., y m } that best capture the semantic meaning of t . in this paper, we approach this as a supervised problem solved specifically using gan @xcite . our gan model consists of a generator g trained to produce a sequence of keyphrases from a given document, and a discriminator d that learns to distinguish between machine-generated and human-curated keyphrases. adversarial learning for text is a challenging task as it is not straight-forward to back-propagate the loss of the discriminator due to the discrete nature of text data @xcite . we use rl to address this issue, where the generator is treated as an rl agent, and its rewards are obtained from the 1 code - https://github.com/avinsit123/ keyphrase-gan discriminator's outputs. fig 1 , shows an overview of our framework. ## conclusions and future work in this paper, we proposed the first gan architecture for keyphrase generation. the model consists of a generator that produces a sequence of