Unsupervised Domain Adaptation for Keyphrase Generation using Citation Contexts (2024)

Abstract: Adapting keyphrase generation models to new domains typically involves few-shot fine-tuning with in-domain labeled data. However, annotating documents with keyphrases is often prohibitively expensive and impractical, requiring expert annotators. This paper presents silk, an unsupervised method designed to address this issue by extracting silver-standard keyphrases from citation contexts to create synthetic labeled data for domain adaptation. Extensive experiments across three distinct domains demonstrate that our method yields high-quality synthetic samples, resulting in significant and consistent improvements in in-domain performance over strong baselines.

## conclusions we have introduced an approach to learn topicsensitive word representations that exploits the document-level context of words and does not require annotated data or linguistic resources. our evaluation on the lexical substitution task suggests that topic distributions capture word senses to some extent. moreover, we obtain statistically significant improvements in the lexical substitution task while not using any syntactic information. the best results are achieved by our hard topiclabeled model which learns topic-sensitive representations by assigning topics to target words. if adding depth to our context-sensitive autoencoder will improve its performance in the contextual word similarity task. figure 6 shows that as we increase the depth of our autoencoders, their performances initially improve. the csae-lgc model that uses both local and global context benefits more from greater number of hidden layers than csae-lc that only uses local context. we attribute this to the use of global context in csae-lgc that leads to more accurate representations of words in their context. we also note that with just a single hidden layer, csae-lgc largely improves the performance as compared to csae-lc. ji et al. (2016) presented a context-sensitive rnn-based language model that integrates representations of previous sentences into the language model of the current sentence. they showed that this approach outperforms several rnn language models on a text coherence task. @xcite proposed a context-sensitive rnn model that uses latent dirichlet allocation @xcite to extract topic-specific word embeddings. their best-performing model regards each topic that is associated to a word in a sentence as a pseudo word, learns topic and word embeddings, and then concatenates the embeddings to obtain topic-specific word embeddings. mikolov and zweig (2012) extended a basic rnn language model @xcite by an additional feature layer to integrate external information (such as topic information) about inputs into the model. they showed that such information improves the perplexity of language models. in contrast to previous research, we integrate context into deep autoencoders. to the best of our knowledge, this is the first work to do so. also, in this paper, we depart from most previous approaches by demonstrating the value of context information in sentence-level semantic similarity and ranking tasks such as qa ranking tasks. our approach to the ranking problems, both for answer ranking and question ranking, is different from previous approaches in the sense that we judge the relevance between inputs based on their context information. we showed that adding sentential or document context information about questions (or answers) leads to better rankings.