Taxonomy-guided Semantic Indexing for Academic Paper Search (2024)

Abstract: Academic paper search is an essential task for efficient literature discovery and scientific advancement. While dense retrieval has advanced various ad-hoc searches, it often struggles to match the underlying academic concepts between queries and documents, which is critical for paper search. To enable effective academic concept matching for paper search, we propose Taxonomy-guided Semantic Indexing (TaxoIndex) framework. TaxoIndex extracts key concepts from papers and organizes them as a semantic index guided by an academic taxonomy, and then leverages this index as foundational knowledge to identify academic concepts and link queries and documents. As a plug-and-play framework, TaxoIndex can be flexibly employed to enhance existing dense retrievers. Extensive experiments show that TaxoIndex brings significant improvements, even with highly limited training data, and greatly enhances interpretability.

## introduction academic paper search is essential for efficient literature discovery and access to technical solutions. recently, dense retrieval has advanced in various ad-hoc searches @xcite . it encodes queries and documents as dense embeddings, measuring relevance by embedding similarity. these embeddings effectively capture textual meanings via pre-trained language models trained on massive corpora. while effective in general domains like web search, it often shows limitations in paper search @xcite . in paper search, it is crucial to match the underlying academic concepts between queries and documents, rather than relying on surface text and its meanings. academic concepts refer to fundamental ideas, theories, and methodologies that constitute * corresponding author the contents of papers. users often seek information on specific concepts when searching for papers. for example, consider the query "learning to win by reading manuals in a monte-carlo framework". this query encompasses various concepts: optimizing decision-making (learning to win), acquiring knowledge from text (reading manuals), and reinforcement learning using probabilistic sampling (monte-carlo). accordingly, retrievers should find papers that comprehensively cover these concepts. one critical limitation of existing dense retrievers is that such academic concepts are often not effectively captured, making them insufficiently considered in relevance prediction. identifying underlying concepts from surface text requires an inherent understanding of domain-specific contents, which is not sufficiently obtained from general corpora. this challenge is even greater for queries. as shown in the previous example, user queries often encompass various academic concepts in highly limited contexts. moreover, queries usually have different expression styles (e.g., terminology choice, language style) from documents, making it difficult to match common concepts. figure 1 (left) shows the results of a dense retriever for the query, which ranks an irrelevant document (paper a) at the top-1. although the paper mentions 'solving a goal' and 'comprehension of text', which have similar meanings to 'learning to win' and 'reading manuals' in the query, its focus is on text comprehension and dataset creation, which largely differs from the query concepts. this shows that the overall textual similarity captured by dense retrievers is insufficient to reflect specific academic concepts, leading to suboptimal results. to address this limitation, we introduce a new approach that extracts key concepts from papers in advance, and leverages this knowledge to incorporate academic concepts into relevance predictions. we construct a semantic index that stores semantic components best describing each paper. the pro-1 [paper a (irrelevant)] "a challenge dataset for the opendomain machine comprehension of text. we present mctest, a freely available set of stories and associated questions intended for research on the machine comprehension of text. previous work on … solving a more restricted goal …" @xcite ] "using reinforcement learning to learn how to play text-based games. the ability to learn optimal control policies in systems where action space is defined by sentences in natural language … optimisation of dialogue systems. text-based games with multiple endings …" dense retriever ( ): paper a (top-1), paper b (top-89) taxoindex offers several advantages for existing dense retrievers. first, taxoindex largely improves retrieval quality by effectively matching academic concepts. notably, index learning offers a new type 1 academic taxonomies are widely used for study categorization in various institutions (e.g., acm computing classification system) and can be readily obtained from the web. our primary contributions are as follows: (1) we propose taxoindex, which systematically constructs and leverages semantic index, for effective academic concept matching in paper search. (2) we design an index construction strategy that represents each paper at both topic and phrase levels, with the guidance of academic taxonomy. (3) we introduce an index learning strategy that allows for identifying the most related concepts from an input text. (4) we evaluate taxoindex with extensive quantitative and ablative experiments and comprehensive case studies. ## related work dense retrieval. the advancement of pre-trained language models (plms) has led to significant progress in dense retrieval. recent studies have enhanced retrieval quality through retrieval-oriented pre-training @xcite , advanced hard negative mining @xcite , and distillation from crossencoder @xcite . synthetic query generation has also been explored to supplement training data @xcite . figure 2 : a conceptual illustration of the taxonomy-guided semantic index construction. we extract and store core topics and indicative phrases that best represent each paper in the form of a forward index. on the other hand, many studies have focused on pre-training methods specialized for the academic domain. in addition to pre-training on academic corpora @xcite , researchers have exploited metadata associated with papers. cohan et al. (2020); ostendorff et al. (2022) use citations, liu et al. (2022) further utilizes venues, authors, and affiliations. mysore et al. (2022) uses co-citation contexts, and singh et al. (2023); zhang et al. (2023b) employs multi-task learning of tasks such as citation prediction and paper classification. complementary to the approach of leveraging such paper metadata, we focus on organizing and exploiting knowledge in the textual corpus. tax-oindex can be flexibly integrated to enhance the aforementioned models. indexing for dense retrieval. indexing refers to the process of collecting, parsing, and storing data to enhance retrieval @xcite . statistical and sparse retrieval often uses inverted indexes for term matching signals @xcite . dense retrieval relies on approximate nearest neighbor (ann) indexes to avoid costly brute-force searches. document embeddings are pre-computed offline, and ann indexes are constructed by techniques such as hashing @xcite , quantization @xcite , and clustering @xcite . our index is designed to extract academic concepts and leverage them to improve the accuracy of dense retrievers. as taxoindex encodes each text as an embedding, existing ann indexes can still be applied to accelerate search speed. enhancing retrieval with additional contexts. several studies have enhanced retrieval by providing supplementary contexts. our work falls into this direction. pseudo-relevance feedback @xcite @xcite utilizes the top-ranked results from an initial retrieval. recent generative approaches @xcite generate relevant contexts using plms. @xcite utilizes topic distri-butions of queries and documents. however, they are often limited in paper search due to the difficulty of generating proper domain-specific contexts. moreover, these contexts are obtained and added on-the-fly during inference, making it difficult to provide information tailored to backbone retriever. ## problem formulation academic taxonomy. an academic taxonomy t refers to a hierarchical tree structure outlining academic topics (figure 2 ). each node represents an academic topic, with child nodes corresponding to its sub-topics. widely used for study categorization in various institutions, academic taxonomies can be readily obtained from the web and automatically expanded by identifying new topics from a growing corpus @xcite . we utilize the fields of study taxonomy from microsoft academic @xcite , which covers 19 disciplines (e.g., computer science, biology). problem definition. to perform retrieval on a new corpus d, a plm-based dense retriever is typically fine-tuned using a training set of relevant querydocument pairs. our goal is to develop a plugand-play framework, which facilitates academic concept matching with the guidance of a given taxonomy t , to improve the backbone retriever. ## taxoindex framework we present taxonomy-guided index construction in §4.1, index-grounded fine-tuning in §4.2, and retrieval process with taxoindex in §4.3. ## conclusion we propose taxoindex to match academic concepts in paper search effectively. taxoindex extracts key concepts from papers and constructs a semantic index guided by an academic taxonomy. it then trains an add-on module to identify and incorporate these concepts, enhancing dense retrievers. extensive experiments show that taxoindex yields significant improvements, even with limited training data. we expect that taxoindex will effectively improve retrieval quality in various domains where underlying search intents are not sufficiently revealed by surface text. specifically, e-commerce is an interesting and promising domain. in this domain, users often express their information needs in various forms rather than searching by the ex-act product name. they might include desired attributes, characteristics, or even specific use cases. taxoindex can be applied to better capture users' search intents in such scenarios. we leave further investigation as future work.