KTRL + F : Knowledge-Augmented In-Document Search (2024)

Abstract: We introduce a new problem KTRL+F, a knowledge-augmented in-document search that necessitates real-time identification of all semantic targets within a document with the awareness of external sources through a single natural query. KTRL+F addresses following unique challenges for in-document search: 1) utilizing knowledge outside the document for extended use of additional information about targets, and 2) balancing between real-time applicability with the performance.We analyze various baselines in KTRL+F and find limitations of existing models, such as hallucinations, high latency, or difficulties in leveraging external knowledge. Therefore, we propose a Knowledge-Augmented Phrase Retrieval model that shows a promising balance between speed and performance by simply augmenting external knowledge in phrase embedding. We also conduct a user study to verify whether solving KTRL+F can enhance search experience for users. It demonstrates that even with our simple model, users can reduce the time for searching with less queries and reduced extra visits to other sources for collecting evidence. We encourage the research community to work on KTRL+F to enhance more efficient in-document information access.

## introduction despite significant advancement in many natural language processing applications, facilitated by transformer-based models @xcite , real-time in-document search still leans heavily on conventional lexical matching tools like the "find" function (ctrl+f) and regular expressions. these tools, while fast, have clear limitations, especially with ambiguous keywords or multiple targets. machine reading comprehension (mrc) seems a promising solution to these issues. it reads documents, comprehends their context, and answers questions @xcite . however, mrc focuses on explicit contents, limiting its value when users need knowledge not directly in the document @xcite @xcite . consider a scenario where users read a news article and seek for information on the "social network platform of china." (figure 1 ). typically, users refer to external sources such as wikipedia to gather additional details not explicitly mentioned in news related to candidates, such as wechat, baidu, and twitter. an alternative is harnessing the capabilities of powerful pre-trained language models @xcite . however, their generative nature poses challenges for real-time search task. to overcome the limitations of previous methods and enhance the efficiency and comprehensiveness of in-document search, we present a new problem ktrl+f (knowledge-augmented in-document search). this task aims to reduce redundancy and better meet the requirements of real users. given a natural language query and a long input document, ktrl+f is designed to fulfill three key criteria: (req 1) find all semantic targets. (req 2) utilizes external knowledge. (req 3) operates in real-time. in the absence of a suitable dataset to evaluate ktrl+f, we curate a new dataset with unique queries demanding matching external evidence. to measure model performance in ktrl+f, we introduce a set of reformulated metrics tailored to measure processing speed while maintaining robust and high performance. we conduct an extensive analysis of various baselines for ktrl+f and find several limitations including hallucination, slow speed with generative models, and challenges in incorporating external knowledge into mrc models (see §6.2 for details). to strike a balance between real-time processing speed and achieving high performance through effective utilization of additional knowledge, we introduce a simple yet effective extension of phrase retrieval @xcite : knowledge-augmented phrase retrieval. this model seamlessly extends the phrase retrieval to cater to in-document search scenarios, all while integrating external knowledge without the need for additional training steps. our experiments support that by simply adding the knowledge embedding and the phrase embedding, knowledge-augmented phrase retrieval exhibits the potential to reflect external knowledge without sacrificing latency. furthermore, we conduct a user study to show the necessity of ktrl+f utilizing a chrome extension plugin that operates in the real web environments, built upon our model. results of the study demonstrate that search experience of users can be enhanced even with our simple model with seamless access to external knowledge during indocument searches. we encourage the research community to take on the unique challenge of solving ktrl+f requiring balance between performance and speed to enhance more efficient and effective information access. knowledge-augmented information retrieval is an approach to enrich external information within the text embedding. the introduction of a knowledge-augmented design aims to supplement deficient contextual information, thereby enhancing the richness of text embedding. numerous studies tackle knowledge augmentation across various nlp tasks @xcite @xcite @xcite @xcite . the integration of information from diverse sources leads to an improved language understanding ability. however, the application of knowledge augmentation in information retrieval tasks has received comparatively less attention. @xcite attempts to improve text embeddings for retrieval by enriching context information through embeddings derived from a given context, without specifically focusing on external knowledge. meanwhile, @xcite utilizes contextualized embeddings as vocabulary embeddings for text tokens in a generative retriever, thereby enhancing contextual information for basic text tokens. additionally, raina et al. (2023) focuses on the retrieval augmented text embedding to efficiently reuse prebuilt dense representation with lightweight representation, and also discusses the necessity of systems for utilizing external contextual information to include contextual information outside the given context in text embedding tasks. in contrast to these approaches, ktrl+f directs its attention on augmenting knowledge from external sources for entities in a novel in-document retrieval task. this involves extracting information not present in the given text, thus expanding the capabilities of the information retrieval process. ## ktrl+f: knowledge-augmented in-document search in this section, we define ktrl+f, which is knowledge-augmented in-document search task and its unique characteristics ( §3.1). then we describe the evaluation metrics to measure each requirement ( §3.2). ## ktrl+f dataset we introduce a data construction pipeline to assemble essential components of ktrl+f: input document, query, corresponding targets, and external knowledge (figure 2 ). then we describe human verification procedures to ensure quality. ## knowledge-augmented phrase retrieval the challenge of ktrl+f is to effectively balance real-time applicability and high performance while utilizing efficient use of external knowledge. to meet the three requirements of ktrl+f, we propose knowledge-augmented phrase retrieval extending the phrase retrieval architecture of densephrases @xcite within the setting of in-document search and enriching external knowledge about potential targets with external knowledge linking and knowledge aggregation modules as illustrated in figure 3 . notably, our model doesn't require an additional training step. ## user study to verify whether solving ktrl+f can enhance search experience of users in the real web environments, we build chrome extension plugin (ktrl+f plugin) built on our model. ## conclusion & future work in this paper, we introduce ktrl+f, a knowledgeaugmented in-document search that requires identifying all semantic targets with a single natural query in real-time. ktrl+f tackles unique challenge for in-document search that requires capturing targets containing additional information beyond the input document by utilizing external knowledge while balancing speed and performance. we highlight limitations in existing models, such as hallucinations, high latency, or difficulties to incorporate external knowledge. and show that our knowledge-augmented phrase retrieval, simple extension of phrase retrieval architecture can be a robust model for ktrl+f. moreover, the study demonstrates that even our straightforward model, with seamless access to external knowledge during in-document searches, significantly enhances the user search experience. future work could extend ktrl+f to reflect updated knowledge, such as news, or domain-specific knowledge bases, such as the medical domain, which cannot be easily handled by large language models alone @xcite @xcite . the scalability and practicality of ktrl+f will open up opportunities for various advancements in the field of information retrieval and knowledge augmentation.