Exploring Hint Generation Approaches for Open-Domain Question Answering (2024)

Abstract: Automatic Question Answering (QA) systems rely on contextual information to provide accurate answers. Commonly, contexts are prepared through either retrieval-based or generation-based methods. The former involves retrieving relevant documents from a corpus like Wikipedia, whereas the latter uses generative models such as Large Language Models (LLMs) to generate the context. In this paper, we introduce a novel context preparation approach called HINTQA, which employs Automatic Hint Generation (HG) techniques. Unlike traditional methods, HINTQA prompts LLMs to produce hints about potential answers for the question rather than generating relevant context. We evaluate our approach across three QA datasets including TriviaQA, Natural Questions, and Web Questions, examining how the number and order of hints impact performance. Our findings show that the HINTQA surpasses both retrieval-based and generation-based approaches. We demonstrate that hints enhance the accuracy of answers more than retrieved and generated contexts.

## introduction automatic question answering (qa) systems @xcite have recently garnered significant attention. they allow users to receive direct responses to posed questions. qa systems typically comprise three main components: context-preparator, reranker, and reader @xcite . the context-preparator component aims to supply relevant context to the user question. the reranker then prioritizes the documents based on their relevance to the question or to potential answers @xcite . lastly, the reader extracts the answer from the provided context. ## conclusion in this paper, we introduced a novel approach to the context-preparator in qa systems called hintqa that generates hints instead of relying on retrieved passages or generated contexts. to thoroughly test its effectiveness, we designed a variety of experimental setups, aiming to cover a broad spectrum of possible scenarios. our findings reveal that this new approach consistently surpasses traditional baseline methods, including both retrieval-based and generation-based approaches, on the triviaqa, nq, and webq datasets across multiple evaluation metrics. moreover, we demonstrated that different configurations, such as employing various llms as the core of the hg method and adjusting ranking methods and the number of hints, significantly boost the performance of our approach. our future work will focus on more complex questions such as multi-hop questions @xcite requiring comprehensive reasoning.