Hierarchical Retrieval with Evidence Curation for Open-Domain Financial Question Answering on Standardized Documents (2025)

Abstract: Retrieval-augmented generation (RAG) based large language models (LLMs) are widely used in finance for their excellent performance on knowledge-intensive tasks. However, standardized documents (e.g., SEC filing) share similar formats such as repetitive boilerplate texts,and similar table structures. This similarity forces traditional RAG methods to misidentify near-duplicate text, leading to duplicate retrieval that undermines accuracy and completeness. To address these issues, we propose the Hierarchical Retrieval with Evidence Curation (HiREC) framework. Our approach first performs hierarchical retrieval to reduce confusion among similar texts. It first retrieve related documents and then selects the most relevant passages from the documents. The evidence curation process removes irrelevant passages. When necessary, it automatically generates complementary queries to collect missing information. To evaluate our approach, we construct and release a Large-scale Open-domain Financial (LOFin) question answering benchmark that includes 145,897 SEC documents and 1,595 question-answer pairs. Our code and data are available at https://github.com/deep-over/LOFin-bench-HiREC.

## introduction retrieval-augmented generation (rag) @xcite with large language models (llms) have significantly improved performance in knowledge-intensive tasks. due to its ability to improve both factual accuracy and timeliness, extensive research @xcite has investigated applying rag in searching financial information where accurate and up-todate information is crucial for decision-making. financial documents, such as annual reports (10-k) in sec filings, are highly structured and follow standardized templates across companies and periods, often containing similar tables and repetitive narratives. as shown in figure 1 , 2023 10-k reports from amazon, meta, and walmart exhibit nearly identical table structures with similar titles and indicators, differing mainly in numerical values. consequently, when asked, what is the difference in operating income ratio between amazon and walmart in 2023?, a conventional rag system may struggle to distinguish among these similar passages, retrieving irrelevant or redundant information that leads to inaccurate answers. to tackle these challenges from the standardized format of financial documents, we propose the hirec (hierarchical retrieval and evidence curation) framework. hirec consists of two main components: hierarchical retrieval and evidence curation. the hierarchical retrieval first retrieves related documents and then selects the most relevant passages from the documents, thereby reducing confusion from near-duplicate text. as illustrated in figure 1 , narrowing the candidate set to documents from amazon and walmart enables the system to focus on highly relevant passages. however, the prevalence of comparative questions in financial qa often leads to incomplete retrieval of essential evidence. to address this, the evidence curation stage filters out irrelevant passages and generates complementary queries when necessary. for example if only amazon operating income is retrieved then a complementary query is generated to fetch walmart operating income so that all necessary evidence is gathered for an accurate answer. to evaluate our approach, we assess qa performance in an open-domain setting. existing financial question-answering benchmarks @xcite rely on small-scale corpora that include at most about 1,300 documents and very limited test sets, which do not reflect realistic financial scenarios. we propose lofin (largescale open-domain financial qa), a comprehensive financial question-answering benchmark built on a large-scale corpus containing approximately 145,000 sec filings from companies in the s&p 500. lofin includes 1,595 open-domain questionanswering test instances and addresses challenges in standardized document retrieval, such as nearduplicate tables and repetitive narratives, that are not evident in smaller datasets. in addition, we release the entire benchmark as open-source to support future research in the field. experimental results indicate that hirec improves performance by at least 13% compared to existing rag methods. in addition, our framework consistently outperforms commercial llms with web search engines such as searchgpt @xcite and perplexity @xcite . our contributions are threefold: with the emergence of financial qa, research on financial rag has also been progressing. there are graph-based methods @xcite tailored to the financial domain as well as hybrid approaches @xcite . in addition, studies on financial document chunking offer valuable insights @xcite . retrieval augmented generation rag has evolved in diverse ways @xcite . hierarchical retrieval was proposed for cases where sections are clearly demarcated, typically employing a two-step document-passage process @xcite . the difference from previous studies is that while they segment documents into sections, we segment at the level of individual documents. there are also studies that utilize filtering to enhance the quality of the contexts input into rag @xcite . unlike previous approaches that required training, we manage quality using only an llm. iterative retrieval is typically proposed for multi-hop qa. a standard iterative method uses the context retrieved in the first step as part of the query for subsequent iterations @xcite . self-rag @xcite also conducts iterative retrieval that includes the generation process. however, our approach does not utilize previously retrieved context because we specifically need to discover missing information. ## large-scale open-domain financial qa in this section, we present lofin, a benchmark that overcomes the limitations of current financial qa datasets by expanding the retrieval corpus and increasing open-domain qa pairs (see ## hierarchical retrieval with evidence curation (hirec) framework in this section, we introduce the hirec framework. figure 2 shows the overall framework and process. the framework comprises two main components: hierarchical retrieval and evidence curation. during the hierarchical retrieval stage a hierarchical approach retrieves passages p r that are relevant to the question q. in the evidence curation process the retrieved passages are filtered to retain only those directly pertinent and then evaluated to determine whether they provide sufficient information to answer the question. if the information is insufficient a complementary question q c is generated to reinitiate the retrieval process (complementary pass). otherwise if the evidence is sufficient the filtered passage set p f is forwarded to the answer generator to produce the final answer (main pass). when the maximum iteration i max is reached evidence curation halts and the passages retrieved using q c are merged with the previously filtered passages to generate the answer. the pseudocode of hirec is described in algorithm 1. for the llm-based components of our framework, we use instructionstyle prompts tailored to each module's objective. appendix d provides the full prompt templates and design process. ## conclusion we introduced hirec, a retrieval-augmented framework for question answering over standardized financial documents. our hierarchical retrieval reduces confusion from repeated boilerplate content, while evidence curation filters irrelevant passages and recovers missing information. to evaluate under realistic open-domain conditions, we constructed lofin, a large-scale benchmark with 1,595 qa pairs across 145,000 sec filings. the dataset includes multi-document and multihop questions that go beyond prior financial qa benchmarks. experiments show that hirec consistently outperforms state-of-the-art baselines in both retrieval quality and answer accuracy, while maintaining cost efficiency through selective passage use. overall, our findings suggest that hirec provides a scalable and effective solution for opendomain qa over complex, standardized financial documents.