O kra L ong: A Flexible Retrieval-Augmented Framework for Long-Text Question Answering (2025)

Abstract: Large Language Models (LLMs) encounter challenges in efficiently answering long-text questions, as seen in applications like enterprise document analysis and financial report comprehension. While conventional solutions employ long-context processing or Retrieval-Augmented Generation (RAG), they suffer from prohibitive input expenses or incomplete information. Recent advancements adopt context compression and dynamic retrieval loops, but still sacrifice critical details or incur iterative costs. To address these limitations, we propose OkraLong, a novel framework that flexibly optimizes the entire processing workflow. Unlike prior static or coarse-grained adaptive strategies, OkraLong adopts fine-grained orchestration through three synergistic components: analyzer, organizer and executor. The analyzer characterizes the task states, which guide the organizer in dynamically scheduling the workflow. The executor carries out the execution and generates the final answer. Experimental results demonstrate that OkraLong not only enhances answer accuracy by 5.7%-41.2%, but also achieves cost savings of 1.3x-4.7x.

## introduction large language models (llms) have been extensively utilized to handle external knowledge and unseen data, which is a common scenario in realworld applications such as enterprise search and data analysis @xcite . a critical challenge in these domains, however, lies in querying and comprehending long-form text @xcite . for example, a company may need to query its proprietary technique documents; a financial expert may need to extract insights from the latest corporate reports; and a research group may need to assimilate cutting-edge academic papers to guide their innovations. to tackle long-text questions, two prevalent methodologies are typically utilized: long-context (lc) and retrieval-augmented generation (rag) @xcite . the lc approach leverages the llm's inherent ability to process extensive texts by inputting entire content, enabling responses grounded in global contextual awareness @xcite . in contrast, rag employs a lightweight retriever to first identify question-relevant text segments, which are then analyzed by the llm to get the answer @xcite @xcite . however, when deployed in practical settings, these strategies encounter significant limitations in cost-effectiveness and accuracy. first, given that the cost of llms escalate with data volume, employing lc with voluminous text may prove prohibitively expensive (e.g., a financial report may span hundreds of pages) @xcite . while rag reduces input length by filtering irrelevant text, the context content still incurs moderate costs and risk omitting critical information @xcite . furthermore, real-world queries vary widely, from simple fact extraction to multi-step reasoning. rigid approaches like static retrieval struggle to adapt to this diversity, leading to information loss and inaccuracies @xcite . recent efforts to mitigate these limitations primarily focus on two paradigms: context compression and dynamic rags. as shown in figure 1 , compression-based approaches typically operate on extensive text segments, removing non-informative tokens or iteratively summarize the content using small generative models @xcite @xcite . however, these methods risk losing critical specific details and incur latency overhead due to heavy reliance on small models @xcite . dynamic rag approaches employ iterative retrievalgeneration cycles to adaptively make retrieval decisions @xcite @xcite . however, iterative workflow requires frequent llm calls, escalating financial costs, and the existing adaptive mechanisms remain coarsegrained, failing to optimize the performance effectively in varied scenarios. to address these limitations, we propose okra-long, a novel retrieval-augmented framework that systematically optimizes long-text question answering. unlike above approaches that rely on fixed workflow patterns, okralong flexibly orchestrates various pipelines according to different task scenarios. as illustrated in figure 2 , our framework comprises three synergistic components: (1) analyzer: a fine-tuned lightweight model that proactively characterizes task states, utilizing question semantics and preliminary retrieved contexts; (2) adaptive organizer: a dynamic scheduler that generates optimized execution plans, based on previous analysis; (3) executor: a modular operator suite that supports the execution of diverse processing pipelines and strategies. distinct from prior adaptive rag methods @xcite that make coarse-grained decisions (e.g. whether to generate iteratively or retrieve additional data) , okralong is designed to fine-grainedly optimize the entire processing workflow, covering multiple modules and various pipelines. first, to improve accuracy performance, okralong constructs the flexibility to tailor strategies for different tasks. for example, comparative tasks (e.g., who won the most awards, a, b or c?) demand separate entity retrieval, while multi-step questions (e.g., what is the place of birth of the director of film clowning around?) trigger iterative reasoning process. second, for financial effectiveness, our cost-aware organizer dynamically allocates token budgets and information resources. for instance, general summarizing questions receive multiple aggregated contexts, whereas fact extraction questions utilize targeted context slicing. it is worth noting that these flexible orchestrations are facilitated by the task-understanding analyzer and we also develop several innovative executing operators to support tailored strategies. we evaluate okralong using an extensive collection of long-text question-answering datasets, spanning multiple domains, covering various question types. the experimental results demonstrate that okralong not only enhances answer accuracy by 5.7%-41.2%, but also provides superior cost savings of 1.3x-4.7x. ## conclusion in this paper, we propose okralong, a flexible and efficient retrieval-augmented framework for longtext question answering. this innovative framework adaptively orchestrates the entire workflow through its three synergistic components: analyzer, organizer, and executor. okralong characterizes task states, dynamically organize the workflow, and carries out the execution to generate final answers. we conduct comprehensive evaluations across six diverse datasets, spanning multiple domains and task types. the experimental results indicate that okralong not only enhances answering quality but also delivers significant cost-effectiveness. compared to pre-existing methods, okralong demonstrates superior performance in handling long-text questions, thereby providing a highly efficient solution for practical deployment.