Self- DC : When to Reason and When to Act? Self Divide-and-Conquer for Compositional Unknown Questions (2025)

Abstract: Previous research has typically concentrated on leveraging the internal knowledge of Large Language Models (LLMs) to answer known questions (i.e., internal reasoning such as generate-then-read). In contrast, for questions that fall outside their known scope, these models rely on external knowledge retrieval to provide accurate responses (i.e., external acting such as retrieve-then-read). However, few previous works consider the compositional questions , which consist of several known and unknown sub-questions, necessitating the dynamic combination of previous two methods (i.e., internal reasoning and external acting ) to achieve a better trade-off between effectiveness and efficiency. To this end, we introduce a Self D ivide-and- C onquer ( Self-DC ) framework, accompanying with the first C ompositional u nknown Q uestion- A nswering dataset (CuQA). This framework enables LLMs to adaptively choose between using internal knowledge and retrieving external knowledge as needed, resulting in a better trade-off between effectiveness and efficiency. Experimental results on two datasets demonstrate that Self-DC can achieve comparable or even better performance with much fewer external calls compared with several strong baselines.

## introduction large language models (llms) @xcite possess extensive world knowledge thanks to the scaling of size of pretraining data and model @xcite , resulting in exceptional capabilities to answer opendomain questions using internal known knowledge encoded in their parameters @xcite . however, due to the cutoff date of training data, it is difficult for them to answer questions out of their known knowledge (a.k.a., unknown questions), which necessitates the augmentation of external retrieval @xcite @xcite , such as google search and wikipedia. ## data collection in this section, we thoroughly introduce how to collect the compositional unknown question-answer dataset (cuqa) automatically, with the minimum human efforts to filter unqualified samples. ## analysis in this section, we present a comprehensive analysis of self-dc mainly using the cuqa dataset, covering three key aspects: efficiency analysis, the choices of α and β and different iteration depth on latest model gpt-4o-mini. ## case study figure 7 and figure 8 show an example from cuqa and freshqa dataset respectively. we can found that self-dc is capable to call different functions to address various compositional questions including known and unknown sub-questions. ## conclusion in this paper, we firstly introduce compositional unknown questions, which contain several known and unknown sub-questions. we build a benchmark, named cuqa, to evaluate the performance and efficiency of existing compositional reasoning methods. furthermore, we present a self divideand-conquer (self-dc) method to adaptively call external or internal knowledge, which not only demonstrates comparable or even better performance compared with existing complex iterative retrieval methods with fewer retrieval calls but also shows a promising potential to elicit internal capabilities of llms while minimizing external reliance.