NOVA : An Iterative Planning Framework for Enhancing Scientific Innovation with Large Language Models (2025)

Abstract: Scientific innovation is pivotal for humanity, and harnessing large language models (LLMs) to generate research ideas could transform discovery. However, existing LLMs often produce simplistic and repetitive suggestions due to their limited ability in acquiring external knowledge for innovation. To address this problem, we introduce an enhanced planning and search methodology designed to boost the creative potential of LLM-based systems. Our approach involves an iterative process to purposely plan the retrieval of external knowledge, progressively enriching the idea generation with broader and deeper insights. Validation through automated and human assessments demonstrates that our framework substantially elevates the quality of generated ideas, particularly in novelty and diversity. The number of unique novel ideas produced by our framework is 3.4 times higher than without it. Moreover, our method outperforms the current state-of-the-art, generating at least 2.5 times more top-rated ideas based on 170 seed papers in a Swiss Tournament evaluation. Our code is available at https://github.com/hflyzju/Nova

## introduction in recent years, llms have demonstrated remarkable progress across various challenging tasks, including solving mathematical problems @xcite , proving mathematical theory @xcite , and generating code to solve analytical or computational tasks @xcite . these advances have opened up new possibilities to utilize llms to accelerate research @xcite , including generating novel research ideas @xcite @xcite @xcite @xcite . our work addresses the fundamental challenge of enabling llms to generate high-caliber research ideas by introducing a systematic framework that combines theoretical principles from scientific discovery with dynamic knowledge exploration. unlike existing approaches that rely primarily on retrieval or prompt engineering, we propose a structured method for expanding both the breadth and depth of idea generation. existing studies @xcite have tackled this challenge by integrating additional knowledge into the idea generation process. @xcite enrich the process by incorporating co-occurrence entities with existing knowledge, prompting llms to generate ideas based on these entities. @xcite suggest an iterative approach to retrieve topicrelevant papers through the semantic scholar api, utilizing retrieval-augmented generation (rag) for idea generation. they find that "llm-generated ideas are judged as more novel (@xmath0.05) than human experts". however, they also show that "llms lack diversity in idea generation". we argue that this limitation stems from the constrained scope and lack of direction in knowledge acquisition within these methods. @xcite , researchagent @xcite , ai-scientist @xcite and ai-researcher @xcite . here, "si theory guided" means generating seed ideas under the guidance of scientific innovation theory as mentioned in section 3.1, and "planning guided" refers to planning guided rag, as detailed in section 3.2. broadening the scope of the search, both in terms of breadth and depth, presents a significant challenge. the crux of the issue lies in determining what knowledge to retrieve. traditional methods of entity and keyword retrieval are not goal-oriented and frequently yield knowledge that is not conducive to fostering innovation. in order to address the above problem, we introduce an iterative planning framework for llmbased idea generation that specifically targets the enhancement of the novelty and diversity of the ideas produced. starting with seed ideas that are generated using scientific innovation theories and related knowledge, our framework undergoes multiple iterations of planning and searching. in each iteration, the model is tasked with devising a search plan aimed at identifying papers from multiple fields that will enhance the novelty and diversity of the current set of ideas. as depicted in figure 1 , the proposed iterative planning framework significantly enhances the quality of ideas generated from recent 170 llmrelated papers (from top conferences like acl, iclr, and cvpr). the number of high-quality ideas (as measured by the swiss tournament score @xcite ) is at least 2.5 times greater than those produced by other state-of-the-art methods. moreover, the number of unique novel ideas generated by our iterative planning framework is 3.4 times higher compared to approaches that do not incorporate such a framework. ## nova pipeline nova implements a three-stage pipeline (figure 2 ) that systematically integrates scientific innovation theories with dynamic knowledge exploration. @xcite the process begins with multi-source seed idea generation, incorporating theoretical principles, latest research trends, and domain expertise. (2) these initial ideas undergo iterative refinement through our novel planning mechanism, which identifies and integrates knowledge from multiple distinct fields per iteration. (3) finally, the system expands promising ideas through structured elaboration and validation. ## experiment to validate our proposed iterative planning framework, we perform comprehensive comparisons with state-of-the-art research idea generation methods and conduct an ablation study. ## conclusion in this paper, we propose an llm-based scientific innovation method, nova, which introduces iterative planning and search to retrieve external knowledge for innovation. nova leverages the internal knowledge of llms to generate search plans for external knowledge retrieval, significantly enhancing the effectiveness of the retrieval process. the ablation study demonstrates the effect of the iterative planning and search framework on promoting the novelty of generating ideas. the automatic and human evaluations show that nova significantly and consistently outperforms state-of-the-art scientific innovation methods. future work could explore: (1) incorporating multi-modal knowledge sources beyond textual academic literature, (2) developing reinforced and adaptive planning strategies that dynamically adjust the exploration depth based on idea potential, and (3) investigating methods for maintaining idea diversity while scaling to larger knowledge bases. 6 limitations while experimental results demonstrate nova's effectiveness in enhancing research ideation, we identify several promising directions for future work: knowledge integration scope. current implementation focuses primarily on computer science literature, opening opportunities for exploring cross-domain knowledge integration to potentially enable even more innovative interdisciplinary insights. computational efficiency. as the system explores multiple knowledge fields iteratively, there are opportunities to optimize the search and integration processes while maintaining idea quality. future work could investigate more efficient knowledge retrieval and filtering mechanisms. planning strategies. while our current planning mechanism significantly improves idea diversity, future research could explore incorporating reward functions or learning-based approaches to further enhance planning effectiveness. these directions suggest rich opportunities for extending nova's capabilities while building on its demonstrated strengths in enhancing research ideation.