R esearch A gent: Iterative Research Idea Generation over Scientific Literature with Large Language Models (2025)

Abstract: The pace of scientific research, vital for improving human life, is complex, slow, and needs specialized expertise. Meanwhile, novel, impactful research often stems from both a deep understanding of prior work, and a cross-pollination of ideas across domains and fields. To enhance the productivity of researchers, we propose ResearchAgent, which leverages the encyclopedic knowledge and linguistic reasoning capabilities of Large Language Models (LLMs) to assist them in their work. This system automatically defines novel problems, proposes methods and designs experiments, while iteratively refining them based on the feedback from collaborative LLM-powered reviewing agents. Specifically, starting with a core scientific paper, ResearchAgent is augmented not only with relevant publications by connecting information over an academic graph but also entities retrieved from a knowledge store derived from shared underlying concepts mined across numerous papers. Then, mimicking a scientific approach to improving ideas with peer discussions, we leverage multiple LLM-based ReviewingAgents that provide reviews and feedback via iterative revision processes. These reviewing agents are instantiated with human preference-aligned LLMs whose criteria for evaluation are elicited from actual human judgments via LLM prompting. We experimentally validate our ResearchAgent on scientific publications across multiple disciplines, showing its effectiveness in generating novel, clear, and valid ideas based on both human and model-based evaluation results. Our initial foray into AI-mediated scientific research has important implications for the development of future systems aimed at supporting researchers in their ideation and operationalization of novel work.

## introduction scientific research plays a crucial role in driving innovation, advancing knowledge, solving problems, expanding our understanding of the world, and ultimately improving the lives of people in tangible ways. this process usually consists of two key components: the formulation of new research ideas and the validation of these ideas through wellcrafted experiments, which are typically conducted by human researchers @xcite @xcite . however, this is a slow, effort-intensive process, which requires reading and synthesizing overwhelming amounts of knowledge over the vast corpus of rapidly growing scientific literature to formulate research ideas, as well as design and perform experimental validations of those ideas. for example, the number of academic papers published per year is more than 7 million @xcite . similarly, the process of testing a new pharmaceutical drug requires deep expertise, and is massively expensive and labor-intensive, often taking several years @xcite . in the meantime, recent large language models (llms) @xcite @xcite have shown impressive capabili-ties in processing and generating text with remarkable accuracy, even outperforming human experts across diverse specialized domains including math, physics, history, law, medicine, and ethics. they are able to process and analyze large volumes of data at speeds and scales far exceeding human capabilities, have internalized large swaths of human knowledge from being trained on virtually the entire web, and can identify patterns, trends, and correlations that may not be immediately apparent to human researchers (such as the usage of quantum mechanics in medical imaging or applying psychological insights in ai). this renders them ideally poised to become foundational tools to accelerate the two phases of the scientific research process: ideation of novel research opportunities, and scientific validation of those research hypotheses. a few recent papers in the domain of llmaugmented scientific discovery have focused on the second phase. specifically, they attempt @xcite @xcite to mainly accelerate the experimental validation process, by writing code for machinelearning models, facilitating the exploration of chemical spaces, or advancing the simulation of molecular dynamics. thus, in this paper, we leverage llms in the first phase of scientific research -specifically idea generation, whose key focus is conceptualizing novel research questions, methodologies, and experiments. to the best of our knowledge, our work is the first to leverage and evaluate the capabilities of llms to act as mediators in scientific idea generation in an open-ended setting. given our goal to build an llm-powered re-searchagent, we draw inspiration from how human researchers position themselves to come up with novel research ideas. we draw distinctions between three key components of their workflow: a broad and deep understanding of related scientific literature, an encyclopedic view of concepts and how they relate to one another both within and across domains, and a community of colleagues on which to rely for feedback and constructive criticism. we model each of these three aspects in our researchagent. specifically, in order to imbibe related work, the system begins with a core scientific paper and then explores a range of related papers through references and citation relationships. further, to develop an encyclopedic view of related concepts, we build and then augment researchagent with an entity-centric knowledge store derived from co-occurrences of key concepts in the scien-tific literature. this repository is aimed at capturing novel underlying relationships within and across domains, thereby increasing the chances of a crosspollination of ideas @xcite . finally, to simulate robust feedback mechanisms, we instantiate a number of llm-powered reviewingagents that help the researchagent to iterate on research idea generation with constructive critiques. crucially, these reviewingagents are prompted with evaluation criteria that are induced from real researchers' judgements, thus aligning them with actual scientific preferential standards. an illustration of our system is provided in figure 1 . we validate the effectiveness of researchagent for research idea generation based on scientific literature across multiple disciplines. then, on a battery of tests conducted with both human-and modelbased evaluations, we demonstrate that researchagent outperforms strong llm-powered baselines by large margins, generating more clear, relevant, and significant ideas that are especially novel. furthermore, analyses show the efficacy of our comprehensive approach to modeling researchagent: the entity-centric knowledge store and the iterative idea refinement steps help the system generate meaningfully better ideas compared with an instantiation that is purely based on prior related work. these findings highlight the immense potential of ai-mediated research assistants like researchagent to enhance the ideation process in scientific research. in practice, it can support researchers by identifying knowledge gaps, proposing novel problem statements, and suggesting potential methodologies early in the research process. also, it can assist in designing experiments and streamline the writing and refinement of research papers by generating drafts and offering feedback on how to effectively frame contributions and cite relevant work. ## method we present researchagent, a system that automatically proposes research ideas with llms. ## experimental results and analyses main results our main results on scoring with human and model-based evaluations are provided in figure 2 . these demonstrate that our full re-searchagent outperforms all baselines by large margins on every metric across problems, methods, and experiment designs (constituting the complete research ideas). particularly, the full researchagent augmented with relevant entities exhibits strong gains on metrics related to creativity (such as originality for problems and innovativeness for methods) since entities may offer novel concepts and views that may not be observable in the group of citation-based papers alone. in addition, the results of pairwise comparisons between models with both human and model-based evaluations -shown in figure 3 -demonstrate that the full researchagent shows the highest win ratio over its baselines.