How Do Large Language Models Perform on PDE Discovery: A Coarse-to-fine Perspective (2025)

Abstract: This paper studies the problem of how to use large language models (LLMs) to identify the underlying partial differential equations (PDEs) out of very limited observations of a physical system. Previous methods usually utilize physical-informed neural networks (PINNs) to learn the PDE solver and coefficient of PDEs simultaneously, which could suffer from performance degradation under extreme data scarcity. Towards this end, this paper attempts to utilize LLMs to solve this problem without further fine-tuning by proposing a novel framework named LLM for PDE Discovery (LLM4PD). The core of our LLM4PD is to utilize a coarse-to-fine paradigm to automatically discover underlying PDEs. In the coarse phase, LLM4PD selects the crucial terms from a library with hierarchical prompts and incorporates a review agent to enhance the accuracy. In the fine phase, LLM4PD interacts with a PDE solver to optimize the coefficient of the selected terms with the optimization trajectory. We also provide an adaptive hybrid optimization strategy switching between fine-tuning and exploration to balance stability and efficiency. Extensive experiments on several systems validate the effectiveness of our proposed LLM4PD in different settings.

## introduction partial differential equations (pdes) have been found to drive a range of physical systems ranging from fluid dynamics @xcite to quantum mechanics @xcite . however, in the real world, there are newly found complex systems where the underlying laws are unavailable @xcite . towards this end, pde discovery @xcite @xcite has raised extensive interest, which aims to identify the governing pdes of systems from the observed data. early data-driven approaches usually utilize shallow tools such as sparse regression @xcite @xcite to learn the coefficient of pdes. recently, deep learning approaches have achieved great success in this domain @xcite @xcite , which generally utilize the power of physics-informed neural networks (pinns) @xcite to implicitly represent pdes and their derivates. then, they minimize the error at the observation points and the regression loss of pde coefficients. with a strong optimizer, they can recover the governing pdes from the data. despite their great success, these approaches @xcite @xcite usually are trained in a supervised end-to-end manner, which requires extensive observations of complex systems. however, the measurement of systems could be prohibitively expensive or even impossible in the real world and data scarcity would significantly degrade the performance @xcite . towards this end, we study the problem of pde discovery under very limited data. recently, large language models (llms) have demonstrated the strong capability of few-shot learning @xcite @xcite and zero-shot learning @xcite @xcite @xcite without supervised fine-tuning and optimization @xcite . therefore, we aim to utilize llms to solve the problem in this work. in fact, incorporating llms into pde discovery is a highly challenging task, which requires us to answer the following two questions. (i) how to enhance the understanding of llms on pde understanding? llms are usually trained on a large amount of text data @xcite , which is different from the pde discovery problem. due to the lack of physics knowledge, llms could have difficulty in understanding our problem @xcite . (ii) how to generate the coefficients of pdes using llms? llms basically follow the paradigm of next-token prediction (he and su, 2024), which could barely generate the optimal coefficients in one shot. therefore, we need to design a holistic framework to generate the optimal coefficients gradually. towards this end, we propose a novel framework named llm4pd for llm-based pde discovery under limited data. the high-level idea is to utilize a coarse-to-fine strategy to optimize the coefficients of pdes. in the coarse phase, we utilize llms to select the crucial terms from a library of derivative terms. to enhance the understanding of llms on pdes, we introduce hierarchical prompts from physics, data, and reasoning views, which can effectively guide pre-trained llms to align with physics knowledge. we also design a review agent to ensure the correctness of the selected terms before moving to the next phase. to utilize llms for effective coefficient generation, we evaluate the error between the simulation data of the current coefficients and the observations, which would be stored in a memory bank. in the fine phase, we adopt an optimization lens to follow the trajectory of the generated coefficients. to balance the stability and efficiency, we design an adaptive optimization strategy which randomly switches between the large-step exploration and small-step fine-tuning. large-step exploration can help the model escape from local minima, while small-step fine-tuning can help the model explore the search space more efficiently. to prevent the coarse phase from generating wrong terms, we take an interleaved strategy across the coarse and fine phases based on the error reduction status. extensive experiments on both several datasets demonstrate the effectiveness of our llm4pd compared with competing baselines. in summary, our contributions are three-fold: ❶ problem connection. we connect pde discovery with llms and propose a novel approach to recover the underlying pdes from the scarce data without training. ❷ novel methodology. our llm4pd follows a coarse-to-fine paradigm, which utilizes hierarchical prompts to reason the candidate terms and optimize the coefficients by incorporating a pde solver and the optimization trajectory with an adaptive strategy. ## conclusion in this paper, we propose a novel framework named llm4pd, which can effectively recover the underlying pdes under extreme label scarcity. our proposed llm4pd adopts a coarse-to-fine framework. in the coarse phase, we identify the crucial terms from a library, followed by an llm-based review agent for further checking. in the fine phase, we combine llms with a pde solver, which provides feedback using the limited data source. extensive experiments on benchmark datasets validate the superiority of the proposed llm4pd.