jablonkagroup/rise_ai_scientists huggingface dataset:
Authors: Hanzheng Li | Xi Fang | Yixuan Li | Chaozheng Huang | Junjie Wang | Xi Wang | Hongzhe Bai | Bojun Hao | Shenyu Lin | Huiqi Liang | Linfeng Zhang | Guolin Ke
Source: Open MIND (preprint, 2025-12-29)
Field: Materials Science — Materials Chemistry

RxnBench: A Multimodal Benchmark for Evaluating Large Language Models on Chemical Reaction Understanding from Scientific Literature The integration of Multimodal Large Language Models (MLLMs) into chemistry promises to revolutionize scientific discovery, yet their ability to comprehend the dense, graphical language of reactions within authentic literature remains underexplored. Here, we introduce RxnBench, a multi-tiered benchmark designed to rigorously evaluate MLLMs on chemical reaction understanding from scientific PDFs. RxnBench comprises two tasks: Single-Figure QA (SF-QA), which tests fine-grained visual perception and mechanistic reasoning using 1,525 questions derived from 305 curated reaction schemes, and Full-Document QA (FD-QA), which challenges models to synthesize information from 108 articles, requiring cross-modal integration of text, schemes, and tables. Our evaluation of MLLMs reveals a critical capability gap: while models excel at extracting explicit text, they struggle with deep chemical logic and precise structural recognition. Notably, models with inference-time reasoning significantly outperform standard architectures, yet none achieve 50\% accuracy on FD-QA. These findings underscore the urgent need for domain-specific visual encoders and stronger reasoning engines to advance autonomous AI chemists. Benchmark (surveying) ; Scientific reasoning ; Perception ; Scientific modelling ; Scientific literature ; Visualization ; Visual language ; Language model Machine Learning in Materials Science ; Computational Drug Discovery Methods ; Biomedical Text Mining and Ontologies
Authors: Chaojian Li | Zhifan Ye | Massimiliano Lupo Pasini | Jong Youl Choi | Cheng Wan | Prasanna Balaprakash
Source:  (article, 2025-06-22)
Field: Materials Science — Materials Chemistry

Scaling Laws of Graph Neural Networks for Atomistic Materials Modeling* Atomistic materials modeling is a critical task with wide-ranging applications, from drug discovery to materials science, where accurate predictions of the target material property can lead to significant advancements in scientific discovery. Graph Neural Networks (GNNs) represent the state-of-the-art approach for modeling atomistic material data thanks to their capacity to capture complex relational structures. While machine learning performance has historically improved with larger models and datasets, GNNs for atomistic materials modeling remain relatively small compared to large language models (LLMs), which leverage billions of parameters and terabyte-scale datasets to achieve remarkable performance in their respective domains. To address this gap, we explore the scaling limits of GNNs for atomistic materials modeling by developing a foundational model with billions of parameters, trained on extensive datasets in terabytescale. Our approach incorporates techniques from LLM libraries to efficiently manage large-scale data and models, enabling both effective training and deployment of these large-scale GNN models. This work addresses three fundamental questions in scaling GNNs: the potential for scaling GNN model architectures, the effect of dataset size on model accuracy, and the applicability of LLM-inspired techniques to GNN architectures. Specifically, the outcomes of this study include (1) insights into the scaling laws for GNNs, highlighting the relationship between model size, dataset volume, and accuracy, (2) a foundational GNN model optimized for atomistic materials modeling, and (3) a GNN codebase enhanced with advanced LLM-based training techniques. Our findings lay the groundwork for large-scale GNNs with billions of parameters and terabyte-scale datasets, establishing a scalable pathway for future advancements in atomistic materials modeling. Leverage (statistics) ; Scalability ; Codebase ; Scaling ; Scaling law ; Multiscale modeling ; Graph ; Property (philosophy) Machine Learning in Materials Science ; Electron and X-Ray Spectroscopy Techniques ; X-ray Diffraction in Crystallography
Authors: Wanshan Ning | Musen Li | Jeffrey R. Reimers | Rika Kobayashi
Source: ArXiv.org (preprint, 2025-12-10)
Field: Materials Science — Materials Chemistry

Optimizing Data Extraction from Materials Science Literature: A Study of Tools Using Large Language Models Large Language Models (LLMs) are increasingly utilized for large-scale extraction and organization of unstructured data owing to their exceptional Natural Language Processing (NLP) capabilities. Empowering materials design, vast amounts of data from experiments and simulations are scattered across numerous scientific publications, but high-quality experimental databases are scarce. This study considers the effectiveness and practicality of five representative AI tools (ChemDataExtractor, BERT-PSIE, ChatExtract, LangChain, and Kimi) to extract bandgaps from 200 randomly selected Materials Science publications in two presentations (arXiv and publisher versions), comparing the results to those obtained by human processing. Although the integrity of data extraction has not met expectations, encouraging results have been achieved in terms of precision and the ability to eliminate irrelevant papers from human consideration. Our analysis highlights both the strengths and limitations of these tools, offering insights into improving future data extraction techniques for enhanced scientific discovery and innovation. In conjunction with recent research, we provide guidance on feasible improvements for future data extraction methodologies, helping to bridge the gap between unstructured scientific data and structured, actionable databases. Bridge (graph theory) ; Data extraction ; Information extraction ; Natural language ; Conjunction (astronomy) ; Unstructured data ; Knowledge extraction ; Extraction (chemistry) Machine Learning in Materials Science ; Artificial Intelligence in Healthcare and Education ; Scientific Computing and Data Management
Authors: Peter Jansen | Siti Lailatul Mohd Hassan | Pragnya Narasimha
Source: ArXiv.org (preprint, 2025-11-30)
Field: Materials Science — Materials Chemistry

CodeDistiller: Automatically Generating Code Libraries for Scientific Coding Agents Automated Scientific Discovery (ASD) systems can help automatically generate and run code-based experiments, but their capabilities are limited by the code they can reliably generate from parametric knowledge alone. As a result, current systems either mutate a small number of manually-crafted experiment examples, or operate solely from parametric knowledge, limiting quality and reach. We introduce CodeDistiller, a system that automatically distills large collections of scientific Github repositories into a vetted library of working domain-specific code examples, allowing ASD agents to expand their capabilities without manual effort. Using a combination of automatic and domain-expert evaluation on 250 materials science repositories, we find the best model is capable of producing functional examples for 74% of repositories, while our downstream evaluation shows an ASD agent augmented with a CodeDistiller generated library produces more accurate, complete, and scientifically sound experiments than an agent with only general materials-science code examples. Code (set theory) ; Limiting ; Coding (social sciences) ; Source code ; Parametric statistics ; Quality (philosophy) ; Code generation Machine Learning in Materials Science ; Scientific Computing and Data Management ; Software Engineering Research
Authors: Rob Yang
Source: ArXiv.org (preprint, 2025-08-25)
Field: Materials Science — Materials Chemistry

Unlearning as Ablation: Toward a Falsifiable Benchmark for Generative Scientific Discovery Bold claims about AI's role in science-from "AGI will cure all diseases" to promises of radically accelerated discovery-raise a central epistemic question: do large language models (LLMs) truly generate new knowledge, or do they merely remix memorized fragments? We propose unlearning-as-ablation as a falsifiable probe of constructive scientific discovery. The idea is to systematically remove a target result together with its forget-closure (supporting lemmas, paraphrases, and multi-hop entailments) and then evaluate whether the model can re-derive the result from only permitted axioms and tools. Success would indicate generative capability beyond recall; failure would expose current limits. Unlike prevailing motivations for unlearning-privacy, copyright, or safety-our framing repositions it as an epistemic probe for AI-for-Science. We outline a minimal pilot in mathematics and algorithms to illustrate feasibility, and sketch how the same approach could later be extended to domains such as physics or chemistry. This is a position paper: our contribution is conceptual and methodological, not empirical. We aim to stimulate discussion on how principled ablation tests could help distinguish models that reconstruct knowledge from those that merely retrieve it, and how such probes might guide the next generation of AI-for-Science benchmarks. Falsifiability ; Sketch ; Generative grammar ; Axiom ; Constructive ; Scientific discovery ; Benchmark (surveying) ; GRASP ; Framing (construction) Machine Learning in Materials Science ; Scientific Computing and Data Management ; Explainable Artificial Intelligence (XAI)
Authors: Yaping Zhang | Qixuan Zhang | Xingquan Zhang | Zhiyuan Chen | Wenwen Zhuang | Yupu Liang | Lu Xiang | Yang Zhao | Jiajun Zhang | Yu Zhou | Chengqing Zong
Source: arXiv (Cornell University) (preprint, 2025-12-28)
Field: Materials Science — Materials Chemistry

HiSciBench: A Hierarchical Multi-disciplinary Benchmark for Scientific Intelligence from Reading to Discovery The rapid advancement of large language models (LLMs) and multimodal foundation models has sparked growing interest in their potential for scientific research. However, scientific intelligence encompasses a broad spectrum of abilities ranging from understanding fundamental knowledge to conducting creative discovery, and existing benchmarks remain fragmented. Most focus on narrow tasks and fail to reflect the hierarchical and multi-disciplinary nature of real scientific inquiry. We introduce \textbf{HiSciBench}, a hierarchical benchmark designed to evaluate foundation models across five levels that mirror the complete scientific workflow: \textit{Scientific Literacy} (L1), \textit{Literature Parsing} (L2), \textit{Literature-based Question Answering} (L3), \textit{Literature Review Generation} (L4), and \textit{Scientific Discovery} (L5). HiSciBench contains 8,735 carefully curated instances spanning six major scientific disciplines, including mathematics, physics, chemistry, biology, geography, and astronomy, and supports multimodal inputs including text, equations, figures, and tables, as well as cross-lingual evaluation. Unlike prior benchmarks that assess isolated abilities, HiSciBench provides an integrated, dependency-aware framework that enables detailed diagnosis of model capabilities across different stages of scientific reasoning. Comprehensive evaluations of leading models, including GPT-5, DeepSeek-R1, and several multimodal systems, reveal substantial performance gaps: while models achieve up to 69\% accuracy on basic literacy tasks, performance declines sharply to 25\% on discovery-level challenges. HiSciBench establishes a new standard for evaluating scientific Intelligence and offers actionable insights for developing models that are not only more capable but also more reliable. The benchmark will be publicly released to facilitate future research. Benchmark (surveying) ; Focus (optics) ; Scientific discovery ; Scientific literature ; Reading (process) ; Foundation (evidence) ; Benchmarking Machine Learning in Materials Science ; Topic Modeling ; Artificial Intelligence in Healthcare and Education
Authors: Yaping Zhang | Qixuan Zhang | Xingquan Zhang | Zhiyuan Chen | Wenwen Zhuang | Yupu Liang | Lu Xiang | Yang Zhao | Jiajun Zhang | Yu Zhou | Chengqing Zong
Source: ArXiv.org (article, 2025-12-28)
Field: Materials Science — Materials Chemistry

HiSciBench: A Hierarchical Multi-disciplinary Benchmark for Scientific Intelligence from Reading to Discovery The rapid advancement of large language models (LLMs) and multimodal foundation models has sparked growing interest in their potential for scientific research. However, scientific intelligence encompasses a broad spectrum of abilities ranging from understanding fundamental knowledge to conducting creative discovery, and existing benchmarks remain fragmented. Most focus on narrow tasks and fail to reflect the hierarchical and multi-disciplinary nature of real scientific inquiry. We introduce \textbf{HiSciBench}, a hierarchical benchmark designed to evaluate foundation models across five levels that mirror the complete scientific workflow: \textit{Scientific Literacy} (L1), \textit{Literature Parsing} (L2), \textit{Literature-based Question Answering} (L3), \textit{Literature Review Generation} (L4), and \textit{Scientific Discovery} (L5). HiSciBench contains 8,735 carefully curated instances spanning six major scientific disciplines, including mathematics, physics, chemistry, biology, geography, and astronomy, and supports multimodal inputs including text, equations, figures, and tables, as well as cross-lingual evaluation. Unlike prior benchmarks that assess isolated abilities, HiSciBench provides an integrated, dependency-aware framework that enables detailed diagnosis of model capabilities across different stages of scientific reasoning. Comprehensive evaluations of leading models, including GPT-5, DeepSeek-R1, and several multimodal systems, reveal substantial performance gaps: while models achieve up to 69\% accuracy on basic literacy tasks, performance declines sharply to 25\% on discovery-level challenges. HiSciBench establishes a new standard for evaluating scientific Intelligence and offers actionable insights for developing models that are not only more capable but also more reliable. The benchmark will be publicly released to facilitate future research. Benchmark (surveying) ; Focus (optics) ; Scientific discovery ; Scientific literature ; Reading (process) ; Foundation (evidence) ; Benchmarking Machine Learning in Materials Science ; Topic Modeling ; Artificial Intelligence in Healthcare and Education
Authors: K. A. Bogonosov
Source: Russian journal of resources conservation and recycling (article, 2025-03-31)
Field: Materials Science — Materials Chemistry

AI decision support systems for choosing the architecture of physical experiments The study is devoted to a comprehensive analysis of the development and application of artificial intelligence systems in the field of decision support in the design of the architecture of physical experiments. The paper presents a systematization of modern approaches to the integration of machine learning methods, neural networks and expert systems into the planning and optimization of experimental research in various fields of physical sciences. The analysis demonstrates the transformation of traditional experimental paradigms under the influence of artificial intelligence technologies, which is reflected in the transition from intuitive methods of planning experiments to autonomous intelligent systems capable of independently determining optimal research parameters and adapting to the results in real time. Special attention is paid to the analysis of architectural solutions of modern AI decision support systems, including the integration of high-performance computing platforms, robotic complexes for automated experiments and active learning algorithms to optimize research processes. The paper considers in detail the key technological components of such systems: Bayesian optimization algorithms for targeted exploration of the parameter space, methods for sequential planning experiments taking into account uncertainty, approaches to integrating subject knowledge into machine learning models and technologies for autonomous control of experimental equipment. The study reveals the specifics of various types of physical experiments in terms of architecture requirements for supporting AI systems, including materials science research, optimization of energy systems, development of new compounds, and characterization of physico-chemical properties of substances. The results of the analysis highlight the significant potential of AI systems in accelerating scientific discoveries by automating routine experimental planning processes, improving the accuracy of predicting results and optimizing the use of research resources, while maintaining the critical role of expert knowledge in setting research objectives and interpreting the results. Field (mathematics) ; Intelligent decision support system ; Expert system ; Decision support system ; Artificial neural network ; Key (lock) ; Architecture ; Bayesian network ; Bayesian optimization Machine Learning in Materials Science ; Explainable Artificial Intelligence (XAI) ; Scientific Computing and Data Management
Authors: Hyunsoo Park | Kinga O. Mastej | Panyalak Detrattanawichai | Ryan Nduma | Aron Walsh
Source: ChemRxiv (preprint, 2025-07-22)
Field: Materials Science — Materials Chemistry

Closing the synthesis gap in computational materials design Data-driven strategies are reshaping computational materials design by accelerating the prediction of novel compounds with targeted functionalities. Beyond high-throughput screening, the integration of generative artificial intelligence enables exploration across vast chemical spaces comprising millions of known and hypothetical materials. This abundance of candidates presents a challenge: identifying which candidate compounds are not only low in energy, but also synthetically accessible. We assess advances towards closing this synthesis gap for inorganic crystals. These include the incorporation of thermodynamic potentials (from internal energies at 0 K to Gibbs free energies), crucial for evaluating phase stability and reaction driving forces, chemical heuristics (from charge neutrality to electronegativity rules), and machine learning models (from positive–unlabelled learning to large language models) to guide compound selection and prioritisation. Looking forward, the development of more robust synthesisability metrics, synthesis planning tools, and agentic workflows integrating experimental feedback will narrow the divide between virtual screening and real-world materials realisation. Closing (real estate) ; Computer science ; Political science ; Law Machine Learning in Materials Science ; Material Selection and Properties ; Additive Manufacturing and 3D Printing Technologies
Authors: Guoqing Liu | Jian Li | Zihan Zhao | Eray Inanc | Krzysztof Maziarz | José I. Torres | Victor García Satorras | Shinji Ueda | Chris Bishop | Marwin Segler
Source: ArXiv.org (preprint, 2025-12-15)
Field: Materials Science — Materials Chemistry

A Scientific Reasoning Model for Organic Synthesis Procedure Generation Solving computer-aided synthesis planning is essential for enabling fully automated, robot-assisted synthesis workflows and improving the efficiency of drug discovery. A key challenge, however, is bridging the gap between computational route design and practical laboratory execution, particularly the accurate prediction of viable experimental procedures for each synthesis step. In this work, we present QFANG, a scientific reasoning language model capable of generating precise, structured experimental procedures directly from reaction equations, with explicit chain-of-thought reasoning. To develop QFANG, we curated a high-quality dataset comprising 905,990 chemical reactions paired with structured action sequences, extracted and processed from patent literature using large language models. We introduce a Chemistry-Guided Reasoning (CGR) framework that produces chain-of-thought data grounded in chemical knowledge at scale. The model subsequently undergoes supervised fine-tuning to elicit complex chemistry reasoning. Finally, we apply Reinforcement Learning from Verifiable Rewards (RLVR) to further enhance procedural accuracy. Experimental results demonstrate that QFANG outperforms advanced general-purpose reasoning models and nearest-neighbor retrieval baselines, measured by traditional NLP similarity metrics and a chemically aware evaluator using an LLM-as-a-judge. Moreover, QFANG generalizes to certain out-of-domain reaction classes and adapts to variations in laboratory conditions and user-specific constraints. We believe that QFANG's ability to generate high-quality synthesis procedures represents an important step toward bridging the gap between computational synthesis planning and fully automated laboratory synthesis. Bridging (networking) ; Workflow ; Verifiable secret sharing ; Automated reasoning ; Similarity (geometry) ; Key (lock) ; Model-based reasoning ; Program synthesis Machine Learning in Materials Science ; Computational Drug Discovery Methods ; AI-based Problem Solving and Planning
Authors: Shafiq Sharhrah | Neetu Singh | Ganesh Maurya | Manbir Kaur | Chitransh Bose | Rashmi Priyadarshini
Source: Journal of Physics Conference Series (article, 2025-12-01)
Field: Materials Science — Materials Chemistry

Machine Learning and Artificial Intelligence–accelerated Computational Approaches in Materials Science Abstract Material discovery and design are being revolutionised by machine learning and artificial intelligence (AI). Traditional methods are powerful, but they are computationally costly and do not efficiently explore the vast chemical and structural design space. Advances in machine learning have enabled prediction of material properties, accelerated simulations, optimised synthesis conditions, and even inverse design of novel compounds—ranging from supervised, unsupervised, and reinforcement learning to deep neural networks and graph-based architectures. A survey of key methodologies, applications, and related work in machine learning and artificial intelligence-driven computational materials science is presented here. In addition, we outline future directions for physics-based models, autonomous laboratories, and explainable artificial intelligence while discussing the challenges of data quality, transferability, interpretability, and integration with experimental workflows. The pace of innovation in materials science is being accelerated by AI-driven computational approaches. Artificial neural network ; Reinforcement learning ; Key (lock) ; Pace ; Hyper-heuristic ; Computational learning theory ; Deep learning ; Artificial life Machine Learning in Materials Science ; Catalysis and Oxidation Reactions ; Computational Drug Discovery Methods
Authors: Anyang Peng | Chunfang Cai | Mingyu Guo | Duo Zhang | Chengqian Zhang | Wanrun Jiang | Yinan Wang | Antoine Loew | Chengkun Wu | E Weinan | Linfeng Zhang | Han Wang
Source: Research Square (preprint, 2025-09-05)
Field: Materials Science — Materials Chemistry

LAMBench: A Benchmark for Large Atomistic Models Benchmark (surveying) ; Computer science ; Statistical physics ; Physics ; Geography ; Cartography Machine Learning in Materials Science ; Electron and X-Ray Spectroscopy Techniques ; Advanced Materials Characterization Techniques