jablonkagroup/rise_ai_scientists huggingface dataset:
Authors: Gao Tianxi | Cai Yufan | Yuan Yusi | Dong Jin Song
Source: Open MIND (preprint, 2026-03-05)
Field: Materials Science — Materials Chemistry

X-RAY: Mapping LLM Reasoning Capability via Formalized and Calibrated Probes Large language models (LLMs) achieve promising performance, yet their ability to reason remains poorly understood. Existing evaluations largely emphasize task-level accuracy, often conflating pattern matching with reasoning capability. We present X-RAY, an explainable reasoning analysis system that maps the LLM reasoning capability using calibrated, formally verified probes. We model reasoning capability as a function of extractable \textit{structure}, operationalized through formal properties such as constraint interaction, reasoning depth, and solution-space geometry. X-Ray generates probes via formal tools with controlled structural variations, enabling precise isolation of incremental structural information through formal calibration and verification. We evaluate state-of-the-art LLMs on problems ranging from junior-level to advanced in mathematics, physics, and chemistry. Our analysis reveals a systematic asymmetry in LLM reasoning: models are relatively robust to constraint refinement, where additional conditions shrink an existing solution space, but degrade sharply under solution-space restructuring, where modifications alter the underlying structural form of the solution manifold. Moreover, calibrated formal probes differentiate models that appear indistinguishable on standard benchmarks and reveal failure modes that are structurally interpretable rather than opaque. Beyond evaluation, our framework is contamination-free and supports the training and testing of reasoning models. Constraint (computer-aided design) ; Automated reasoning ; Matching (statistics) ; Operationalization ; Reasoning system ; Spatial intelligence ; Function (biology) ; Isolation (microbiology) ; Calibration Machine Learning in Materials Science ; Topic Modeling ; Explainable Artificial Intelligence (XAI)
Authors: Youcef Bagdad | Bruno O. Villoutreix
Source: ChemRxiv (article, 2026-02-05)
Field: Materials Science — Materials Chemistry

Natural Language Control of Computational Chemistry Software « DataWarrior » Through Macro Orchestration We present a tool that enables users to control the DataWarrior cheminformatics platform using natural language commands. DataWarrior is a powerful open-source tool for computational chemistry, but mastering its graphical interface to execute operations requires time and expertise, making it sometimes difficult for non-experts to leverage its full potential. Our solution employs specialized AI agents powered by Large Language Models (LLMs) that interact with DataWarrior's built-in macro system (automation scripts) through Model Context Protocol (MCP). These AI agents understand user requests, select appropriate macros, modify their parameters and execute them accurately to bridge the gap between what users want to do and how the software executes it. Beyond workflow automation, the system enables users to query their datasets directly through natural language, asking analytical questions and receiving grounded responses based on actual data values. The system works with both open-source models that can be installed locally on the user's machine (such as LLaMA 3.1 via Ollama) and cloud-based services (GPT, Claude), making it accessible and flexible. The system currently implements conversational dataset querying and core cheminformatics operations including descriptor calculation, filtering, column deletion, and visualization, with ongoing work to integrate DataWarrior's complete catalog of capabilities. Workflow ; Cheminformatics ; Macro ; Leverage (statistics) ; Software ; Graphical user interface ; Natural language ; Context (archaeology) ; Orchestration ; Scripting language Machine Learning in Materials Science ; Scientific Computing and Data Management ; Biomedical Text Mining and Ontologies
Authors: Ethan Holbrook | Juan C. Verduzco | Alejandro Strachan
Source: arXiv (Cornell University) (preprint, 2026-03-21)
Field: Materials Science — Materials Chemistry

Evaluating LLM-generated code for domain-specific languages: molecular dynamics with LAMMPS Large language models (LLMs) are changing the way researchers interact with code and data in scientific computing. While their ability to generate general-purpose code is well established, their effectiveness in producing scientifically valid code/input scripting for domain-specific languages (DSLs) remains largely unexplored. We propose an evaluation procedure that enables domain experts (who may not be experts in the DSL) to assess the validity of LLM-generated input files for LAMMPS, a widely used molecular dynamics (MD) code, and to use those assessments to evaluate the performance of state-of-the-art LLMs and identify common issues. Key to the evaluation procedure are a normalization step to generate canonical files and an extensible parser for syntax analysis. The following steps isolate common errors without incurring costly tests (in time and computational resources). Once a working input file is generated, LLMs can accelerate verification tests. Our findings highlight limitations of LLMs in generating scientific DSLs and a practical path forward for their integration into domain-specific computational ecosystems by domain experts. Scripting language ; Parsing ; Domain (mathematical analysis) ; Code (set theory) ; Key (lock) ; Syntax ; Normalization (sociology) ; Compiler Machine Learning in Materials Science ; Natural Language Processing Techniques ; Model-Driven Software Engineering Techniques
Authors: Russ McBride
Source: Strategy Science (article, 2026-01-01)
Field: Materials Science — Materials Chemistry

Mean Articulation Machines The performance of large language models (LLMs), both good and bad, derives from their core architecture as text pattern detection and generation machines that are sensitive to the frequency of the data upon which they are trained. They are amazing “mean articulation machines” in this sense. Using conceptual analysis and recent benchmark data, the paper identifies those strategic tasks that fall within the reliable competence of LLMs and those that remain fundamentally misaligned with LLM’s associationistic architecture. The result is a practical continuum identifying where LLMs offer genuine leverage and where human cognition remains indispensable. The most challenging tasks—novel scientific and strategic breakthroughs—are currently out of reach for LLMs because of inherent limitations in their architecture. Because breakthroughs are described with text does not imply that we can simply mine text for the next novel breakthrough. In clarifying the boundary of current LLM capabilities, the paper aims to help strategic decision makers deploy these tools more effectively as powerful assistants for the majority of tasks that lie on the tractable side of the continuum. History: Accepted for the Special Issue: Can AI Do Strategy? Supplemental Material: The online appendix is available at https://doi.org/10.1287/stsc.2025.0439 . Leverage (statistics) ; Competence (human resources) ; Strategic planning ; Architecture ; Articulation (sociology) ; Benchmark (surveying) ; Competence-based management Machine Learning in Materials Science ; Topic Modeling ; Computational and Text Analysis Methods
Authors: Danny Arlen de Jesus Gomez-Ramirez
Source: Open MIND (preprint, 2026-02-17)
Field: Materials Science — Materials Chemistry

On The Main Challenges And New Perspectives On Generating Robust Benchmark Datasets For Large Language Models In Modern Advanced Mathematics Large language models write fluent prose yet still struggle with verifiable, compositional reasoning in advanced mathematics; we address this gap with a compact, cognitively grounded protocol that mirrors how mathematicians think. Our framework instantiates seven human--inspired dimensions--concept formation, dualization, negative knowledge, transfer, and more--via meta--prompts drawn from active research problems, not toy exercises, and audits full solution traces for faithfulness and invariant control. Under identical conditions, we benchmark four state--of-the--art systems and observe a global breaking degree of more than ninety percent on stress tests. In general terms, error forensics reveal systematic failures in lemma synthesis, long--horizon planning, premise selection, and counterexample search. From these findings we suggest the systematic integration of the aforementioned new tactic to enhance concrete levers--rationale SFT, process supervision with process reward models, and stepwise preference learning--that directly target step--level correctness. We further outline an Artificial Mathematical Intelligence (AMI) agenda to model concept creation and proof discovery along these lines. Together, the protocol and interventions chart a reproducible path toward the systematic design of genuinely creative mathematical reasoning in LLMs and related IA--based systems. Counterexample ; Lemma (botany) ; Benchmark (surveying) ; Premise ; Process (computing) ; Automated reasoning ; Invariant (physics) Machine Learning in Materials Science ; Mathematics, Computing, and Information Processing ; Topic Modeling
Authors: Danny Arlen de Jesus Gomez-Ramirez
Source: Zenodo (CERN European Organization for Nuclear Research) (preprint, 2026-02-17)
Field: Materials Science — Materials Chemistry

On The Main Challenges And New Perspectives On Generating Robust Benchmark Datasets For Large Language Models In Modern Advanced Mathematics Large language models write fluent prose yet still struggle with verifiable, compositional reasoning in advanced mathematics; we address this gap with a compact, cognitively grounded protocol that mirrors how mathematicians think. Our framework instantiates seven human--inspired dimensions--concept formation, dualization, negative knowledge, transfer, and more--via meta--prompts drawn from active research problems, not toy exercises, and audits full solution traces for faithfulness and invariant control. Under identical conditions, we benchmark four state--of-the--art systems and observe a global breaking degree of more than ninety percent on stress tests. In general terms, error forensics reveal systematic failures in lemma synthesis, long--horizon planning, premise selection, and counterexample search. From these findings we suggest the systematic integration of the aforementioned new tactic to enhance concrete levers--rationale SFT, process supervision with process reward models, and stepwise preference learning--that directly target step--level correctness. We further outline an Artificial Mathematical Intelligence (AMI) agenda to model concept creation and proof discovery along these lines. Together, the protocol and interventions chart a reproducible path toward the systematic design of genuinely creative mathematical reasoning in LLMs and related IA--based systems. Counterexample ; Lemma (botany) ; Benchmark (surveying) ; Premise ; Process (computing) ; Automated reasoning ; Invariant (physics) Machine Learning in Materials Science ; Mathematics, Computing, and Information Processing ; Topic Modeling
Authors: María Fernanda Hernández Rodríguez | Endler Marcel Borges
Source: Journal of Chemical Education (article, 2026-03-18)
Field: Chemistry — Physical and Theoretical Chemistry

Teaching Statistics and Chemometrics Using Large Language Models: The Palmer Penguins Case Study This study investigates the use of large language models (LLMs) to support the teaching of statistics and chemometrics within a STEAM framework. Thirty-nine questions involving correlation plots, Pearson correlation heatmaps, box plots, histograms with kernel density estimates, hypothesis testing, post hoc analyses, and pairwise scatter matrices were applied to three data sets: the Palmer Penguins data set, used to introduce foundational statistical concepts, and two chemistry-oriented data sets (wine and COVID-19 antibody profiling data; Anal. Chem. 2022, 94, 6529–6539). The activity was structured in three stages. First, students submitted the questions directly to the LLMs using natural-language prompts and evaluated the generated responses. Second, results were independently verified using Jamovi to assess numerical accuracy and appropriate statistical test selection. Third, to ensure computational reproducibility, Gemini was prompted to generate R scripts to reproduce the analyses. When the 39 questions were submitted to Microsoft Copilot (GPT-5) and Google Gemini (3), both models produced consistent statistical results without numerical discrepancies relative to independent software verification. The R scripts generated by Gemini reproduced the analyses successfully; minor graphical rendering issues were resolved by correcting the syntax based on R console feedback. These findings indicate that LLMs can function as reliable pedagogical tools for teaching statistics and chemometrics, facilitating data visualization, hypothesis testing, and multivariate exploration while maintaining reproducibility when outputs are critically verified. Scripting language ; Exploratory data analysis ; Chemometrics ; Pairwise comparison ; Bayesian statistics ; Software ; Statistical hypothesis testing ; Data exploration ; Statistical model ; Data visualization Various Chemistry Research Topics ; Computational Drug Discovery Methods ; Genetics, Bioinformatics, and Biomedical Research
Authors: Robinson Ferrer | Damla Turgut | Zhongzhou Chen | Shashank Sonkar
Source: arXiv (Cornell University) (preprint, 2026-03-31)
Field: Materials Science — Materials Chemistry

When Can We Trust LLM Graders? Calibrating Confidence for Automated Assessment Large Language Models (LLMs) show promise for automated grading, but their outputs can be unreliable. Rather than improving grading accuracy directly, we address a complementary problem: \textit{predicting when an LLM grader is likely to be correct}. This enables selective automation where high-confidence predictions are processed automatically while uncertain cases are flagged for human review. We compare three confidence estimation methods (self-reported confidence, self-consistency voting, and token probability) across seven LLMs of varying scale (4B to 120B parameters) on three educational datasets: RiceChem (long-answer chemistry), SciEntsBank, and Beetle (short-answer science). Our experiments reveal that self-reported confidence consistently achieves the best calibration across all conditions (avg ECE 0.166 vs 0.229 for self-consistency). Surprisingly, self-consistency remains 38\% worse despite requiring 5$\times$ the inference cost. Larger models exhibit substantially better calibration though gains vary by dataset and method (e.g., a 28\% ECE reduction for self-reported), with GPT-OSS-120B achieving the best calibration (avg ECE 0.100) and strong discrimination (avg AUC 0.668). We also observe that confidence is strongly top-skewed across methods, creating a ``confidence floor'' that practitioners must account for when setting thresholds. These findings suggest that simply asking LLMs to report their confidence provides a practical approach for identifying reliable grading predictions. Code is available \href{https://github.com/sonkar-lab/llm_grading_calibration}{here}. Grading (engineering) ; Inference ; Calibration ; Confidence interval ; Automation ; Low Confidence Machine Learning in Materials Science ; Computational and Text Analysis Methods ; Topic Modeling
Authors: Youhe Jiang | Ran Yan | You Peng | Wenshuang Li | Taiyi Wang | Fangcheng Fu | Binhang Yuan
Source: arXiv (Cornell University) (preprint, 2026-04-08)
Field: Materials Science — Materials Chemistry

Autopoiesis: A Self-Evolving System Paradigm for LLM Serving Under Runtime Dynamics Modern Large Language Model (LLM) serving operates in highly volatile environments characterized by severe runtime dynamics, such as workload fluctuations and elastic cluster autoscaling. Traditional serving systems rely on static, human-engineered serving policies (e.g., scheduling algorithms and rescheduling strategies) to manage these dynamics. However, these policies must navigate deeply intertwined runtime trade-offs (e.g., scheduling overhead vs. execution efficiency, rescheduling frequency vs. reconfiguration overhead), whose optimal balance is workload-specific and shifts continuously as runtime conditions evolve, rendering any fixed policy fundamentally unable to adapt. We propose Autopoiesis, a novel online self-evolving system that shifts LLM serving from static policy deployment to continuous online policy evolution. First, Autopoiesis introduces an LLM-driven program synthesis workflow to evolve serving policies with respect to real-time observed dynamics, where the evolved policies reflect the optimal decision in navigating the complex, multi-dimensional trade-off space. Second, Autopoiesis enables this synthesis process to operate continuously during serving, observing real-world system behavior, and rewriting the policy code as runtime trade-offs shift, thereby transforming policy design from a one-time offline endeavor into an ongoing system component, enabling autonomous adaptation to evolving runtime conditions. Together, we establish a new paradigm: Serving policies are no longer static artifacts designed by humans before deployment, but living code that LLMs continuously evolve throughout deployment to navigate runtime trade-offs beyond human design. We evaluate Autopoiesis across diverse runtime dynamics and show up to 53% and on average 34% improvements over state-of-the-art LLM serving systems. Control reconfiguration ; Workflow ; Software deployment ; Scheduling (production processes) ; Runtime system ; Runtime verification ; Compiler ; Workload ; Process (computing) Machine Learning in Materials Science ; Natural Language Processing Techniques ; Big Data and Digital Economy
Authors: Tynan J. Perez | Rafael Gomez-Bombarelli
Source: arXiv (Cornell University) (preprint, 2026-03-17)
Field: Materials Science — Materials Chemistry

Self-Conditioned Denoising for Atomistic Representation Learning The success of large-scale pretraining in NLP and computer vision has catalyzed growing efforts to develop analogous foundation models for the physical sciences. However, pretraining strategies using atomistic data remain underexplored. To date, large-scale supervised pretraining on DFT force-energy labels has provided the strongest performance gains to downstream property prediction, out-performing existing methods of self-supervised learning (SSL) which remain limited to ground-state geometries, and/or single domains of atomistic data. We address these shortcomings with Self-Conditioned Denoising (SCD), a backbone-agnostic reconstruction objective that utilizes self-embeddings for conditional denoising across any domain of atomistic data, including small molecules, proteins, periodic materials, and 'non-equilibrium' geometries. When controlled for backbone architecture and pretraining dataset, SCD significantly outperforms previous SSL methods on downstream benchmarks and matches or exceeds the performance of supervised force-energy pretraining. We show that a small, fast GNN pretrained by SCD can achieve competitive or superior performance to larger models pretrained on significantly larger labeled or unlabeled datasets, across tasks in multiple domains. Our code is available at: https://github.com/TyJPerez/SelfConditionedDenoisingAtoms Code (set theory) ; Representation (politics) ; Noise reduction ; Property (philosophy) ; Domain (mathematical analysis) ; Labeled data ; Supervised learning ; Pattern recognition (psychology) ; Feature learning Machine Learning in Materials Science ; Advanced Electron Microscopy Techniques and Applications ; Quantum many-body systems
Authors: Wang Zixian
Source: Open MIND (preprint, 2026-01-18)
Field: Materials Science — Materials Chemistry

Orthogonalized Policy Optimization:Policy Optimization as Orthogonal Projection in Hilbert Space We propose Orthogonalized Policy Optimization (OPO), a principled framework for large language model alignment derived from optimization in the Hilbert function space L2(pi_k). Lifting policy updates from the probability simplex into L2(pi_k) transforms the nonlinear normalization constraint into a linear orthogonality condition _{pi_k} = 0 on the density fluctuation field v = pi/pi_k - 1. By the Hilbert projection theorem, the unique closed-form update is v_star = (omega_alpha - E[omega_alpha]) / mu, where the subtracted mean acts as a chemical potential enforcing probability conservation. This interpretation reveals advantage z-score normalization as a conservation-law projection rather than a variance-reduction heuristic. OPO cleanly decouples sampling geometry, controlled by the escort exponent alpha, from optimization geometry, governed by the stiffness parameter mu, a separation not attainable under KL-based objectives. The same update can also be derived as a Euclidean mirror-descent step and as the linear-response law of near-equilibrium statistical mechanics, establishing its structural uniqueness within ratio geometry. Structurally, OPO induces constant curvature, non-saturating linear gradient dynamics, and an intrinsic chi-square trust region. Experiments on MATH benchmarks show that the Hilbert projection formulation prevents gradient saturation typical of KL-constrained methods. By sustaining non-vanishing gradients in high-confidence regimes, OPO avoids premature plateaus and achieves stronger long-horizon training rewards and improved out-of-distribution generalization compared to clipping-based baselines. Hilbert space ; Uniqueness ; Projection (relational algebra) ; Simplex ; Euclidean space ; Orthogonality ; Nonlinear system ; Optimization problem ; Gradient descent ; Nonlinear programming Machine Learning in Materials Science ; Reinforcement Learning in Robotics ; Stochastic Gradient Optimization Techniques
Authors: Samuel Edusa
Source: Research Square (preprint, 2026-04-13)
Field: Materials Science — Materials Chemistry

ConsultChain: Progressive Context Distillation Across Heterogeneous LLM Fleets for Token-Optimal Inference Security token ; Context (archaeology) ; Inference ; Workflow ; Cloud computing ; Context switch ; Inference engine ; Latency (audio) Machine Learning in Materials Science ; Domain Adaptation and Few-Shot Learning ; Ferroelectric and Negative Capacitance Devices