OpenRaiser/Intern-Atlas huggingface dataset:
Clinical knowledge in LLMs does not translate to human interactions
Venue: arXiv cs.HC 2025
Year: 2025

Global healthcare providers are exploring use of large language models (LLMs) to provide medical advice to the public. LLMs now achieve nearly perfect scores on medical licensing exams, but this does not necessarily translate to accurate performance in real-world settings. We tested if LLMs can assist members of the public in identifying underlying conditions and choosing a course of action (disposition) in ten medical scenarios in a controlled study with 1,298 participants. Participants were randomly assigned to receive assistance from an LLM (GPT-4o, Llama 3, Command R+) or a source of their choice (control). Tested alone, LLMs complete the scenarios accurately, correctly identifying conditions in 94.9% of cases and disposition in 56.3% on average. However, participants using the same LLMs identified relevant conditions in less than 34.5% of cases and disposition in less than 44.2%, both no better than the control group. We identify user interactions as a challenge to the deployment of LLMs for medical advice. Standard benchmarks for medical knowledge and simulated patient interactions do not predict the failures we find with human participants. Moving forward, we recommend systematic human user testing to evaluate interactive capabilities prior to public deployments in healthcare.
Revisiting Transformers through the Lens of Low Entropy and Dynamic Sparsity
Venue: arXiv cs.LG 2025
Year: 2025

Compression has been a critical lens to understand the success of Transformers. In the past, we have typically taken the target distribution as a criterion to evaluate a model's compression performance. Nevertheless,it often remains challenging to precisely assess how well the model achieves compression and to compare the information content of the learned distribution with that of the target distribution during compression,as the target distribution is typically unknown and entropy computation often incurs exponential cost. In this work, we explore these issues under a controlled experimental setup. We find that Transformers exhibit a unique inductive bias in data compression: beyond approaching the target distribution, they tend to favor learning lower-entropy distributions, with this tendency becoming more pronounced as the model size increases. This preference prevents Transformers from perfectly aligning with the target distribution, instead further compressing its information content. Furthermore, we show that the FFN module plays a critical role in driving this bias. In addition, while models remove informational redundancy from data during compression, they also exhibit redundancy within their parameters, which enables compression and can be characterized through dynamic sparsity. However, the dynamic sparsity patterns in Transformers, particularly in attention and FFN modules, demand further exploration. As for this, we show that larger Transformers show stronger preferences for bypassing attention computations via residual connections and have lower proportion of active neurons. Interestingly, we also find that training instability in larger models strongly correlates with sudden increases in dead neurons. Our work contributes to a deeper understanding of Transformers from the lens of entropy and dynamic sparsity.
LawFlow: Collecting and Simulating Lawyers' Thought Processes on Business Formation Case Studies
Venue: arXiv cs.CL 2025
Year: 2025

Legal practitioners, particularly those early in their careers, face complex, high-stakes tasks that require adaptive, context-sensitive reasoning. While AI holds promise in supporting legal work, current datasets and models are narrowly focused on isolated subtasks and fail to capture the end-to-end decision-making required in real-world practice. To address this gap, we introduce LawFlow, a dataset of complete end-to-end legal workflows collected from trained law students, grounded in real-world business entity formation scenarios. Unlike prior datasets focused on input-output pairs or linear chains of thought, LawFlow captures dynamic, modular, and iterative reasoning processes that reflect the ambiguity, revision, and client-adaptive strategies of legal practice. Using LawFlow, we compare human and LLM-generated workflows, revealing systematic differences in structure, reasoning flexibility, and plan execution. Human workflows tend to be modular and adaptive, while LLM workflows are more sequential, exhaustive, and less sensitive to downstream implications. Our findings also suggest that legal professionals prefer AI to carry out supportive roles, such as brainstorming, identifying blind spots, and surfacing alternatives, rather than executing complex workflows end-to-end. Our results highlight both the current limitations of LLMs in supporting complex legal workflows and opportunities for developing more collaborative, reasoning-aware legal AI systems. All data and code are available on our project page (https://minnesotanlp.github.io/LawFlow-website/).
Towards Automated Detection of Inline Code Comment Smells
Venue: arXiv cs.SE 2025
Year: 2025

Code comments are important in software development because they directly influence software maintainability and overall quality. Bad practices of code comments lead to code comment smells, negatively impacting software maintenance. Recent research has been conducted on classifying inline code comment smells, yet automatically detecting these still remains a challenge. We aim to automatically detect and classify inline code comment smells through machine learning (ML) models and a large language model (LLM) to determine how accurately each smell type can be detected. We enhanced a previously labeled dataset, where comments are labeled according to a determined taxonomy, by augmenting it with additional code segments and their associated comments. GPT 4, a large language model, was used to classify code comment smells on both the original and augmented datasets to evaluate its performance. In parallel, we trained and tested seven different machine learning algorithms on the augmented dataset to compare their classification performance against GPT 4. The performance of models, particularly Random Forest, which achieved an overall accuracy of 69 percent, along with Gradient Boosting and Logistic Regression, each achieving 66 percent and 65 percent, respectively, establishes a solid baseline for future research in this domain. The Random Forest model outperformed all other ML models, by achieving the highest Matthews Correlation Coefficient (MCC) score of 0.44. The augmented dataset improved the overall classification accuracy of the GPT 4 model predictions from 34 percent to 55 percent. This study contributes to software maintainability by exploring the automatic detection and classification of inline code comment smells. We have made our augmented dataset and code artifacts available online, offering a valuable resource for developing automated comment smell detection tools.
Redefining Hybrid Blockchains: A Balanced Architecture
Venue: arXiv cs.CR 2025
Year: 2025

Blockchain technology has completely revolutionized the field of decentralized finance with the emergence of a variety of cryptocurrencies and digital assets. However, widespread adoption of this technology by governments and enterprises has been limited by concerns regarding the technology's scalability, governance, and economic sustainability. This paper aims to introduce a novel hybrid blockchain architecture that balances scalability, governance, and decentralization while being economically viable for all parties involved. The new semi-centralized model leverages strategies not prevalent in the field, such as resource and node isolation, containerization, separation of networking and compute layers, use of a Kafka pub-sub network instead of a peer-to-peer network, and stakes-based validator selection to possibly mitigate a variety of issues related to scalability, security, governance, and economic sustainability. Simulations conducted on Kubernetes demonstrate the architecture's ability to achieve over 1000 transactions per second, with consistent performance across scaled deployments, even on a lightweight consumer-grade laptop with resource constraints. The findings highlight the system's scalability, security, and economic viability, offering a robust framework for enterprise and government adoption.
Advancing Face-to-Face Emotion Communication: A Multimodal Dataset (AFFEC)
Venue: arXiv cs.HC 2025
Year: 2025

Emotion recognition has the potential to play a pivotal role in enhancing human-computer interaction by enabling systems to accurately interpret and respond to human affect. Yet, capturing emotions in face-to-face contexts remains challenging due to subtle nonverbal cues, variations in personal traits, and the real-time dynamics of genuine interactions. Existing emotion recognition datasets often rely on limited modalities or controlled conditions, thereby missing the richness and variability found in real-world scenarios. In this work, we introduce Advancing Face-to-Face Emotion Communication (AFFEC), a multimodal dataset designed to address these gaps. AFFEC encompasses 84 simulated emotional dialogues across six distinct emotions, recorded from 73 participants over more than 5,000 trials and annotated with more than 20,000 labels. It integrates electroencephalography (EEG), eye-tracking, galvanic skin response (GSR), facial videos, and Big Five personality assessments. Crucially, AFFEC explicitly distinguishes between felt emotions (the participant's internal affect) and perceived emotions (the observer's interpretation of the stimulus). Baseline analyses spanning unimodal features and straightforward multimodal fusion demonstrate that even minimal processing yields classification performance significantly above chance, especially for arousal. Incorporating personality traits further improves predictions of felt emotions, highlighting the importance of individual differences. By bridging controlled experimentation with more realistic face-to-face stimuli, AFFEC offers a unique resource for researchers aiming to develop context-sensitive, adaptive, and personalized emotion recognition models.
Scientific Open-Source Software Is Less Likely to Become Abandoned Than One Might Think! Lessons from Curating a Catalog of Maintained Scientific Software
Venue: arXiv cs.SE 2025
Year: 2025

Scientific software is essential to scientific innovation and in many ways it is distinct from other types of software. Abandoned (or unmaintained), buggy, and hard to use software, a perception often associated with scientific software can hinder scientific progress, yet, in contrast to other types of software, its longevity is poorly understood. Existing data curation efforts are fragmented by science domain and/or are small in scale and lack key attributes. We use large language models to classify public software repositories in World of Code into distinct scientific domains and layers of the software stack, curating a large and diverse collection of over 18,000 scientific software projects. Using this data, we estimate survival models to understand how the domain, infrastructural layer, and other attributes of scientific software affect its longevity. We further obtain a matched sample of non-scientific software repositories and investigate the differences. We find that infrastructural layers, downstream dependencies, mentions of publications, and participants from government are associated with a longer lifespan, while newer projects with participants from academia had shorter lifespan. Against common expectations, scientific projects have a longer lifetime than matched non-scientific open-source software projects. We expect our curated attribute-rich collection to support future research on scientific software and provide insights that may help extend longevity of both scientific and other projects.
Sparks: Multi-Agent Artificial Intelligence Model Discovers Protein Design Principles
Venue: arXiv cs.AI 2025
Year: 2025

Advances in artificial intelligence (AI) promise autonomous discovery, yet most systems still resurface knowledge latent in their training data. We present Sparks, a multi-modal multi-agent AI model that executes the entire discovery cycle that includes hypothesis generation, experiment design and iterative refinement to develop generalizable principles and a report without human intervention. Applied to protein science, Sparks uncovered two previously unknown phenomena: (i) a length-dependent mechanical crossover whereby beta-sheet-biased peptides surpass alpha-helical ones in unfolding force beyond ~80 residues, establishing a new design principle for peptide mechanics; and (ii) a chain-length/secondary-structure stability map revealing unexpectedly robust beta-sheet-rich architectures and a "frustration zone" of high variance in mixed alpha/beta folds. These findings emerged from fully self-directed reasoning cycles that combined generative sequence design, high-accuracy structure prediction and physics-aware property models, with paired generation-and-reflection agents enforcing self-correction and reproducibility. The key result is that Sparks can independently conduct rigorous scientific inquiry and identify previously unknown scientific principles.
GLaMoR: Consistency Checking of OWL Ontologies using Graph Language Models
Venue: arXiv cs.AI 2025
Year: 2025

Semantic reasoning aims to infer new knowledge from existing knowledge, with OWL ontologies serving as a standardized framework for organizing information. A key challenge in semantic reasoning is verifying ontology consistency. However, state-of-the-art reasoners are computationally expensive, and their efficiency decreases as ontology sizes grow. While classical machine learning models have been explored for consistency checking, they struggle to capture complex relationships within ontologies. Large language models (LLMs) have shown promising results for simple reasoning tasks but perform poorly on structured reasoning. The recently introduced Graph Language Model (GLM) offers a way to simultaneously process graph-structured data and text. This paper proposes GLaMoR (Graph Language Model for Reasoning), a reasoning pipeline that transforms OWL ontologies into graph-structured data and adapts the GLM architecture for consistency checking. We evaluate GLaMoR on ontologies from the NCBO BioPortal repository, converting them into triples suitable for model input. Our results show that the GLM outperforms all baseline models, achieving $95\%$ accuracy while being 20 times faster than classical reasoners. The Code is accessible under: https://github.com/JustinMuecke/GLaMoR
Efficient Control Allocation and 3D Trajectory Tracking of a Highly Manoeuvrable Under-actuated Bio-inspired AUV
Venue: arXiv cs.RO 2025
Year: 2025

Fin actuators can be used for for both thrust generation and vectoring. Therefore, fin-driven autonomous underwater vehicles (AUVs) can achieve high maneuverability with a smaller number of actuators, but their control is challenging. This study proposes an analytic control allocation method for underactuated Autonomous Underwater Vehicles (AUVs). By integrating an adaptive hybrid feedback controller, we enable an AUV with 4 actuators to move in 6 degrees of freedom (DOF) in simulation and up to 5-DOF in real-world experiments. The proposed method outperformed state-of-the-art control allocation techniques in 6-DOF trajectory tracking simulations, exhibiting centimeter-scale accuracy and higher energy and computational efficiency. Real-world pool experiments confirmed the method's robustness and efficacy in tracking complex 3D trajectories, with significant computational efficiency gains 0.007 (ms) vs. 22.28 (ms). Our method offers a balance between performance, energy efficiency, and computational efficiency, showcasing a potential avenue for more effective tracking of a large number of DOF for under-actuated underwater robots.
BinPool: A Dataset of Vulnerabilities for Binary Security Analysis
Venue: arXiv cs.CR 2025
Year: 2025

The development of machine learning techniques for discovering software vulnerabilities relies fundamentally on the availability of appropriate datasets. The ideal dataset consists of a large and diverse collection of real-world vulnerabilities, paired so as to contain both vulnerable and patched versions of each program. Naturally, collecting such datasets is a laborious and time-consuming task. Within the specific domain of vulnerability discovery in binary code, previous datasets are either publicly unavailable, lack semantic diversity, involve artificially introduced vulnerabilities, or were collected using static analyzers, thereby themselves containing incorrectly labeled example programs. In this paper, we describe a new publicly available dataset which we dubbed Binpool, containing numerous samples of vulnerable versions of Debian packages across the years. The dataset was automatically curated, and contains both vulnerable and patched versions of each program, compiled at four different optimization levels. Overall, the dataset covers 603 distinct CVEs across 89 CWE classes, 162 Debian packages, and contains 6144 binaries. We argue that this dataset is suitable for evaluating a range of security analysis tools, including for vulnerability discovery, binary function similarity, and plagiarism detection.
Vessel Length Estimation from Magnetic Wake Signature: A Physics-Informed Residual Neural Network Approach
Venue: arXiv cs.NE 2025
Year: 2025

Marine remote sensing enhances maritime surveillance, environmental monitoring, and naval operations. Vessel length estimation, a key component of this technology, supports effective maritime surveillance by empowering features such as vessel classification. Departing from traditional methods relying on two-dimensional hydrodynamic wakes or computationally intensive satellite imagery, this paper introduces an innovative approach for vessel length estimation that leverages the subtle magnetic wake signatures of vessels, captured through a low-complexity one-dimensional profile from a single airborne magnetic sensor scan. The proposed method centers around our characterized nonlinear integral equations that connect the magnetic wake to the vessel length within a realistic finite-depth marine environment. To solve the derived equations, we initially leverage a deep residual neural network (DRNN). The proposed DRNN-based solution framework is shown to be unable to exactly learn the intricate relationships between parameters when constrained by a limited training-dataset. To overcome this issue, we introduce an innovative approach leveraging a physics-informed residual neural network (PIRNN). This model integrates physical formulations directly into the loss function, leading to improved performance in terms of both accuracy and convergence speed. Considering a sensor scan angle of less than $15^\circ$, which maintains a reasonable margin below Kelvin's limit angle of $19.5^\circ$, we explore the impact of various parameters on the accuracy of the vessel length estimation, including sensor scan angle, vessel speed, and sea depth. Numerical simulations demonstrate the superiority of the proposed PIRNN method, achieving mean length estimation errors consistently below 5\% for vessels longer than 100m. For shorter vessels, the errors generally remain under 10\%.
PAD: Phase-Amplitude Decoupling Fusion for Multi-Modal Land Cover Classification
Venue: arXiv cs.CV 2025
Year: 2025

The fusion of Synthetic Aperture Radar (SAR) and RGB imagery for land cover classification remains challenging due to modality heterogeneity and underexploited spectral complementarity. Existing approaches often fail to decouple shared structural features from modality-complementary radiometric attributes, resulting in feature conflicts and information loss. To address this, we propose Phase-Amplitude Decoupling (PAD), a frequency-aware framework that separates phase (modality-shared) and amplitude (modality-complementary) components in the Fourier domain. This design reinforces shared structures while preserving complementary characteristics, thereby enhancing fusion quality. Unlike previous methods that overlook the distinct physical properties encoded in frequency spectra, PAD explicitly introduces amplitude-phase decoupling for multi-modal fusion. Specifically, PAD comprises two key components: 1) Phase Spectrum Correction (PSC), which aligns cross-modal phase features via convolution-guided scaling to improve geometric consistency; and 2) Amplitude Spectrum Fusion (ASF), which dynamically integrates high- and low-frequency patterns using frequency-adaptive multilayer perceptrons, effectively exploiting SAR's morphological sensitivity and RGB's spectral richness. Extensive experiments on WHU-OPT-SAR and DDHR-SK demonstrate state-of-the-art performance. This work establishes a new paradigm for physics-aware multi-modal fusion in remote sensing. The code will be available at https://github.com/RanFeng2/PAD.
ChiseLLM: Unleashing the Power of Reasoning LLMs for Chisel Agile Hardware Development
Venue: arXiv cs.AI 2025
Year: 2025

The growing demand for Domain-Specific Architecture (DSA) has driven the development of Agile Hardware Development Methodology (AHDM). Hardware Construction Language (HCL) like Chisel offers high-level abstraction features, making it an ideal language for HCL-Based AHDM. While Large Language Models (LLMs) excel in code generation tasks, they still face challenges with Chisel generation, particularly regarding syntax correctness and design variability. Recent reasoning models have significantly enhanced code generation capabilities through test-time scaling techniques. However, we found that reasoning models without domain adaptation cannot bring substantial benefits to Chisel code generation tasks. This paper presents ChiseLLM, a solution comprising data processing and transformation, prompt-guided reasoning trace synthesis, and domain-adapted model training. We constructed high-quality datasets from public RTL code resources and guided the model to adopt structured thinking patterns through prompt enhancement methods. Experiments demonstrate that our ChiseLLM-7B and ChiseLLM-32B models improved syntax correctness by 18.85% and 26.32% respectively over base models, while increasing variability design ability by 47.58% compared to baseline reasoning models. Our datasets and models are publicly available, providing high-performance, cost-effective models for HCL-Based AHDM, and offering an effective baseline for future research. Github repository: https://github.com/observerw/ChiseLLM
The Trichotomy of Regular Property Testing
Venue: arXiv cs.DS 2025
Year: 2025

Property testing is concerned with the design of algorithms making a sublinear number of queries to distinguish whether the input satisfies a given property or is far from having this property. A seminal paper of Alon, Krivelevich, Newman, and Szegedy in 2001 introduced property testing of formal languages: the goal is to determine whether an input word belongs to a given language, or is far from any word in that language. They constructed the first property testing algorithm for the class of all regular languages. This opened a line of work with improved complexity results and applications to streaming algorithms. In this work, we show a trichotomy result: the class of regular languages can be divided into three classes, each associated with an optimal query complexity. Our analysis yields effective characterizations for all three classes using so-called minimal blocking sequences, reasoning directly and combinatorially on automata.