OpenRaiser/Intern-Atlas huggingface dataset:
SWE-Mirror: Scaling Issue-Resolving Datasets by Mirroring Issues Across Repositories
Venue: arXiv cs.SE 2025
Year: 2025

Creating large-scale verifiable training datasets for issue-resolving tasks is a critical yet notoriously difficult challenge. Existing methods on automating the Gym environment setup process for real-world issues suffer from low success rates and high overhead. Meanwhile, synthesizing new tasks within existing Gym environments leaves the vast pool of authentic, human-reported problems untapped. To maximize the utilization of existing Gym environments and also the rich data of issue-resolving history on GitHub, we introduce SWE-Mirror, a pipeline that distills a real-world issue's semantic essence, mirrors it into another repository with a configured Gym environment, and re-animates it as a verifiable issue-resolving task. SWE-Mirror reuses existing Gym environments along with the vast pool of issue-resolving history hosted on GitHub to construct a large-scale dataset of mirrored authentic and verifiable tasks. Applying SWE-Mirror to 40 repositories across 4 languages, we have curated a dataset with 60,671 issue-resolving tasks and demonstrated the value of our dataset by training and evaluating coding agents at various scale. Post-training experiments show that models trained with the dataset exhibit improvements in issue-resolving capabilities. Furthermore, by extending the dataset size to over 12,000 high-quality trajectories, we established a new state-of-the-art (SOTA) among Qwen2.5-Coder-Instruct based LLMs on the OpenHands agent framework, which increases the resolve rate on SWE-Bench-Verified by +21.8% for the 7B model and +46.0% for the 32B model and validates the effectiveness of our approach.
ChemBOMAS: Accelerated BO in Chemistry with LLM-Enhanced Multi-Agent System
Venue: arXiv cs.LG 2025
Year: 2025

Bayesian optimization (BO) is a powerful tool for scientific discovery in chemistry, yet its efficiency is often hampered by the sparse experimental data and vast search space. Here, we introduce ChemBOMAS: a large language model (LLM)-enhanced multi-agent system that accelerates BO through synergistic data- and knowledge-driven strategies. Firstly, the data-driven strategy involves an 8B-scale LLM regressor fine-tuned on a mere 1% labeled samples for pseudo-data generation, robustly initializing the optimization process. Secondly, the knowledge-driven strategy employs a hybrid Retrieval-Augmented Generation approach to guide LLM in dividing the search space while mitigating LLM hallucinations. An Upper Confidence Bound algorithm then identifies high-potential subspaces within this established partition. Across the LLM-refined subspaces and supported by LLM-generated data, BO achieves the improvement of effectiveness and efficiency. Comprehensive evaluations across multiple scientific benchmarks demonstrate that ChemBOMAS set a new state-of-the-art, accelerating optimization efficiency by up to 5-fold compared to baseline methods.
AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning
Venue: arXiv cs.LG 2025
Year: 2025

Developing autonomous LLM agents capable of making a series of intelligent decisions to solve complex, real-world tasks is a fast-evolving frontier. Like human cognitive development, agents are expected to acquire knowledge and skills through exploration and interaction with the environment. Despite advances, the community still lacks a unified, interactive reinforcement learning (RL) framework that can effectively train such agents from scratch -- without relying on supervised fine-tuning (SFT) -- across diverse and realistic environments. To bridge this gap, we introduce AgentGym-RL, a new framework to train LLM agents for multi-turn interactive decision-making through RL. The framework features a modular and decoupled architecture, ensuring high flexibility and extensibility. It encompasses a wide variety of real-world scenarios, and supports mainstream RL algorithms. Furthermore, we propose ScalingInter-RL, a training approach designed for exploration-exploitation balance and stable RL optimization. In early stages, it emphasizes exploitation by restricting the number of interactions, and gradually shifts towards exploration with larger horizons to encourage diverse problem-solving strategies. In this way, the agent develops more diverse behaviors and is less prone to collapse under long horizons. We perform extensive experiments to validate the stability and effectiveness of both the AgentGym-RL framework and the ScalingInter-RL approach. Our agents match or surpass commercial models on 27 tasks across diverse environments. We offer key insights and will open-source the complete AgentGym-RL framework -- including code and datasets -- to empower the research community in developing the next generation of intelligent agents.
Scaling Truth: The Confidence Paradox in AI Fact-Checking
Venue: arXiv cs.SI 2025
Year: 2025

The rise of misinformation underscores the need for scalable and reliable fact-checking solutions. Large language models (LLMs) hold promise in automating fact verification, yet their effectiveness across global contexts remains uncertain. We systematically evaluate nine established LLMs across multiple categories (open/closed-source, multiple sizes, diverse architectures, reasoning-based) using 5,000 claims previously assessed by 174 professional fact-checking organizations across 47 languages. Our methodology tests model generalizability on claims postdating training cutoffs and four prompting strategies mirroring both citizen and professional fact-checker interactions, with over 240,000 human annotations as ground truth. Findings reveal a concerning pattern resembling the Dunning-Kruger effect: smaller, accessible models show high confidence despite lower accuracy, while larger models demonstrate higher accuracy but lower confidence. This risks systemic bias in information verification, as resource-constrained organizations typically use smaller models. Performance gaps are most pronounced for non-English languages and claims originating from the Global South, threatening to widen existing information inequalities. These results establish a multilingual benchmark for future research and provide an evidence base for policy aimed at ensuring equitable access to trustworthy, AI-assisted fact-checking.
Fluid Antenna Systems: A Geometric Approach to Error Probability and Fundamental Limits
Venue: arXiv cs.IT 2025
Year: 2025

The fluid antenna system (FAS) concept is an emerging paradigm that promotes the utilization of the feature of shape and position reconfigurability in antennas to broaden the design of wireless communication systems. This also means that spatial diversity can be exploited in an unconventional way. However, a rigorous framework for error probability analysis of FAS under realistic spatially correlated channels has been lacking. In this paper, we fill this gap by deriving a tight, closed-form asymptotic expression for the symbol error rate (SER) that establishes the fundamental scaling law linking the system's SER to the channel's spatial correlation structure. A key insight of our analysis is that the achievable diversity gain is governed not by the number of antenna ports, but by the channel's effective rank. To find this critical parameter, we propose a novel dual-pronged approach. First of all, we develop a geometry-based algorithm that extracts distinct performance thresholds from the channel's eigenvalue spectrum. Second, we theoretically prove that the effective rank converges to a fundamental limit dictated solely by the antenna's normalized aperture width. We further establish the equivalence between the threshold identified by the geometric algorithm and the derived theoretical limit, providing rigorous validation for the proposed method. Our effective rank model achieves higher accuracy than existing approaches in the literature. Building on this framework, we offer a complete characterization of diversity and coding gains. The analysis leads to a definitive design insight: FAS performance improvements are fundamentally driven by enlarging the antenna's explorable aperture, which increases the effective channel rank, whereas increasing port density within a fixed aperture yields diminishing returns.
Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation
Venue: arXiv cs.CL 2025
Year: 2025

Large language models are rapidly transforming social science research by enabling the automation of labor-intensive tasks like data annotation and text analysis. However, LLM outputs vary significantly depending on the implementation choices made by researchers (e.g., model selection or prompting strategy). Such variation can introduce systematic biases and random errors, which propagate to downstream analyses and cause Type I (false positive), Type II (false negative), Type S (wrong sign), or Type M (exaggerated effect) errors. We call this phenomenon where configuration choices lead to incorrect conclusions LLM hacking. We find that intentional LLM hacking is strikingly simple. By replicating 37 data annotation tasks from 21 published social science studies, we show that, with just a handful of prompt paraphrases, virtually anything can be presented as statistically significant. Beyond intentional manipulation, our analysis of 13 million labels from 18 different LLMs across 2361 realistic hypotheses shows that there is also a high risk of accidental LLM hacking, even when following standard research practices. We find incorrect conclusions in approximately 31% of hypotheses for state-of-the-art LLMs, and in half the hypotheses for smaller language models. While higher task performance and stronger general model capabilities reduce LLM hacking risk, even highly accurate models remain susceptible. The risk of LLM hacking decreases as effect sizes increase, indicating the need for more rigorous verification of LLM-based findings near significance thresholds. We analyze 21 mitigation techniques and find that human annotations provide crucial protection against false positives. Common regression estimator correction techniques can restore valid inference but trade off Type I vs. Type II errors. We publish a list of practical recommendations to prevent LLM hacking.
RewardDance: Reward Scaling in Visual Generation
Venue: arXiv cs.CV 2025
Year: 2025

Reward Models (RMs) are critical for improving generation models via Reinforcement Learning (RL), yet the RM scaling paradigm in visual generation remains largely unexplored. It primarily due to fundamental limitations in existing approaches: CLIP-based RMs suffer from architectural and input modality constraints, while prevalent Bradley-Terry losses are fundamentally misaligned with the next-token prediction mechanism of Vision-Language Models (VLMs), hindering effective scaling. More critically, the RLHF optimization process is plagued by Reward Hacking issue, where models exploit flaws in the reward signal without improving true quality. To address these challenges, we introduce RewardDance, a scalable reward modeling framework that overcomes these barriers through a novel generative reward paradigm. By reformulating the reward score as the model's probability of predicting a "yes" token, indicating that the generated image outperforms a reference image according to specific criteria, RewardDance intrinsically aligns reward objectives with VLM architectures. This alignment unlocks scaling across two dimensions: (1) Model Scaling: Systematic scaling of RMs up to 26 billion parameters; (2) Context Scaling: Integration of task-specific instructions, reference examples, and chain-of-thought (CoT) reasoning. Extensive experiments demonstrate that RewardDance significantly surpasses state-of-the-art methods in text-to-image, text-to-video, and image-to-video generation. Crucially, we resolve the persistent challenge of "reward hacking": Our large-scale RMs exhibit and maintain high reward variance during RL fine-tuning, proving their resistance to hacking and ability to produce diverse, high-quality outputs. It greatly relieves the mode collapse problem that plagues smaller models.
Pattern-Based File and Data Access with Python Glob: A Comprehensive Guide for Computational Research
Venue: arXiv cs.SE 2025
Year: 2025

Pattern-based file access is a fundamental but often under-documented aspect of computational research. The Python glob module provides a simple yet powerful way to search, filter, and ingest files using wildcard patterns, enabling scalable workflows across disciplines. This paper introduces glob as a versatile tool for data science, business analytics, and artificial intelligence applications. We demonstrate use cases including large-scale data ingestion, organizational data analysis, AI dataset construction, and reproducible research practices. Through concrete Python examples with widely used libraries such as pandas,scikit-learn, and matplotlib, we show how glob facilitates efficient file traversal and integration with analytical pipelines. By situating glob within the broader context of reproducible research and data engineering, we highlight its role as a methodological building block. Our goal is to provide researchers and practitioners with a concise reference that bridges foundational concepts and applied practice, making glob a default citation for file pattern matching in Python-based research workflows.
Uncertainty Estimation using Variance-Gated Distributions
Venue: arXiv cs.LG 2025
Year: 2025

Evaluation of per-sample uncertainty quantification from neural networks is essential for decision-making involving high-risk applications. A common approach is to use the predictive distribution from Bayesian or approximation models and decompose the corresponding predictive uncertainty into epistemic (model-related) and aleatoric (data-related) components. However, additive decomposition has recently been questioned. In this work, we propose an intuitive framework for uncertainty estimation and decomposition based on the signal-to-noise ratio of class probability distributions across different model predictions. We introduce a variance-gated measure that scales predictions by a confidence factor derived from ensembles. We use this measure to discuss the existence of a collapse in the diversity of committee machines.
Morphology-Preserving Remeshing Approach to Particulate Microstructures via Harmonic Decomposition
Venue: arXiv cs.GR 2025
Year: 2025

Harmonic decomposition of surfaces, such as spherical and spheroidal harmonics, is used to analyze morphology, reconstruct, and generate surface inclusions of particulate microstructures. However, obtaining high-quality meshes of engineering microstructures using these approaches remains an open question. In harmonic approaches, we usually reconstruct surfaces by evaluating the harmonic bases on equidistantly sampled simplicial complexes of the base domains (e.g., triangular spheroids and disks). However, this traditional sampling does not account for local changes in the Jacobian of the basis functions, resulting in nonuniform discretization after reconstruction or generation. As it impacts the accuracy and time step, high-quality discretization of microstructures is crucial for efficient numerical simulations (e.g., finite element and discrete element methods). To circumvent this issue, we propose an efficient hierarchical diffusion-based approach for resampling the surface-i.e., performing a reparameterization-to yield an equalized mesh triangulation. Analogous to heat problems, we use nonlinear diffusion to resample the curvilinear coordinates of the analysis domain, thereby enlarging small triangles at the expense of large triangles on surfaces. We tested isotropic and anisotropic diffusion schemes on the recent spheroidal and hemispheroidal harmonics methods. The results show a substantial improvement in the quality metrics for surface triangulation. Unlike traditional surface reconstruction and meshing techniques, this approach preserves surface morphology, along with the areas and volumes of surfaces. We discuss the results and the associated computational costs for large 2D and 3D microstructures, such as digital twins of concrete and stone masonry, and their future applications.
A Systematic Mapping Study on Chatbots in Programming Education
Venue: arXiv cs.SE 2025
Year: 2025

Educational chatbots have gained prominence as support tools for teaching programming, particularly in introductory learning contexts. This paper presents a Systematic Mapping Study (SMS) that investigated how such agents have been developed and applied in programming education. From an initial set of 3,216 publications, 54 studies were selected and analyzed based on five research subquestions, addressing chatbot types, programming languages used, educational content covered, interaction models, and application contexts. The results reveal a predominance of chatbots designed for Python instruction, focusing on fundamental programming concepts, and employing a wide variety of pedagogical approaches and technological architectures. In addition to identifying trends and gaps in the literature, this study provides insights to inform the development of new educational tools for programming instruction.
Benchmarking Energy Efficiency of Large Language Models Using vLLM
Venue: arXiv cs.SE 2025
Year: 2025

The prevalence of Large Language Models (LLMs) is having an growing impact on the climate due to the substantial energy required for their deployment and use. To create awareness for developers who are implementing LLMs in their products, there is a strong need to collect more information about the energy efficiency of LLMs. While existing research has evaluated the energy efficiency of various models, these benchmarks often fall short of representing realistic production scenarios. In this paper, we introduce the LLM Efficiency Benchmark, designed to simulate real-world usage conditions. Our benchmark utilizes vLLM, a high-throughput, production-ready LLM serving backend that optimizes model performance and efficiency. We examine how factors such as model size, architecture, and concurrent request volume affect inference energy efficiency. Our findings demonstrate that it is possible to create energy efficiency benchmarks that better reflect practical deployment conditions, providing valuable insights for developers aiming to build more sustainable AI systems.
Towards Trustworthy AI: Characterizing User-Reported Risks across LLMs "In the Wild"
Venue: arXiv cs.CY 2025
Year: 2025

While Large Language Models (LLMs) are rapidly integrating into daily life, research on their risks often remains lab-based and disconnected from the problems users encounter "in the wild." While recent HCI research has begun to explore these user-facing risks, it typically concentrates on a singular LLM chatbot like ChatGPT or an isolated risk like privacy. To gain a holistic understanding of multi-risk across LLM chatbots, we analyze online discussions on Reddit around seven major LLM chatbots through the U.S. NIST's AI Risk Management Framework. We find that user-reported risks are unevenly distributed and platform-specific. While "Valid and Reliable" risk is the most frequently mentioned, each product also exhibits a unique "risk fingerprint;" for instance, user discussions associate GPT more with "Safe" and "Fair" issues, Gemini with "Privacy," and Claude with "Secure and Resilient" risks. Furthermore, the nature of these risks differs by their prevalence: less frequent risks like "Explainability" and "Privacy" manifest as nuanced user trade-offs, more common ones like "Fairness" are experienced as direct personal harms. Our findings reveal gaps between risks reported by system-centered studies and by users, highlighting the need for user-centered approaches that support users in their daily use of LLM chatbots.
Documents Are People and Words Are Items: A Psychometric Approach to Textual Data with Contextual Embeddings
Venue: arXiv cs.CL 2025
Year: 2025

This research introduces a novel psychometric method for analyzing textual data using large language models. By leveraging contextual embeddings to create contextual scores, we transform textual data into response data suitable for psychometric analysis. Treating documents as individuals and words as items, this approach provides a natural psychometric interpretation under the assumption that certain keywords, whose contextual meanings vary significantly across documents, can effectively differentiate documents within a corpus. The modeling process comprises two stages: obtaining contextual scores and performing psychometric analysis. In the first stage, we utilize natural language processing techniques and encoder based transformer models to identify common keywords and generate contextual scores. In the second stage, we employ various types of factor analysis, including exploratory and bifactor models, to extract and define latent factors, determine factor correlations, and identify the most significant words associated with each factor. Applied to the Wiki STEM corpus, our experimental results demonstrate the method's potential to uncover latent knowledge dimensions and patterns within textual data. This approach not only enhances the psychometric analysis of textual data but also holds promise for applications in fields rich in textual information, such as education, psychology, and law.
Corruption-Tolerant Asynchronous Q-Learning with Near-Optimal Rates
Venue: arXiv cs.LG 2025
Year: 2025

We consider the problem of learning the optimal policy in a discounted, infinite-horizon reinforcement learning (RL) setting where the reward signal is subject to adversarial corruption. Such corruption, which may arise from extreme noise, sensor faults, or malicious attacks, can severely degrade the performance of classical algorithms such as Q-learning. To address this challenge, we propose a new provably robust variant of the Q-learning algorithm that operates effectively even when a fraction of the observed rewards are arbitrarily perturbed by an adversary. Under the asynchronous sampling model with time-correlated data, we establish that despite adversarial corruption, the finite-time convergence rate of our algorithm matches that of existing results for the non-adversarial case, up to an additive term proportional to the fraction of corrupted samples. Moreover, we derive an information-theoretic lower bound revealing that the additive corruption term in our upper bounds is unavoidable. Next, we propose a variant of our algorithm that requires no prior knowledge of the statistics of the true reward distributions. The analysis of this setting is particularly challenging and is enabled by carefully exploiting a refined Azuma-Hoeffding inequality for almost-martingales, a technical tool that might be of independent interest. Collectively, our contributions provide the first finite-time robustness guarantees for asynchronous Q-learning, bridging a significant gap in robust RL.