Orionnnn/hub-stats huggingface dataset:
GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection
  Behavior
Authors: name: Penghao Wu, name: Shengnan Ma, name: Bo Wang, name: Jiaheng Yu, name: Lewei Lu, name: Ziwei Liu
Published: 2025-06-09 17:59:57

Multimodal Large Language Models (MLLMs) have shown great potential in
revolutionizing Graphical User Interface (GUI) automation. However, existing
GUI models mostly rely on learning from nearly error-free offline trajectories,
thus lacking reflection and error recovery capabilities. To bridge this gap, we
propose GUI-Reflection, a novel framework that explicitly integrates
self-reflection and error correction capabilities into end-to-end multimodal
GUI models throughout dedicated training stages: GUI-specific pre-training,
offline supervised fine-tuning (SFT), and online reflection tuning.
GUI-reflection enables self-reflection behavior emergence with fully automated
data generation and learning processes without requiring any human annotation.
Specifically, 1) we first propose scalable data pipelines to automatically
construct reflection and error correction data from existing successful
trajectories. While existing GUI models mainly focus on grounding and UI
understanding ability, we propose the GUI-Reflection Task Suite to learn and
evaluate reflection-oriented abilities explicitly. 2) Furthermore, we built a
diverse and efficient environment for online training and data collection of
GUI models on mobile devices. 3) We also present an iterative online reflection
tuning algorithm leveraging the proposed environment, enabling the model to
continuously enhance its reflection and error correction abilities. Our
framework equips GUI agents with self-reflection and correction capabilities,
paving the way for more robust, adaptable, and intelligent GUI automation, with
all data, models, environments, and tools to be released publicly.
Through the Valley: Path to Effective Long CoT Training for Small
  Language Models
Authors: name: Renjie Luo, name: Jiaxi Li, name: Chen Huang, name: Wei Lu
Published: 2025-06-09 12:56:41

Long chain-of-thought (CoT) supervision has become a common strategy to
enhance reasoning in language models. While effective for large models, we
identify a phenomenon we call Long CoT Degradation, in which small language
models (SLMs; <=3B parameters) trained on limited long CoT data experience
significant performance deterioration. Through extensive experiments on the
Qwen2.5, LLaMA3 and Gemma3 families, we demonstrate that this degradation is
widespread across SLMs. In some settings, models trained on only 8k long CoT
examples lose up to 75% of their original performance before fine-tuning.
Strikingly, we further observe that for some particularly small models, even
training on 220k long CoT examples fails to recover or surpass their original
performance prior to fine-tuning. Our analysis attributes this effect to error
accumulation: while longer responses increase the capacity for multi-step
reasoning, they also amplify the risk of compounding mistakes. Furthermore, we
find that Long CoT Degradation may negatively impacts downstream reinforcement
learning (RL), although this can be alleviated by sufficiently scaled
supervised fine-tuning (SFT). Our findings challenge common assumptions about
the benefits of long CoT training for SLMs and offer practical guidance for
building more effective small-scale reasoning models.
BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulation
Authors: name: Hongyu Wang, name: Chuyan Xiong, name: Ruiping Wang, name: Xilin Chen
Published: 2025-06-09 08:15:11

Vision-Language-Action (VLA) models have shown impressive capabilities across
a wide range of robotics manipulation tasks. However, their growing model size
poses significant challenges for deployment on resource-constrained robotic
systems. While 1-bit pretraining has proven effective for enhancing the
inference efficiency of large language models with minimal performance loss,
its application to VLA models remains underexplored. In this work, we present
BitVLA, the first 1-bit VLA model for robotics manipulation, in which every
parameter is ternary, i.e., {-1, 0, 1}. To further reduce the memory footprint
of the vision encoder, we propose the distillation-aware training strategy that
compresses the full-precision encoder to 1.58-bit weights. During this process,
a full-precision encoder serves as a teacher model to better align latent
representations. Despite the lack of large-scale robotics pretraining, BitVLA
achieves performance comparable to the state-of-the-art model OpenVLA-OFT with
4-bit post-training quantization on the LIBERO benchmark, while consuming only
29.8% of the memory. These results highlight BitVLA's promise for deployment on
memory-constrained edge devices. We release the code and model weights in
https://github.com/ustcwhy/BitVLA.
Medical World Model: Generative Simulation of Tumor Evolution for
  Treatment Planning
Authors: name: Yijun Yang, name: Zhao-Yang Wang, name: Qiuping Liu, name: Shuwen Sun, name: Kang Wang, name: Rama Chellappa, name: Zongwei Zhou, name: Alan Yuille, name: Lei Zhu, name: Yu-Dong Zhang, name: Jieneng Chen
Published: 2025-06-02 23:50:40

Providing effective treatment and making informed clinical decisions are
essential goals of modern medicine and clinical care. We are interested in
simulating disease dynamics for clinical decision-making, leveraging recent
advances in large generative models. To this end, we introduce the Medical
World Model (MeWM), the first world model in medicine that visually predicts
future disease states based on clinical decisions. MeWM comprises (i)
vision-language models to serve as policy models, and (ii) tumor generative
models as dynamics models. The policy model generates action plans, such as
clinical treatments, while the dynamics model simulates tumor progression or
regression under given treatment conditions. Building on this, we propose the
inverse dynamics model that applies survival analysis to the simulated
post-treatment tumor, enabling the evaluation of treatment efficacy and the
selection of the optimal clinical action plan. As a result, the proposed MeWM
simulates disease dynamics by synthesizing post-treatment tumors, with
state-of-the-art specificity in Turing tests evaluated by radiologists.
Simultaneously, its inverse dynamics model outperforms medical-specialized GPTs
in optimizing individualized treatment protocols across all metrics. Notably,
MeWM improves clinical decision-making for interventional physicians, boosting
F1-score in selecting the optimal TACE protocol by 13%, paving the way for
future integration of medical world models as the second readers.
When Semantics Mislead Vision: Mitigating Large Multimodal Models
  Hallucinations in Scene Text Spotting and Understanding
Authors: name: Yan Shu, name: Hangui Lin, name: Yexin Liu, name: Yan Zhang, name: Gangyan Zeng, name: Yan Li, name: Yu Zhou, name: Ser-Nam Lim, name: Harry Yang, name: Nicu Sebe
Published: 2025-06-05 19:53:19

Large Multimodal Models (LMMs) have achieved impressive progress in visual
perception and reasoning. However, when confronted with visually ambiguous or
non-semantic scene text, they often struggle to accurately spot and understand
the content, frequently generating semantically plausible yet visually
incorrect answers, which we refer to as semantic hallucination. In this work,
we investigate the underlying causes of semantic hallucination and identify a
key finding: Transformer layers in LLM with stronger attention focus on scene
text regions are less prone to producing semantic hallucinations. Thus, we
propose a training-free semantic hallucination mitigation framework comprising
two key components: (1) ZoomText, a coarse-to-fine strategy that identifies
potential text regions without external detectors; and (2) Grounded Layer
Correction, which adaptively leverages the internal representations from layers
less prone to hallucination to guide decoding, correcting hallucinated outputs
for non-semantic samples while preserving the semantics of meaningful ones. To
enable rigorous evaluation, we introduce TextHalu-Bench, a benchmark of over
1,730 samples spanning both semantic and non-semantic cases, with manually
curated question-answer pairs designed to probe model hallucinations. Extensive
experiments demonstrate that our method not only effectively mitigates semantic
hallucination but also achieves strong performance on public benchmarks for
scene text spotting and understanding.
Sparsified State-Space Models are Efficient Highway Networks
Authors: name: Woomin Song, name: Jihoon Tack, name: Sangwoo Mo, name: Seunghyuk Oh, name: Jinwoo Shin
Published: 2025-05-27 04:07:23

State-space models (SSMs) offer a promising architecture for sequence
modeling, providing an alternative to Transformers by replacing expensive
self-attention with linear recurrences. In this paper, we propose a simple yet
effective trick to enhance SSMs within given computational budgets by
sparsifying them. Our intuition is that tokens in SSMs are highly redundant due
to gradual recurrent updates, and dense recurrence operations block the
delivery of past information. In particular, we observe that upper layers of
SSMs tend to be more redundant as they encode global information, while lower
layers encode local information. Motivated by this, we introduce Simba, a
hierarchical sparsification method for SSMs based on token pruning. Simba
sparsifies upper layers more than lower layers, encouraging the upper layers to
behave like highways. To achieve this, we propose a novel token pruning
criterion for SSMs, measuring the global impact of tokens on the final output
by accumulating local recurrences. We demonstrate that Simba outperforms the
baseline model, Mamba, with the same FLOPS in various natural language tasks.
Moreover, we illustrate the effect of highways, showing that Simba not only
enhances efficiency but also improves the information flow across long
sequences. Code is available at https://github.com/woominsong/Simba.
MIRIAD: Augmenting LLMs with millions of medical query-response pairs
Authors: name: Qinyue Zheng, name: Salman Abdullah, name: Sam Rawal, name: Cyril Zakka, name: Sophie Ostmeier, name: Maximilian Purk, name: Eduardo Reis, name: Eric J. Topol, name: Jure Leskovec, name: Michael Moor
Published: 2025-06-06 13:52:32

LLMs are bound to transform healthcare with advanced decision support and
flexible chat assistants. However, LLMs are prone to generate inaccurate
medical content. To ground LLMs in high-quality medical knowledge, LLMs have
been equipped with external knowledge via RAG, where unstructured medical
knowledge is split into small text chunks that can be selectively retrieved and
integrated into the LLMs context. Yet, existing RAG pipelines rely on raw,
unstructured medical text, which can be noisy, uncurated and difficult for LLMs
to effectively leverage. Systematic approaches to organize medical knowledge to
best surface it to LLMs are generally lacking. To address these challenges, we
introduce MIRIAD, a large-scale, curated corpus of 5,821,948 medical QA pairs,
each rephrased from and grounded in a passage from peer-reviewed medical
literature using a semi-automated pipeline combining LLM generation, filtering,
grounding, and human annotation. Unlike prior medical corpora, which rely on
unstructured text, MIRIAD encapsulates web-scale medical knowledge in an
operationalized query-response format, which enables more targeted retrieval.
Experiments on challenging medical QA benchmarks show that augmenting LLMs with
MIRIAD improves accuracy up to 6.7% compared to unstructured RAG baselines with
the same source corpus and with the same amount of retrieved text. Moreover,
MIRIAD improved the ability of LLMs to detect medical hallucinations by 22.5 to
37% (increase in F1 score). We further introduce MIRIAD-Atlas, an interactive
map of MIRIAD spanning 56 medical disciplines, enabling clinical users to
visually explore, search, and refine medical knowledge. MIRIAD promises to
unlock a wealth of down-stream applications, including medical information
retrievers, enhanced RAG applications, and knowledge-grounded chat interfaces,
which ultimately enables more reliable LLM applications in healthcare.
AssetOpsBench: Benchmarking AI Agents for Task Automation in Industrial
  Asset Operations and Maintenance
Authors: name: Dhaval Patel, name: Shuxin Lin, name: James Rayfield, name: Nianjun Zhou, name: Roman Vaculin, name: Natalia Martinez, name: Fearghal O'donncha, name: Jayant Kalagnanam
Published: 2025-06-04 10:57:35

AI for Industrial Asset Lifecycle Management aims to automate complex
operational workflows -- such as condition monitoring, maintenance planning,
and intervention scheduling -- to reduce human workload and minimize system
downtime. Traditional AI/ML approaches have primarily tackled these problems in
isolation, solving narrow tasks within the broader operational pipeline. In
contrast, the emergence of AI agents and large language models (LLMs)
introduces a next-generation opportunity: enabling end-to-end automation across
the entire asset lifecycle. This paper envisions a future where AI agents
autonomously manage tasks that previously required distinct expertise and
manual coordination. To this end, we introduce AssetOpsBench -- a unified
framework and environment designed to guide the development, orchestration, and
evaluation of domain-specific agents tailored for Industry 4.0 applications. We
outline the key requirements for such holistic systems and provide actionable
insights into building agents that integrate perception, reasoning, and control
for real-world industrial operations. The software is available at
https://github.com/IBM/AssetOpsBench.
Peer-Ranked Precision: Creating a Foundational Dataset for Fine-Tuning
  Vision Models from DataSeeds' Annotated Imagery
Authors: name: Sajjad Abdoli, name: Freeman Lewin, name: Gediminas Vasiliauskas, name: Fabian Schonholz
Published: 2025-06-06 01:50:28

The development of modern Artificial Intelligence (AI) models, particularly
diffusion-based models employed in computer vision and image generation tasks,
is undergoing a paradigmatic shift in development methodologies. Traditionally
dominated by a "Model Centric" approach, in which performance gains were
primarily pursued through increasingly complex model architectures and
hyperparameter optimization, the field is now recognizing a more nuanced
"Data-Centric" approach. This emergent framework foregrounds the quality,
structure, and relevance of training data as the principal driver of model
performance. To operationalize this paradigm shift, we introduce the
DataSeeds.AI sample dataset (the "DSD"), initially comprised of approximately
10,610 high-quality human peer-ranked photography images accompanied by
extensive multi-tier annotations. The DSD is a foundational computer vision
dataset designed to usher in a new standard for commercial image datasets.
Representing a small fraction of DataSeed.AI's 100 million-plus image catalog,
the DSD provides a scalable foundation necessary for robust commercial and
multimodal AI development. Through this in-depth exploratory analysis, we
document the quantitative improvements generated by the DSD on specific models
against known benchmarks and make the code and the trained models used in our
evaluation publicly available.
When Models Know More Than They Can Explain: Quantifying Knowledge
  Transfer in Human-AI Collaboration
Authors: name: Quan Shi, name: Carlos E. Jimenez, name: Shunyu Yao, name: Nick Haber, name: Diyi Yang, name: Karthik Narasimhan
Published: 2025-06-05 20:48:16

Recent advancements in AI reasoning have driven substantial improvements
across diverse tasks. A critical open question is whether these improvements
also yields better knowledge transfer: the ability of models to communicate
reasoning in ways humans can understand, apply, and learn from. To investigate
this, we introduce Knowledge Integration and Transfer Evaluation (KITE), a
conceptual and experimental framework for Human-AI knowledge transfer
capabilities and conduct the first large-scale human study (N=118) explicitly
designed to measure it. In our two-phase setup, humans first ideate with an AI
on problem-solving strategies, then independently implement solutions,
isolating model explanations' influence on human understanding. Our findings
reveal that although model benchmark performance correlates with collaborative
outcomes, this relationship is notably inconsistent, featuring significant
outliers, indicating that knowledge transfer requires dedicated optimization.
Our analysis identifies behavioral and strategic factors mediating successful
knowledge transfer. We release our code, dataset, and evaluation framework to
support future work on communicatively aligned models.
Sentinel: SOTA model to protect against prompt injections
Authors: name: Dror Ivry, name: Oran Nahum
Published: 2025-06-05 14:07:15

Large Language Models (LLMs) are increasingly powerful but remain vulnerable
to prompt injection attacks, where malicious inputs cause the model to deviate
from its intended instructions. This paper introduces Sentinel, a novel
detection model, qualifire/prompt-injection-sentinel, based on the
\answerdotai/ModernBERT-large architecture. By leveraging ModernBERT's advanced
features and fine-tuning on an extensive and diverse dataset comprising a few
open-source and private collections, Sentinel achieves state-of-the-art
performance. This dataset amalgamates varied attack types, from role-playing
and instruction hijacking to attempts to generate biased content, alongside a
broad spectrum of benign instructions, with private datasets specifically
targeting nuanced error correction and real-world misclassifications. On a
comprehensive, unseen internal test set, Sentinel demonstrates an average
accuracy of 0.987 and an F1-score of 0.980. Furthermore, when evaluated on
public benchmarks, it consistently outperforms strong baselines like
protectai/deberta-v3-base-prompt-injection-v2. This work details Sentinel's
architecture, its meticulous dataset curation, its training methodology, and a
thorough evaluation, highlighting its superior detection capabilities.
CodeContests+: High-Quality Test Case Generation for Competitive
  Programming
Authors: name: Zihan Wang, name: Siyao Liu, name: Yang Sun, name: Hongyan Li, name: Kai Shen
Published: 2025-06-06 07:29:01

Competitive programming, due to its high reasoning difficulty and precise
correctness feedback, has become a key task for both training and evaluating
the reasoning capabilities of large language models (LLMs). However, while a
large amount of public problem data, such as problem statements and solutions,
is available, the test cases of these problems are often difficult to obtain.
Therefore, test case generation is a necessary task for building large-scale
datasets, and the quality of the test cases directly determines the accuracy of
the evaluation. In this paper, we introduce an LLM-based agent system that
creates high-quality test cases for competitive programming problems. We apply
this system to the CodeContests dataset and propose a new version with improved
test cases, named CodeContests+. We evaluated the quality of test cases in
CodeContestsPlus. First, we used 1.72 million submissions with pass/fail labels
to examine the accuracy of these test cases in evaluation. The results
indicated that CodeContests+ achieves significantly higher accuracy than
CodeContests, particularly with a notably higher True Positive Rate (TPR).
Subsequently, our experiments in LLM Reinforcement Learning (RL) further
confirmed that improvements in test case quality yield considerable advantages
for RL.
Splatting Physical Scenes: End-to-End Real-to-Sim from Imperfect Robot
  Data
Authors: name: Ben Moran, name: Mauro Comi, name: Steven Bohez, name: Tom Erez, name: Zhibin Li, name: Leonard Hasenclever
Published: 2025-06-04 16:14:31

Creating accurate, physical simulations directly from real-world robot motion
holds great value for safe, scalable, and affordable robot learning, yet
remains exceptionally challenging. Real robot data suffers from occlusions,
noisy camera poses, dynamic scene elements, which hinder the creation of
geometrically accurate and photorealistic digital twins of unseen objects. We
introduce a novel real-to-sim framework tackling all these challenges at once.
Our key insight is a hybrid scene representation merging the photorealistic
rendering of 3D Gaussian Splatting with explicit object meshes suitable for
physics simulation within a single representation. We propose an end-to-end
optimization pipeline that leverages differentiable rendering and
differentiable physics within MuJoCo to jointly refine all scene components -
from object geometry and appearance to robot poses and physical parameters -
directly from raw and imprecise robot trajectories. This unified optimization
allows us to simultaneously achieve high-fidelity object mesh reconstruction,
generate photorealistic novel views, and perform annotation-free robot pose
calibration. We demonstrate the effectiveness of our approach both in
simulation and on challenging real-world sequences using an ALOHA 2 bi-manual
manipulator, enabling more practical and robust real-to-simulation pipelines.
Truth in the Few: High-Value Data Selection for Efficient Multi-Modal
  Reasoning
Authors: name: Shenshen Li, name: Kaiyuan Deng, name: Lei Wang, name: Hao Yang, name: Chong Peng, name: Peng Yan, name: Fumin Shen, name: Heng Tao Shen, name: Xing Xu
Published: 2025-06-05 08:40:24

While multi-modal large language models (MLLMs) have made significant
progress in complex reasoning tasks via reinforcement learning, it is commonly
believed that extensive training data is necessary for improving multi-modal
reasoning ability, inevitably leading to data redundancy and substantial
computational costs. However, can smaller high-value datasets match or
outperform full corpora for multi-modal reasoning in MLLMs? In this work, we
challenge this assumption through a key observation: meaningful multi-modal
reasoning is triggered by only a sparse subset of training samples, termed
cognitive samples, whereas the majority contribute marginally. Building on this
insight, we propose a novel data selection paradigm termed Reasoning Activation
Potential (RAP), which identifies cognitive samples by estimating each sample's
potential to stimulate genuine multi-modal reasoning by two complementary
estimators: 1) Causal Discrepancy Estimator (CDE) based on the potential
outcome model principle, eliminates samples that overly rely on language priors
by comparing outputs between multi-modal and text-only inputs; 2) Attention
Confidence Estimator (ACE), which exploits token-level self-attention to
discard samples dominated by irrelevant but over-emphasized tokens in
intermediate reasoning stages. Moreover, we introduce a Difficulty-aware
Replacement Module (DRM) to substitute trivial instances with cognitively
challenging ones, thereby ensuring complexity for robust multi-modal reasoning.
Experiments on six datasets show that our RAP method consistently achieves
superior performance using only 9.3% of the training data, while reducing
computational costs by over 43%. Our code is available at
https://github.com/Leo-ssl/RAP.