Orionnnn/hub-stats huggingface dataset:
Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for
  Efficient and Enhanced Video Reasoning
Authors: name: Ziyang Wang, name: Jaehong Yoon, name: Shoubin Yu, name: Md Mohaiminul Islam, name: Gedas Bertasius, name: Mohit Bansal
Published: 2025-07-09 02:06:13

Despite advances in reinforcement learning (RL)-based video reasoning with
large language models (LLMs), data collection and finetuning remain significant
challenges. These methods often rely on large-scale supervised fine-tuning
(SFT) with extensive video data and long Chain-of-Thought (CoT) annotations,
making them costly and hard to scale. To address this, we present Video-RTS, a
new approach to improve video reasoning capability with drastically improved
data efficiency by combining data-efficient RL with a video-adaptive test-time
scaling (TTS) strategy. Based on observations about the data scaling of RL
samples, we skip the resource-intensive SFT step and employ efficient pure-RL
training with output-based rewards, requiring no additional annotations or
extensive fine-tuning. Furthermore, to utilize computational resources more
efficiently, we introduce a sparse-to-dense video TTS strategy that improves
inference by iteratively adding frames based on output consistency. We validate
our approach on multiple video reasoning benchmarks, showing that Video-RTS
surpasses existing video reasoning models by an average of 2.4% in accuracy
using only 3.6% training samples. For example, Video-RTS achieves a 4.2%
improvement on Video-Holmes, a recent and challenging video reasoning
benchmark, and a 2.6% improvement on MMVU. Notably, our pure RL training and
adaptive video TTS offer complementary strengths, enabling Video-RTS's strong
reasoning performance.
Towards Multimodal Understanding via Stable Diffusion as a Task-Aware
  Feature Extractor
Authors: name: Vatsal Agarwal, name: Matthew Gwilliam, name: Gefen Kohavi, name: Eshan Verma, name: Daniel Ulbricht, name: Abhinav Shrivastava
Published: 2025-07-09 17:59:47

Recent advances in multimodal large language models (MLLMs) have enabled
image-based question-answering capabilities. However, a key limitation is the
use of CLIP as the visual encoder; while it can capture coarse global
information, it often can miss fine-grained details that are relevant to the
input query. To address these shortcomings, this work studies whether
pre-trained text-to-image diffusion models can serve as instruction-aware
visual encoders. Through an analysis of their internal representations, we find
diffusion features are both rich in semantics and can encode strong image-text
alignment. Moreover, we find that we can leverage text conditioning to focus
the model on regions relevant to the input question. We then investigate how to
align these features with large language models and uncover a leakage
phenomenon, where the LLM can inadvertently recover information from the
original diffusion prompt. We analyze the causes of this leakage and propose a
mitigation strategy. Based on these insights, we explore a simple fusion
strategy that utilizes both CLIP and conditional diffusion features. We
evaluate our approach on both general VQA and specialized MLLM benchmarks,
demonstrating the promise of diffusion models for visual understanding,
particularly in vision-centric tasks that require spatial and compositional
reasoning. Our project page can be found
https://vatsalag99.github.io/mustafar/.
SRT-H: A Hierarchical Framework for Autonomous Surgery via Language
  Conditioned Imitation Learning
Authors: name: Ji Woong Kim, name: Juo-Tung Chen, name: Pascal Hansen, name: Lucy X. Shi, name: Antony Goldenberg, name: Samuel Schmidgall, name: Paul Maria Scheikl, name: Anton Deguet, name: Brandon M. White, name: De Ru Tsai, name: Richard Cha, name: Jeffrey Jopling, name: Chelsea Finn, name: Axel Krieger
Published: 2025-05-15 13:04:53

Research on autonomous surgery has largely focused on simple task automation
in controlled environments. However, real-world surgical applications demand
dexterous manipulation over extended durations and generalization to the
inherent variability of human tissue. These challenges remain difficult to
address using existing logic-based or conventional end-to-end learning
approaches. To address this gap, we propose a hierarchical framework for
performing dexterous, long-horizon surgical steps. Our approach utilizes a
high-level policy for task planning and a low-level policy for generating robot
trajectories. The high-level planner plans in language space, generating
task-level or corrective instructions that guide the robot through the
long-horizon steps and correct for the low-level policy's errors. We validate
our framework through ex vivo experiments on cholecystectomy, a
commonly-practiced minimally invasive procedure, and conduct ablation studies
to evaluate key components of the system. Our method achieves a 100\% success
rate across eight unseen ex vivo gallbladders, operating fully autonomously
without human intervention. This work demonstrates step-level autonomy in a
surgical procedure, marking a milestone toward clinical deployment of
autonomous surgical systems.
AdamMeme: Adaptively Probe the Reasoning Capacity of Multimodal Large
  Language Models on Harmfulness
Authors: name: Zixin Chen, name: Hongzhan Lin, name: Kaixin Li, name: Ziyang Luo, name: Zhen Ye, name: Guang Chen, name: Zhiyong Huang, name: Jing Ma
Published: 2025-07-02 13:32:30

The proliferation of multimodal memes in the social media era demands that
multimodal Large Language Models (mLLMs) effectively understand meme
harmfulness. Existing benchmarks for assessing mLLMs on harmful meme
understanding rely on accuracy-based, model-agnostic evaluations using static
datasets. These benchmarks are limited in their ability to provide up-to-date
and thorough assessments, as online memes evolve dynamically. To address this,
we propose AdamMeme, a flexible, agent-based evaluation framework that
adaptively probes the reasoning capabilities of mLLMs in deciphering meme
harmfulness. Through multi-agent collaboration, AdamMeme provides comprehensive
evaluations by iteratively updating the meme data with challenging samples,
thereby exposing specific limitations in how mLLMs interpret harmfulness.
Extensive experiments show that our framework systematically reveals the
varying performance of different target mLLMs, offering in-depth, fine-grained
analyses of model-specific weaknesses. Our code is available at
https://github.com/Lbotirx/AdamMeme.
ModelCitizens: Representing Community Voices in Online Safety
Authors: name: Ashima Suvarna, name: Christina Chance, name: Karolina Naranjo, name: Hamid Palangi, name: Sophie Hao, name: Thomas Hartvigsen, name: Saadia Gabriel
Published: 2025-07-07 20:15:18

Automatic toxic language detection is critical for creating safe, inclusive
online spaces. However, it is a highly subjective task, with perceptions of
toxic language shaped by community norms and lived experience. Existing
toxicity detection models are typically trained on annotations that collapse
diverse annotator perspectives into a single ground truth, erasing important
context-specific notions of toxicity such as reclaimed language. To address
this, we introduce MODELCITIZENS, a dataset of 6.8K social media posts and 40K
toxicity annotations across diverse identity groups. To capture the role of
conversational context on toxicity, typical of social media posts, we augment
MODELCITIZENS posts with LLM-generated conversational scenarios.
State-of-the-art toxicity detection tools (e.g. OpenAI Moderation API,
GPT-o4-mini) underperform on MODELCITIZENS, with further degradation on
context-augmented posts. Finally, we release LLAMACITIZEN-8B and
GEMMACITIZEN-12B, LLaMA- and Gemma-based models finetuned on MODELCITIZENS,
which outperform GPT-o4-mini by 5.5% on in-distribution evaluations. Our
findings highlight the importance of community-informed annotation and modeling
for inclusive content moderation. The data, models and code are available at
https://github.com/asuvarna31/modelcitizens.
4KAgent: Agentic Any Image to 4K Super-Resolution
Authors: name: Yushen Zuo, name: Qi Zheng, name: Mingyang Wu, name: Xinrui Jiang, name: Renjie Li, name: Jian Wang, name: Yide Zhang, name: Gengchen Mai, name: Lihong V. Wang, name: James Zou, name: Xiaoyu Wang, name: Ming-Hsuan Yang, name: Zhengzhong Tu
Published: 2025-07-09 17:59:19

We present 4KAgent, a unified agentic super-resolution generalist system
designed to universally upscale any image to 4K resolution (and even higher, if
applied iteratively). Our system can transform images from extremely low
resolutions with severe degradations, for example, highly distorted inputs at
256x256, into crystal-clear, photorealistic 4K outputs. 4KAgent comprises three
core components: (1) Profiling, a module that customizes the 4KAgent pipeline
based on bespoke use cases; (2) A Perception Agent, which leverages
vision-language models alongside image quality assessment experts to analyze
the input image and make a tailored restoration plan; and (3) A Restoration
Agent, which executes the plan, following a recursive execution-reflection
paradigm, guided by a quality-driven mixture-of-expert policy to select the
optimal output for each step. Additionally, 4KAgent embeds a specialized face
restoration pipeline, significantly enhancing facial details in portrait and
selfie photos. We rigorously evaluate our 4KAgent across 11 distinct task
categories encompassing a total of 26 diverse benchmarks, setting new
state-of-the-art on a broad spectrum of imaging domains. Our evaluations cover
natural images, portrait photos, AI-generated content, satellite imagery,
fluorescence microscopy, and medical imaging like fundoscopy, ultrasound, and
X-ray, demonstrating superior performance in terms of both perceptual (e.g.,
NIQE, MUSIQ) and fidelity (e.g., PSNR) metrics. By establishing a novel agentic
paradigm for low-level vision tasks, we aim to catalyze broader interest and
innovation within vision-centric autonomous agents across diverse research
communities. We will release all the code, models, and results at:
https://4kagent.github.io.
DiffSpectra: Molecular Structure Elucidation from Spectra using
  Diffusion Models
Authors: name: Liang Wang, name: Yu Rong, name: Tingyang Xu, name: Zhenyi Zhong, name: Zhiyuan Liu, name: Pengju Wang, name: Deli Zhao, name: Qiang Liu, name: Shu Wu, name: Liang Wang
Published: 2025-07-09 13:57:20

Molecular structure elucidation from spectra is a foundational problem in
chemistry, with profound implications for compound identification, synthesis,
and drug development. Traditional methods rely heavily on expert interpretation
and lack scalability. Pioneering machine learning methods have introduced
retrieval-based strategies, but their reliance on finite libraries limits
generalization to novel molecules. Generative models offer a promising
alternative, yet most adopt autoregressive SMILES-based architectures that
overlook 3D geometry and struggle to integrate diverse spectral modalities. In
this work, we present DiffSpectra, a generative framework that directly infers
both 2D and 3D molecular structures from multi-modal spectral data using
diffusion models. DiffSpectra formulates structure elucidation as a conditional
generation process. Its denoising network is parameterized by Diffusion
Molecule Transformer, an SE(3)-equivariant architecture that integrates
topological and geometric information. Conditioning is provided by SpecFormer,
a transformer-based spectral encoder that captures intra- and inter-spectral
dependencies from multi-modal spectra. Extensive experiments demonstrate that
DiffSpectra achieves high accuracy in structure elucidation, recovering exact
structures with 16.01% top-1 accuracy and 96.86% top-20 accuracy through
sampling. The model benefits significantly from 3D geometric modeling,
SpecFormer pre-training, and multi-modal conditioning. These results highlight
the effectiveness of spectrum-conditioned diffusion modeling in addressing the
challenge of molecular structure elucidation. To our knowledge, DiffSpectra is
the first framework to unify multi-modal spectral reasoning and joint 2D/3D
generative modeling for de novo molecular structure elucidation.
First Return, Entropy-Eliciting Explore
Authors: name: Tianyu Zheng, name: Tianshun Xing, name: Qingshui Gu, name: Taoran Liang, name: Xingwei Qu, name: Xin Zhou, name: Yizhi Li, name: Zhoufutu Wen, name: Chenghua Lin, name: Wenhao Huang, name: Qian Liu, name: Ge Zhang, name: Zejun Ma
Published: 2025-07-09 16:45:48

Reinforcement Learning from Verifiable Rewards (RLVR) improves the reasoning
abilities of Large Language Models (LLMs) but it struggles with unstable
exploration. We propose FR3E (First Return, Entropy-Eliciting Explore), a
structured exploration framework that identifies high-uncertainty decision
points in reasoning trajectories and performs targeted rollouts to construct
semantically grounded intermediate feedback. Our method provides targeted
guidance without relying on dense supervision. Empirical results on
mathematical reasoning benchmarks(AIME24) show that FR3E promotes more stable
training, produces longer and more coherent responses, and increases the
proportion of fully correct trajectories. These results highlight the
framework's effectiveness in improving LLM reasoning through more robust and
structured exploration.
A Systematic Analysis of Hybrid Linear Attention
Authors: name: Dustin Wang, name: Rui-Jie Zhu, name: Steven Abreu, name: Yong Shan, name: Taylor Kergan, name: Yuqi Pan, name: Yuhong Chou, name: Zheng Li, name: Ge Zhang, name: Wenhao Huang, name: Jason Eshraghian
Published: 2025-07-08 23:54:11

Transformers face quadratic complexity and memory issues with long sequences,
prompting the adoption of linear attention mechanisms using fixed-size hidden
states. However, linear models often suffer from limited recall performance,
leading to hybrid architectures that combine linear and full attention layers.
Despite extensive hybrid architecture research, the choice of linear attention
component has not been deeply explored. We systematically evaluate various
linear attention models across generations - vector recurrences to advanced
gating mechanisms - both standalone and hybridized. To enable this
comprehensive analysis, we trained and open-sourced 72 models: 36 at 340M
parameters (20B tokens) and 36 at 1.3B parameters (100B tokens), covering six
linear attention variants across five hybridization ratios. Benchmarking on
standard language modeling and recall tasks reveals that superior standalone
linear models do not necessarily excel in hybrids. While language modeling
remains stable across linear-to-full attention ratios, recall significantly
improves with increased full attention layers, particularly below a 3:1 ratio.
Our study highlights selective gating, hierarchical recurrence, and controlled
forgetting as critical for effective hybrid models. We recommend architectures
such as HGRN-2 or GatedDeltaNet with a linear-to-full ratio between 3:1 and 6:1
to achieve Transformer-level recall efficiently. Our models are open-sourced at
https://huggingface.co/collections/m-a-p/hybrid-linear-attention-research-686c488a63d609d2f20e2b1e.
Perception-Aware Policy Optimization for Multimodal Reasoning
Authors: name: Zhenhailong Wang, name: Xuehang Guo, name: Sofia Stoica, name: Haiyang Xu, name: Hongru Wang, name: Hyeonjeong Ha, name: Xiusi Chen, name: Yangyi Chen, name: Ming Yan, name: Fei Huang, name: Heng Ji
Published: 2025-07-08 23:22:34

Reinforcement Learning with Verifiable Rewards (RLVR) has proven to be a
highly effective strategy for endowing Large Language Models (LLMs) with robust
multi-step reasoning abilities. However, its design and optimizations remain
tailored to purely textual domains, resulting in suboptimal performance when
applied to multimodal reasoning tasks. In particular, we observe that a major
source of error in current multimodal reasoning lies in the perception of
visual inputs. To address this bottleneck, we propose Perception-Aware Policy
Optimization (PAPO), a simple yet effective extension of GRPO that encourages
the model to learn to perceive while learning to reason, entirely from internal
supervision signals. Notably, PAPO does not rely on additional data curation,
external reward models, or proprietary models. Specifically, we introduce the
Implicit Perception Loss in the form of a KL divergence term to the GRPO
objective, which, despite its simplicity, yields significant overall
improvements (4.4%) on diverse multimodal benchmarks. The improvements are more
pronounced, approaching 8.0%, on tasks with high vision dependency. We also
observe a substantial reduction (30.5%) in perception errors, indicating
improved perceptual capabilities with PAPO. We conduct comprehensive analysis
of PAPO and identify a unique loss hacking issue, which we rigorously analyze
and mitigate through a Double Entropy Loss. Overall, our work introduces a
deeper integration of perception-aware supervision into RLVR learning
objectives and lays the groundwork for a new RL framework that encourages
visually grounded reasoning. Project page: https://mikewangwzhl.github.io/PAPO.
Go to Zero: Towards Zero-shot Motion Generation with Million-scale Data
Authors: name: Ke Fan, name: Shunlin Lu, name: Minyue Dai, name: Runyi Yu, name: Lixing Xiao, name: Zhiyang Dou, name: Junting Dong, name: Lizhuang Ma, name: Jingbo Wang
Published: 2025-07-09 17:52:04

Generating diverse and natural human motion sequences based on textual
descriptions constitutes a fundamental and challenging research area within the
domains of computer vision, graphics, and robotics. Despite significant
advancements in this field, current methodologies often face challenges
regarding zero-shot generalization capabilities, largely attributable to the
limited size of training datasets. Moreover, the lack of a comprehensive
evaluation framework impedes the advancement of this task by failing to
identify directions for improvement. In this work, we aim to push
text-to-motion into a new era, that is, to achieve the generalization ability
of zero-shot. To this end, firstly, we develop an efficient annotation pipeline
and introduce MotionMillion-the largest human motion dataset to date, featuring
over 2,000 hours and 2 million high-quality motion sequences. Additionally, we
propose MotionMillion-Eval, the most comprehensive benchmark for evaluating
zero-shot motion generation. Leveraging a scalable architecture, we scale our
model to 7B parameters and validate its performance on MotionMillion-Eval. Our
results demonstrate strong generalization to out-of-domain and complex
compositional motions, marking a significant step toward zero-shot human motion
generation. The code is available at
https://github.com/VankouF/MotionMillion-Codes.
A Survey on Vision-Language-Action Models for Autonomous Driving
Authors: name: Sicong Jiang, name: Zilin Huang, name: Kangan Qian, name: Ziang Luo, name: Tianze Zhu, name: Yang Zhong, name: Yihong Tang, name: Menglin Kong, name: Yunlong Wang, name: Siwen Jiao, name: Hao Ye, name: Zihao Sheng, name: Xin Zhao, name: Tuopu Wen, name: Zheng Fu, name: Sikai Chen, name: Kun Jiang, name: Diange Yang, name: Seongjin Choi, name: Lijun Sun
Published: 2025-06-30 16:50:02

The rapid progress of multimodal large language models (MLLM) has paved the
way for Vision-Language-Action (VLA) paradigms, which integrate visual
perception, natural language understanding, and control within a single policy.
Researchers in autonomous driving are actively adapting these methods to the
vehicle domain. Such models promise autonomous vehicles that can interpret
high-level instructions, reason about complex traffic scenes, and make their
own decisions. However, the literature remains fragmented and is rapidly
expanding. This survey offers the first comprehensive overview of VLA for
Autonomous Driving (VLA4AD). We (i) formalize the architectural building blocks
shared across recent work, (ii) trace the evolution from early explainer to
reasoning-centric VLA models, and (iii) compare over 20 representative models
according to VLA's progress in the autonomous driving domain. We also
consolidate existing datasets and benchmarks, highlighting protocols that
jointly measure driving safety, accuracy, and explanation quality. Finally, we
detail open challenges - robustness, real-time efficiency, and formal
verification - and outline future directions of VLA4AD. This survey provides a
concise yet complete reference for advancing interpretable socially aligned
autonomous vehicles. Github repo is available at
https://github.com/JohnsonJiang1996/Awesome-VLA4AD{SicongJiang/Awesome-VLA4AD}.
AutoTriton: Automatic Triton Programming with Reinforcement Learning in
  LLMs
Authors: name: Shangzhan Li, name: Zefan Wang, name: Ye He, name: Yuxuan Li, name: Qi Shi, name: Jianling Li, name: Yonggang Hu, name: Wanxiang Che, name: Xu Han, name: Zhiyuan Liu, name: Maosong Sun
Published: 2025-07-08 05:38:24

Kernel development in deep learning requires optimizing computational units
across hardware while balancing memory management, parallelism, and
hardware-specific optimizations through extensive empirical tuning. Although
domain-specific languages like Triton simplify GPU programming by abstracting
low-level details, developers must still manually tune critical parameters such
as tile sizes and memory access patterns through iterative experimentation,
creating substantial barriers to optimal performance and wider adoption. In
this work, we introduce AutoTriton, the first model dedicated to Triton
programming powered by reinforcement learning (RL). AutoTriton performs
supervised fine-tuning (SFT) to be equipped with essential Triton programming
expertise using a high-quality data gathering pipeline, and conducts RL with
Group Relative Policy Optimization (GRPO) algorithm, combining a rule-based
reward and an execution-based reward to further improve Triton programming
ability, sequentially. Experiments across five evaluation channels of
TritonBench and KernelBench illustrate that our 8B model AutoTriton achieves
performance comparable to mainstream large models, including Claude-4-Sonnet
and DeepSeek-R1-0528. Further experimental analysis demonstrates the crucial
role of each module within AutoTriton, including the SFT stage, the RL stage,
and the reward design strategy. These findings underscore the promise of RL for
automatically generating high-performance kernels, and since high-performance
kernels are core components of AI systems, this breakthrough establishes an
important foundation for building more efficient AI systems. The model and code
will be available at https://github.com/AI9Stars/AutoTriton.