Orionnnn/hub-stats huggingface dataset:
BaseReward: A Strong Baseline for Multimodal Reward Model
Authors: name: Yi-Fan Zhang, name: Haihua Yang, name: Huanyu Zhang, name: Yang Shi, name: Zezhou Chen, name: Haochen Tian, name: Chaoyou Fu, name: Haotian Wang, name: Kai Wu, name: Bo Cui, name: Xu Wang, name: Jianfei Pan, name: Haotian Wang, name: Zhang Zhang, name: Liang Wang
Published: 2025-09-19 16:25:26

The rapid advancement of Multimodal Large Language Models (MLLMs) has made
aligning them with human preferences a critical challenge. Reward Models (RMs)
are a core technology for achieving this goal, but a systematic guide for
building state-of-the-art Multimodal Reward Models (MRMs) is currently lacking
in both academia and industry. Through exhaustive experimental analysis, this
paper aims to provide a clear ``recipe'' for constructing high-performance
MRMs. We systematically investigate every crucial component in the MRM
development pipeline, including reward modeling paradigms (e.g.,
Naive-RM, Critic-based RM, and Generative RM), reward head
architecture, training strategies, data curation (covering
over ten multimodal and text-only preference datasets), backbone model
and model scale, and ensemble methods.
  Based on these experimental insights, we introduce BaseReward, a
powerful and efficient baseline for multimodal reward modeling. BaseReward
adopts a simple yet effective architecture, built upon a {Qwen2.5-VL} backbone,
featuring an optimized two-layer reward head, and is trained on a carefully
curated mixture of high-quality multimodal and text-only preference data. Our
results show that BaseReward establishes a new SOTA on major benchmarks such as
MM-RLHF-Reward Bench, VL-Reward Bench, and Multimodal Reward Bench,
outperforming previous models. Furthermore, to validate its practical utility
beyond static benchmarks, we integrate BaseReward into a real-world
reinforcement learning pipeline, successfully enhancing an MLLM's performance
across various perception, reasoning, and conversational tasks. This work not
only delivers a top-tier MRM but, more importantly, provides the community with
a clear, empirically-backed guide for developing robust reward models for the
next generation of MLLMs.
RPG: A Repository Planning Graph for Unified and Scalable Codebase
  Generation
Authors: name: Jane Luo, name: Xin Zhang, name: Steven Liu, name: Jie Wu, name: Yiming Huang, name: Yangyu Huang, name: Chengyu Yin, name: Ying Xin, name: Jianfeng Liu, name: Yuefeng Zhan, name: Hao Sun, name: Qi Chen, name: Scarlett Li, name: Mao Yang
Published: 2025-09-19 17:58:14

Large language models excel at function- and file-level code generation, yet
generating complete repositories from scratch remains a fundamental challenge.
This process demands coherent and reliable planning across proposal- and
implementation-level stages, while natural language, due to its ambiguity and
verbosity, is ill-suited for faithfully representing complex software
structures. To address this, we introduce the Repository Planning Graph (RPG),
a persistent representation that unifies proposal- and implementation-level
planning by encoding capabilities, file structures, data flows, and functions
in one graph. RPG replaces ambiguous natural language with an explicit
blueprint, enabling long-horizon planning and scalable repository generation.
Building on RPG, we develop ZeroRepo, a graph-driven framework for repository
generation from scratch. It operates in three stages: proposal-level planning
and implementation-level refinement to construct the graph, followed by
graph-guided code generation with test validation. To evaluate this setting, we
construct RepoCraft, a benchmark of six real-world projects with 1,052 tasks.
On RepoCraft, ZeroRepo produces repositories averaging nearly 36K LOC, roughly
3.9times the strongest baseline (Claude Code) and about 64times other
baselines. It attains 81.5% functional coverage and a 69.7% pass rate,
exceeding Claude Code by 27.3 and 35.8 percentage points, respectively. Further
analysis shows that RPG models complex dependencies, enables progressively more
sophisticated planning through near-linear scaling, and enhances LLM
understanding of repositories, thereby accelerating agent localization.
WhisTLE: Deeply Supervised, Text-Only Domain Adaptation for Pretrained
  Speech Recognition Transformers
Authors: name: Akshat Pandey, name: Karun Kumar, name: Raphael Tang
Published: 2025-09-12 17:59:09

Pretrained automatic speech recognition (ASR) models such as Whisper perform
well but still need domain adaptation to handle unseen vocabulary and parlance.
In many real-world settings, collecting speech data is impractical,
necessitating text-only adaptation. We propose WhisTLE, a deeply supervised,
text-only adaptation method for pretrained encoder-decoder ASR models. WhisTLE
trains a variational autoencoder (VAE) to model encoder outputs from text and
fine-tunes the decoder using the learned text-to-latent encoder, optionally
combined with text-to-speech (TTS) adaptation. At inference, the original
encoder is restored, incurring no extra runtime cost. Across four out-of-domain
datasets and four ASR models, WhisTLE with TTS reduces word error rate (WER) by
12.3% relative to TTS-only adaptation and outperforms all non-WhisTLE baselines
in 27 of 32 scenarios.
Lynx: Towards High-Fidelity Personalized Video Generation
Authors: name: Shen Sang, name: Tiancheng Zhi, name: Tianpei Gu, name: Jing Liu, name: Linjie Luo
Published: 2025-09-19 00:31:57

We present Lynx, a high-fidelity model for personalized video synthesis from
a single input image. Built on an open-source Diffusion Transformer (DiT)
foundation model, Lynx introduces two lightweight adapters to ensure identity
fidelity. The ID-adapter employs a Perceiver Resampler to convert
ArcFace-derived facial embeddings into compact identity tokens for
conditioning, while the Ref-adapter integrates dense VAE features from a frozen
reference pathway, injecting fine-grained details across all transformer layers
through cross-attention. These modules collectively enable robust identity
preservation while maintaining temporal coherence and visual realism. Through
evaluation on a curated benchmark of 40 subjects and 20 unbiased prompts, which
yielded 800 test cases, Lynx has demonstrated superior face resemblance,
competitive prompt following, and strong video quality, thereby advancing the
state of personalized video generation.
A Vision-Language-Action-Critic Model for Robotic Real-World
  Reinforcement Learning
Authors: name: Shaopeng Zhai, name: Qi Zhang, name: Tianyi Zhang, name: Fuxian Huang, name: Haoran Zhang, name: Ming Zhou, name: Shengzhe Zhang, name: Litao Liu, name: Sixu Lin, name: Jiangmiao Pang
Published: 2025-09-19 12:44:29

Robotic real-world reinforcement learning (RL) with vision-language-action
(VLA) models is bottlenecked by sparse, handcrafted rewards and inefficient
exploration. We introduce VLAC, a general process reward model built upon
InternVL and trained on large scale heterogeneous datasets. Given pairwise
observations and a language goal, it outputs dense progress delta and done
signal, eliminating task-specific reward engineering, and supports one-shot
in-context transfer to unseen tasks and environments. VLAC is trained on
vision-language datasets to strengthen perception, dialogic and reasoning
capabilities, together with robot and human trajectories data that ground
action generation and progress estimation, and additionally strengthened to
reject irrelevant prompts as well as detect regression or stagnation by
constructing large numbers of negative and semantically mismatched samples.
With prompt control, a single VLAC model alternately generating reward and
action tokens, unifying critic and policy. Deployed inside an asynchronous
real-world RL loop, we layer a graded human-in-the-loop protocol (offline
demonstration replay, return and explore, human guided explore) that
accelerates exploration and stabilizes early learning. Across four distinct
real-world manipulation tasks, VLAC lifts success rates from about 30\% to
about 90\% within 200 real-world interaction episodes; incorporating
human-in-the-loop interventions yields a further 50% improvement in sample
efficiency and achieves up to 100% final success.
Video2Roleplay: A Multimodal Dataset and Framework for Video-Guided
  Role-playing Agents
Authors: name: Xueqiao Zhang, name: Chao Zhang, name: Jingtao Xu, name: Yifan Zhu, name: Xin Shi, name: Yi Yang, name: Yawei Luo
Published: 2025-09-17 02:50:54

Role-playing agents (RPAs) have attracted growing interest for their ability
to simulate immersive and interactive characters. However, existing approaches
primarily focus on static role profiles, overlooking the dynamic perceptual
abilities inherent to humans. To bridge this gap, we introduce the concept of
dynamic role profiles by incorporating video modality into RPAs. To support
this, we construct Role-playing-Video60k, a large-scale, high-quality dataset
comprising 60k videos and 700k corresponding dialogues. Based on this dataset,
we develop a comprehensive RPA framework that combines adaptive temporal
sampling with both dynamic and static role profile representations.
Specifically, the dynamic profile is created by adaptively sampling video
frames and feeding them to the LLM in temporal order, while the static profile
consists of (1) character dialogues from training videos during fine-tuning,
and (2) a summary context from the input video during inference. This joint
integration enables RPAs to generate greater responses. Furthermore, we propose
a robust evaluation method covering eight metrics. Experimental results
demonstrate the effectiveness of our framework, highlighting the importance of
dynamic role profiles in developing RPAs.
RGB-Only Supervised Camera Parameter Optimization in Dynamic Scenes
Authors: name: Fang Li, name: Hao Zhang, name: Narendra Ahuja
Published: 2025-09-18 16:29:07

Although COLMAP has long remained the predominant method for camera parameter
optimization in static scenes, it is constrained by its lengthy runtime and
reliance on ground truth (GT) motion masks for application to dynamic scenes.
Many efforts attempted to improve it by incorporating more priors as
supervision such as GT focal length, motion masks, 3D point clouds, camera
poses, and metric depth, which, however, are typically unavailable in casually
captured RGB videos. In this paper, we propose a novel method for more accurate
and efficient camera parameter optimization in dynamic scenes solely supervised
by a single RGB video. Our method consists of three key components: (1)
Patch-wise Tracking Filters, to establish robust and maximally sparse
hinge-like relations across the RGB video. (2) Outlier-aware Joint
Optimization, for efficient camera parameter optimization by adaptive
down-weighting of moving outliers, without reliance on motion priors. (3) A
Two-stage Optimization Strategy, to enhance stability and optimization speed by
a trade-off between the Softplus limits and convex minima in losses. We
visually and numerically evaluate our camera estimates. To further validate
accuracy, we feed the camera estimates into a 4D reconstruction method and
assess the resulting 3D scenes, and rendered 2D RGB and depth maps. We perform
experiments on 4 real-world datasets (NeRF-DS, DAVIS, iPhone, and TUM-dynamics)
and 1 synthetic dataset (MPI-Sintel), demonstrating that our method estimates
camera parameters more efficiently and accurately with a single RGB video as
the only supervision.
BTL-UI: Blink-Think-Link Reasoning Model for GUI Agent
Authors: name: Shaojie Zhang, name: Ruoceng Zhang, name: Pei Fu, name: Shaokang Wang, name: Jiahui Yang, name: Xin Du, name: Shiqi Cui, name: Bin Qin, name: Ying Huang, name: Zhenbo Luo, name: Jian Luan
Published: 2025-09-19 04:03:44

In the field of AI-driven human-GUI interaction automation, while rapid
advances in multimodal large language models and reinforcement fine-tuning
techniques have yielded remarkable progress, a fundamental challenge persists:
their interaction logic significantly deviates from natural human-GUI
communication patterns. To fill this gap, we propose "Blink-Think-Link" (BTL),
a brain-inspired framework for human-GUI interaction that mimics the human
cognitive process between users and graphical interfaces. The system decomposes
interactions into three biologically plausible phases: (1) Blink - rapid
detection and attention to relevant screen areas, analogous to saccadic eye
movements; (2) Think - higher-level reasoning and decision-making, mirroring
cognitive planning; and (3) Link - generation of executable commands for
precise motor control, emulating human action selection mechanisms.
Additionally, we introduce two key technical innovations for the BTL framework:
(1) Blink Data Generation - an automated annotation pipeline specifically
optimized for blink data, and (2) BTL Reward -- the first rule-based reward
mechanism that enables reinforcement learning driven by both process and
outcome. Building upon this framework, we develop a GUI agent model named
BTL-UI, which demonstrates consistent state-of-the-art performance across both
static GUI understanding and dynamic interaction tasks in comprehensive
benchmarks. These results provide conclusive empirical validation of the
framework's efficacy in developing advanced GUI Agents.
MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid
  Vision Tokenizer
Authors: name: Yanghao Li, name: Rui Qian, name: Bowen Pan, name: Haotian Zhang, name: Haoshuo Huang, name: Bowen Zhang, name: Jialing Tong, name: Haoxuan You, name: Xianzhi Du, name: Zhe Gan, name: Hyunjik Kim, name: Chao Jia, name: Zhenbang Wang, name: Yinfei Yang, name: Mingfei Gao, name: Zi-Yi Dou, name: Wenze Hu, name: Chang Gao, name: Dongxu Li, name: Philipp Dufter, name: Zirui Wang, name: Guoli Yin, name: Zhengdong Zhang, name: Chen Chen, name: Yang Zhao, name: Ruoming Pang, name: Zhifeng Chen
Published: 2025-09-19 17:58:00

Unified multimodal Large Language Models (LLMs) that can both understand and
generate visual content hold immense potential. However, existing open-source
models often suffer from a performance trade-off between these capabilities. We
present Manzano, a simple and scalable unified framework that substantially
reduces this tension by coupling a hybrid image tokenizer with a well-curated
training recipe. A single shared vision encoder feeds two lightweight adapters
that produce continuous embeddings for image-to-text understanding and discrete
tokens for text-to-image generation within a common semantic space. A unified
autoregressive LLM predicts high-level semantics in the form of text and image
tokens, with an auxiliary diffusion decoder subsequently translating the image
tokens into pixels. The architecture, together with a unified training recipe
over understanding and generation data, enables scalable joint learning of both
capabilities. Manzano achieves state-of-the-art results among unified models,
and is competitive with specialist models, particularly on text-rich
evaluation. Our studies show minimal task conflicts and consistent gains from
scaling model size, validating our design choice of a hybrid tokenizer.
EdiVal-Agent: An Object-Centric Framework for Automated, Scalable,
  Fine-Grained Evaluation of Multi-Turn Editing
Authors: name: Tianyu Chen, name: Yasi Zhang, name: Zhi Zhang, name: Peiyu Yu, name: Shu Wang, name: Zhendong Wang, name: Kevin Lin, name: Xiaofei Wang, name: Zhengyuan Yang, name: Linjie Li, name: Chung-Ching Lin, name: Jianwen Xie, name: Oscar Leong, name: Lijuan Wang, name: Ying Nian Wu, name: Mingyuan Zhou
Published: 2025-09-16 17:45:39

Instruction-based image editing has advanced rapidly, yet reliable and
interpretable evaluation remains a bottleneck. Current protocols either (i)
depend on paired reference images -- resulting in limited coverage and
inheriting biases from prior generative models -- or (ii) rely solely on
zero-shot vision-language models (VLMs), whose prompt-based assessments of
instruction following, content consistency, and visual quality are often
imprecise.
  To address this, we introduce EdiVal-Agent, an automated, scalable, and
fine-grained evaluation framework for multi-turn instruction-based editing from
an object-centric perspective, supported by a suite of expert tools. Given an
image, EdiVal-Agent first decomposes it into semantically meaningful objects,
then synthesizes diverse, context-aware editing instructions. For evaluation,
it integrates VLMs with open-vocabulary object detectors to assess instruction
following, uses semantic-level feature extractors to evaluate content
consistency, and leverages human preference models to judge visual quality. We
show that combining VLMs with object detectors yields stronger agreement with
human judgments in instruction-following evaluation compared to using VLMs
alone and CLIP-based metrics. Furthermore, the pipeline's modular design allows
future tools to be seamlessly integrated, enhancing evaluation accuracy over
time.
  Instantiating this pipeline, we build EdiVal-Bench, a multi-turn editing
benchmark covering 9 instruction types and 11 state-of-the-art editing models
spanning autoregressive (AR) (including Nano Banana, GPT-Image-1),
flow-matching, and diffusion paradigms. We demonstrate that EdiVal-Agent can be
used to identify existing failure modes, thereby informing the development of
the next generation of editing models. Project page:
https://tianyucodings.github.io/EdiVAL-page/.
Can Multimodal LLMs See Materials Clearly? A Multimodal Benchmark on
  Materials Characterization
Authors: name: Zhengzhao Lai, name: Youbin Zheng, name: Zhenyang Cai, name: Haonan Lyu, name: Jinpu Yang, name: Hongqing Liang, name: Yan Hu, name: Benyou Wang
Published: 2025-09-11 09:50:16

Materials characterization is fundamental to acquiring materials information,
revealing the processing-microstructure-property relationships that guide
material design and optimization. While multimodal large language models
(MLLMs) have recently shown promise in generative and predictive tasks within
materials science, their capacity to understand real-world characterization
imaging data remains underexplored. To bridge this gap, we present MatCha, the
first benchmark for materials characterization image understanding, comprising
1,500 questions that demand expert-level domain expertise. MatCha encompasses
four key stages of materials research comprising 21 distinct tasks, each
designed to reflect authentic challenges faced by materials scientists. Our
evaluation of state-of-the-art MLLMs on MatCha reveals a significant
performance gap compared to human experts. These models exhibit degradation
when addressing questions requiring higher-level expertise and sophisticated
visual perception. Simple few-shot and chain-of-thought prompting struggle to
alleviate these limitations. These findings highlight that existing MLLMs still
exhibit limited adaptability to real-world materials characterization
scenarios. We hope MatCha will facilitate future research in areas such as new
material discovery and autonomous scientific agents. MatCha is available at
https://github.com/FreedomIntelligence/MatCha.
Developer-LLM Conversations: An Empirical Study of Interactions and
  Generated Code Quality
Authors: name: Suzhen Zhong, name: Ying Zou, name: Bram Adams
Published: 2025-09-12 16:52:49

Large Language Models (LLMs) are becoming integral to modern software
development workflows, assisting developers with code generation, API
explanation, and iterative problem-solving through natural language
conversations. Despite widespread adoption, there is limited understanding of
how developers interact with LLMs in practice and how these conversational
dynamics influence task outcomes, code quality, and software engineering
workflows. To address this, we leverage CodeChat, a large dataset comprising
82,845 real-world developer-LLM conversations, containing 368,506 code snippets
generated across over 20 programming languages, derived from the WildChat
dataset. We find that LLM responses are substantially longer than developer
prompts, with a median token-length ratio of 14:1. Multi-turn conversations
account for 68% of the dataset and often evolve due to shifting requirements,
incomplete prompts, or clarification requests. Topic analysis identifies web
design (9.6% of conversations) and neural network training (8.7% of
conversations) as the most frequent LLM-assisted tasks. Evaluation across five
languages (i.e., Python, JavaScript, C++, Java, and C#) reveals prevalent and
language-specific issues in LLM-generated code: generated Python and JavaScript
code often include undefined variables (83.4% and 75.3% of code snippets,
respectively); Java code lacks required comments (75.9%); C++ code frequently
omits headers (41.1%) and C# code shows unresolved namespaces (49.2%). During a
conversation, syntax and import errors persist across turns; however,
documentation quality in Java improves by up to 14.7%, and import handling in
Python improves by 3.7% over 5 turns. Prompts that point out mistakes in code
generated in prior turns and explicitly request a fix are most effective for
resolving errors.
Agentic Software Engineering: Foundational Pillars and a Research
  Roadmap
Authors: name: Ahmed E. Hassan, name: Hao Li, name: Dayi Lin, name: Bram Adams, name: Tse-Hsun Chen, name: Yutaro Kashiwa, name: Dong Qiu
Published: 2025-09-07 21:40:10

Agentic Software Engineering (SE 3.0) represents a new era where intelligent
agents are tasked not with simple code generation, but with achieving complex,
goal-oriented SE objectives. To harness these new capabilities while ensuring
trustworthiness, we must recognize a fundamental duality within the SE field in
the Agentic SE era, comprising two symbiotic modalities: SE for Humans and SE
for Agents. This duality demands a radical reimagining of the foundational
pillars of SE (actors, processes, tools, and artifacts) which manifest
differently across each modality. We propose two purpose-built workbenches to
support this vision. The Agent Command Environment (ACE) serves as a command
center where humans orchestrate and mentor agent teams, handling outputs such
as Merge-Readiness Packs (MRPs) and Consultation Request Packs (CRPs). The
Agent Execution Environment (AEE) is a digital workspace where agents perform
tasks while invoking human expertise when facing ambiguity or complex
trade-offs. This bi-directional partnership, which supports agent-initiated
human callbacks and handovers, gives rise to new, structured engineering
activities (i.e., processes) that redefine human-AI collaboration, elevating
the practice from agentic coding to true agentic software engineering. This
paper presents the Structured Agentic Software Engineering (SASE) vision,
outlining several of the foundational pillars for the future of SE. The paper
culminates in a research roadmap that identifies a few key challenges and
opportunities while briefly discussing the resulting impact of this future on
SE education. Our goal is not to offer a definitive solution, but to provide a
conceptual scaffold with structured vocabulary to catalyze a community-wide
dialogue, pushing the SE community to think beyond its classic, human-centric
tenets toward a disciplined, scalable, and trustworthy agentic future.