Orionnnn/hub-stats huggingface dataset:
Watch and Learn: Learning to Use Computers from Online Videos
Authors: name: Chan Hee Song, name: Yiwen Song, name: Palash Goyal, name: Yu Su, name: Oriana Riva, name: Hamid Palangi, name: Tomas Pfister
Published: 2025-10-06 10:29:00

Computer use agents (CUAs) need to plan task workflows grounded in diverse,
ever-changing applications and environments, but learning is hindered by the
scarcity of large-scale, high-quality training data in the target application.
Existing datasets are domain-specific, static, and costly to annotate, while
current synthetic data generation methods often yield simplistic or misaligned
task demonstrations. To address these limitations, we introduce Watch & Learn
(W&L), a framework that converts human demonstration videos readily available
on the Internet into executable UI trajectories at scale. Instead of directly
generating trajectories or relying on ad hoc reasoning heuristics, we cast the
problem as an inverse dynamics objective: predicting the user's action from
consecutive screen states. This formulation reduces manual engineering, is
easier to learn, and generalizes more robustly across applications. Concretely,
we develop an inverse dynamics labeling pipeline with task-aware video
retrieval, generate over 53k high-quality trajectories from raw web videos, and
demonstrate that these trajectories improve CUAs both as in-context
demonstrations and as supervised training data. On the challenging OSWorld
benchmark, UI trajectories extracted with W&L consistently enhance both
general-purpose and state-of-the-art frameworks in-context, and deliver
stronger gains for open-source models under supervised training. These results
highlight web-scale human demonstration videos as a practical and scalable
foundation for advancing CUAs towards real-world deployment.
Factuality Matters: When Image Generation and Editing Meet Structured
  Visuals
Authors: name: Le Zhuo, name: Songhao Han, name: Yuandong Pu, name: Boxiang Qiu, name: Sayak Paul, name: Yue Liao, name: Yihao Liu, name: Jie Shao, name: Xi Chen, name: Si Liu, name: Hongsheng Li
Published: 2025-10-06 17:56:55

While modern visual generation models excel at creating aesthetically
pleasing natural images, they struggle with producing or editing structured
visuals like charts, diagrams, and mathematical figures, which demand
composition planning, text rendering, and multimodal reasoning for factual
fidelity. To address this, we present the first comprehensive, systematic
investigation of this domain, encompassing data construction, model training,
and an evaluation benchmark. First, we construct a large-scale dataset of 1.3
million high-quality structured image pairs derived from executable drawing
programs and augmented with chain-of-thought reasoning annotations. Building on
it, we train a unified model that integrates a VLM with FLUX.1 Kontext via a
lightweight connector for enhanced multimodal understanding. A three-stage
training curriculum enables progressive feature alignment, knowledge infusion,
and reasoning-augmented generation, further boosted by an external reasoner at
inference time. Finally, we introduce StructBench, a novel benchmark for
generation and editing with over 1,700 challenging instances, and an
accompanying evaluation metric, StructScore, which employs a multi-round Q\&A
protocol to assess fine-grained factual accuracy. Evaluations of 15 models
reveal that even leading closed-source systems remain far from satisfactory.
Our model attains strong editing performance, and inference-time reasoning
yields consistent gains across diverse architectures. By releasing the dataset,
model, and benchmark, we aim to advance unified multimodal foundations for
structured visuals.
Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large
  Multimodal Models
Authors: name: Yunlong Tang, name: Jing Bi, name: Pinxin Liu, name: Zhenyu Pan, name: Zhangyun Tan, name: Qianxiang Shen, name: Jiani Liu, name: Hang Hua, name: Junjia Guo, name: Yunzhong Xiao, name: Chao Huang, name: Zhiyuan Wang, name: Susan Liang, name: Xinyi Liu, name: Yizhi Song, name: Yuhe Nie, name: Jia-Xing Zhong, name: Bozheng Li, name: Daiqing Qi, name: Ziyun Zeng, name: Ali Vosoughi, name: Luchuan Song, name: Zeliang Zhang, name: Daiki Shimada, name: Han Liu, name: Jiebo Luo, name: Chenliang Xu
Published: 2025-10-06 17:10:44

Video understanding represents the most challenging frontier in computer
vision, requiring models to reason about complex spatiotemporal relationships,
long-term dependencies, and multimodal evidence. The recent emergence of
Video-Large Multimodal Models (Video-LMMs), which integrate visual encoders
with powerful decoder-based language models, has demonstrated remarkable
capabilities in video understanding tasks. However, the critical phase that
transforms these models from basic perception systems into sophisticated
reasoning engines, post-training, remains fragmented across the literature.
This survey provides the first comprehensive examination of post-training
methodologies for Video-LMMs, encompassing three fundamental pillars:
supervised fine-tuning (SFT) with chain-of-thought, reinforcement learning (RL)
from verifiable objectives, and test-time scaling (TTS) through enhanced
inference computation. We present a structured taxonomy that clarifies the
roles, interconnections, and video-specific adaptations of these techniques,
addressing unique challenges such as temporal localization, spatiotemporal
grounding, long video efficiency, and multimodal evidence integration. Through
systematic analysis of representative methods, we synthesize key design
principles, insights, and evaluation protocols while identifying critical open
challenges in reward design, scalability, and cost-performance optimization. We
further curate essential benchmarks, datasets, and metrics to facilitate
rigorous assessment of post-training effectiveness. This survey aims to provide
researchers and practitioners with a unified framework for advancing Video-LMM
capabilities. Additional resources and updates are maintained at:
https://github.com/yunlong10/Awesome-Video-LMM-Post-Training
VChain: Chain-of-Visual-Thought for Reasoning in Video Generation
Authors: name: Ziqi Huang, name: Ning Yu, name: Gordon Chen, name: Haonan Qiu, name: Paul Debevec, name: Ziwei Liu
Published: 2025-10-06 17:57:59

Recent video generation models can produce smooth and visually appealing
clips, but they often struggle to synthesize complex dynamics with a coherent
chain of consequences. Accurately modeling visual outcomes and state
transitions over time remains a core challenge. In contrast, large language and
multimodal models (e.g., GPT-4o) exhibit strong visual state reasoning and
future prediction capabilities. To bridge these strengths, we introduce VChain,
a novel inference-time chain-of-visual-thought framework that injects visual
reasoning signals from multimodal models into video generation. Specifically,
VChain contains a dedicated pipeline that leverages large multimodal models to
generate a sparse set of critical keyframes as snapshots, which are then used
to guide the sparse inference-time tuning of a pre-trained video generator only
at these key moments. Our approach is tuning-efficient, introduces minimal
overhead and avoids dense supervision. Extensive experiments on complex,
multi-step scenarios show that VChain significantly enhances the quality of
generated videos.
Hybrid Architectures for Language Models: Systematic Analysis and Design
  Insights
Authors: name: Sangmin Bae, name: Bilge Acun, name: Haroun Habeeb, name: Seungyeon Kim, name: Chien-Yu Lin, name: Liang Luo, name: Junjie Wang, name: Carole-Jean Wu
Published: 2025-10-06 13:30:07

Recent progress in large language models demonstrates that hybrid
architectures--combining self-attention mechanisms with structured state space
models like Mamba--can achieve a compelling balance between modeling quality
and computational efficiency, particularly for long-context tasks. While these
hybrid models show promising performance, systematic comparisons of
hybridization strategies and analyses on the key factors behind their
effectiveness have not been clearly shared to the community. In this work, we
present a holistic evaluation of hybrid architectures based on inter-layer
(sequential) or intra-layer (parallel) fusion. We evaluate these designs from a
variety of perspectives: language modeling performance, long-context
capabilities, scaling analysis, and training and inference efficiency. By
investigating the core characteristics of their computational primitive, we
identify the most critical elements for each hybridization strategy and further
propose optimal design recipes for both hybrid models. Our comprehensive
analysis provides practical guidance and valuable insights for developing
hybrid language models, facilitating the optimization of architectural
configurations.
Paper2Video: Automatic Video Generation from Scientific Papers
Authors: name: Zeyu Zhu, name: Kevin Qinghong Lin, name: Mike Zheng Shou
Published: 2025-10-06 17:58:02

Academic presentation videos have become an essential medium for research
communication, yet producing them remains highly labor-intensive, often
requiring hours of slide design, recording, and editing for a short 2 to 10
minutes video. Unlike natural video, presentation video generation involves
distinctive challenges: inputs from research papers, dense multi-modal
information (text, figures, tables), and the need to coordinate multiple
aligned channels such as slides, subtitles, speech, and human talker. To
address these challenges, we introduce PaperTalker, the first benchmark of 101
research papers paired with author-created presentation videos, slides, and
speaker metadata. We further design four tailored evaluation metrics--Meta
Similarity, PresentArena, PresentQuiz, and IP Memory--to measure how videos
convey the paper's information to the audience. Building on this foundation, we
propose PaperTalker, the first multi-agent framework for academic presentation
video generation. It integrates slide generation with effective layout
refinement by a novel effective tree search visual choice, cursor grounding,
subtitling, speech synthesis, and talking-head rendering, while parallelizing
slide-wise generation for efficiency. Experiments on Paper2Video demonstrate
that the presentation videos produced by our approach are more faithful and
informative than existing baselines, establishing a practical step toward
automated and ready-to-use academic video generation. Our dataset, agent, and
code are available at https://github.com/showlab/Paper2Video.
Good Intentions Beyond ACL: Who Does NLP for Social Good, and Where?
Authors: name: Grace LeFevre, name: Qingcheng Zeng, name: Adam Leif, name: Jason Jewell, name: Denis Peskoff, name: Rob Voigt
Published: 2025-10-06 02:04:42

The social impact of Natural Language Processing (NLP) is increasingly
important, with a rising community focus on initiatives related to NLP for
Social Good (NLP4SG). Indeed, in recent years, almost 20% of all papers in the
ACL Anthology address topics related to social good as defined by the UN
Sustainable Development Goals (Adauto et al., 2023). In this study, we take an
author- and venue-level perspective to map the landscape of NLP4SG, quantifying
the proportion of work addressing social good concerns both within and beyond
the ACL community, by both core ACL contributors and non-ACL authors. With this
approach we discover two surprising facts about the landscape of NLP4SG. First,
ACL authors are dramatically more likely to do work addressing social good
concerns when publishing in venues outside of ACL. Second, the vast majority of
publications using NLP techniques to address concerns of social good are done
by non-ACL authors in venues outside of ACL. We discuss the implications of
these findings on agenda-setting considerations for the ACL community related
to NLP4SG.
Reinforce-Ada: An Adaptive Sampling Framework for Reinforce-Style LLM
  Training
Authors: name: Wei Xiong, name: Chenlu Ye, name: Baohao Liao, name: Hanze Dong, name: Xinxing Xu, name: Christof Monz, name: Jiang Bian, name: Nan Jiang, name: Tong Zhang
Published: 2025-10-06 16:34:09

Reinforcement learning applied to large language models (LLMs) for reasoning
tasks is often bottlenecked by unstable gradient estimates due to fixed and
uniform sampling of responses across prompts. Prior work such as GVM-RAFT
addresses this by dynamically allocating inference budget per prompt to
minimize stochastic gradient variance under a budget constraint. Inspired by
this insight, we propose Reinforce-Ada, an adaptive sampling framework for
online RL post-training of LLMs that continuously reallocates sampling effort
to the prompts with the greatest uncertainty or learning potential. Unlike
conventional two-stage allocation methods, Reinforce-Ada interleaves estimation
and sampling in an online successive elimination process, and automatically
stops sampling for a prompt once sufficient signal is collected. To stabilize
updates, we form fixed-size groups with enforced reward diversity and compute
advantage baselines using global statistics aggregated over the adaptive
sampling phase. Empirical results across multiple model architectures and
reasoning benchmarks show that Reinforce-Ada accelerates convergence and
improves final performance compared to GRPO, especially when using the balanced
sampling variant. Our work highlights the central role of variance-aware,
adaptive data curation in enabling efficient and reliable reinforcement
learning for reasoning-capable LLMs. Code is available at
https://github.com/RLHFlow/Reinforce-Ada.
Judging with Confidence: Calibrating Autoraters to Preference
  Distributions
Authors: name: Zhuohang Li, name: Xiaowei Li, name: Chengyu Huang, name: Guowang Li, name: Katayoon Goshvadi, name: Bo Dai, name: Dale Schuurmans, name: Paul Zhou, name: Hamid Palangi, name: Yiwen Song, name: Palash Goyal, name: Murat Kantarcioglu, name: Bradley A. Malin, name: Yuan Xue
Published: 2025-09-30 20:36:41

The alignment of large language models (LLMs) with human values increasingly
relies on using other LLMs as automated judges, or ``autoraters''. However,
their reliability is limited by a foundational issue: they are trained on
discrete preference labels, forcing a single ground truth onto tasks that are
often subjective, ambiguous, or nuanced. We argue that a reliable autorater
must learn to model the full distribution of preferences defined by a target
population. In this paper, we propose a general framework for calibrating
probabilistic autoraters to any given preference distribution. We formalize the
problem and present two learning methods tailored to different data conditions:
1) a direct supervised fine-tuning for dense, probabilistic labels, and 2) a
reinforcement learning approach for sparse, binary labels. Our empirical
results show that finetuning autoraters with a distribution-matching objective
leads to verbalized probability predictions that are better aligned with the
target preference distribution, with improved calibration and significantly
lower positional bias, all while preserving performance on objective tasks.
Front-Loading Reasoning: The Synergy between Pretraining and
  Post-Training Data
Authors: name: Syeda Nahida Akter, name: Shrimai Prabhumoye, name: Eric Nyberg, name: Mostofa Patwary, name: Mohammad Shoeybi, name: Yejin Choi, name: Bryan Catanzaro
Published: 2025-09-26 20:08:51

The prevailing paradigm for enhancing the reasoning abilities of LLMs
revolves around post-training on high-quality, reasoning-intensive data. While
emerging literature suggests that reasoning data is increasingly incorporated
also during the mid-training stage-a practice that is relatively more
proprietary and less openly characterized-the role of such data in pretraining
remains unclear. In particular, due to the opaqueness of pretraining corpora in
most frontier models, the effect of reasoning data introduced at different
phases of pre- and/or post-training is relatively less reported in the
scientific literature. This raises several important questions: Is adding
reasoning data earlier during pretraining any better than introducing it during
post-training? Could earlier inclusion risk overfitting and harm
generalization, or instead establish durable foundations that later fine-tuning
cannot recover? We conduct the first systematic study of how reasoning
data-varying in scale, diversity, and quality-affects LLM performance when
introduced at different stages of training. We find that front-loading
reasoning data into pretraining is critical (19% avg gain), establishing
foundational capabilities that cannot be fully replicated by later-stage SFT,
even with more data. We uncover an asymmetric principle for optimal data
allocation: pretraining benefits most from broad diversity in reasoning
patterns (11% avg gain), while SFT is more sensitive to data quality (15% avg
gain). We show that high-quality pretraining data has latent effects, activated
only after SFT, and that naively scaling SFT data can be detrimental, washing
away the benefits of early reasoning injection. Our results challenge the
conventional separation of language modeling and reasoning, providing a
principled guide for strategically allocating data across the entire training
pipeline to build more capable models.
Reactive Transformer (RxT) -- Stateful Real-Time Processing for
  Event-Driven Reactive Language Models
Authors: name: Adam Filipek
Published: 2025-10-03 23:18:07

The Transformer architecture has become the de facto standard for Large
Language Models (LLMs), demonstrating remarkable capabilities in language
understanding and generation. However, its application in conversational AI is
fundamentally constrained by its stateless nature and the quadratic
computational complexity (O(L^2)) with respect to sequence length L.
Current models emulate memory by reprocessing an ever-expanding conversation
history with each turn, leading to prohibitive costs and latency in long
dialogues. This paper introduces the Reactive Transformer (RxT), a novel
architecture designed to overcome these limitations by shifting from a
data-driven to an event-driven paradigm. RxT processes each conversational turn
as a discrete event in real-time, maintaining context in an integrated,
fixed-size Short-Term Memory (STM) system. The architecture features a distinct
operational cycle where a generator-decoder produces a response based on the
current query and the previous memory state, after which a memory-encoder and a
dedicated Memory Attention network asynchronously update the STM with a
representation of the complete interaction. This design fundamentally alters
the scaling dynamics, reducing the total user-facing cost of a conversation
from quadratic (O(N^2 cdot T)) to linear (O(N cdot T)) with respect to
the number of interactions N. By decoupling response generation from memory
updates, RxT achieves low latency, enabling truly real-time, stateful, and
economically viable long-form conversations. We validated our architecture with
a series of proof-of-concept experiments on synthetic data, demonstrating
superior performance and constant-time inference latency compared to a baseline
stateless model of comparable size.
Utility-Learning Tension in Self-Modifying Agents
Authors: name: Charles L. Wang, name: Keir Dorchen, name: Peter Jin
Published: 2025-10-05 23:52:16

As systems trend toward superintelligence, a natural modeling premise is that
agents can self-improve along every facet of their own design. We formalize
this with a five-axis decomposition and a decision layer, separating incentives
from learning behavior and analyzing axes in isolation. Our central result
identifies and introduces a sharp utility--learning tension, the structural
conflict in self-modifying systems whereby utility-driven changes that improve
immediate or expected performance can also erode the statistical preconditions
for reliable learning and generalization. Our findings show that
distribution-free guarantees are preserved iff the policy-reachable model
family is uniformly capacity-bounded; when capacity can grow without limit,
utility-rational self-changes can render learnable tasks unlearnable. Under
standard assumptions common in practice, these axes reduce to the same capacity
criterion, yielding a single boundary for safe self-modification. Numerical
experiments across several axes validate the theory by comparing destructive
utility policies against our proposed two-gate policies that preserve
learnability.
Efficient Test-Time Scaling for Small Vision-Language Models
Authors: name: Mehmet Onurcan Kaya, name: Desmond Elliott, name: Dim P. Papadopoulos
Published: 2025-10-03 23:49:06

Small Vision-Language Models (VLMs) provide a computationally efficient
alternative to larger models, at the cost of weaker generalization abilities
and downstream task performance. These shortcomings could be addressed by
test-time scaling techniques, but existing methods are typically
computationally demanding, contradicting the resource-efficient design goals of
small models. To address these limitations, we propose two novel and efficient
test-time scaling strategies that leverage the model-internal features rather
than external supervision: (i) Test-Time Augmentation (TTAug), which generates
multiple augmented inputs and aggregates outputs at the token level without
parameter updates, and (ii) Test-Time Adaptation (TTAdapt), which adapts model
parameters during inference using consensus-based pseudolabels from TTAug.
Through extensive experiments across nine benchmarks, we demonstrate consistent
performance improvements while maintaining computational efficiency suitable
for resource-constrained environments. The generality of our approach is
demonstrated both within models at different scales and across different VLMs
without additional tuning.