Orionnnn/hub-stats huggingface dataset:
Unleashing Hour-Scale Video Training for Long Video-Language
  Understanding
Authors: name: Jingyang Lin, name: Jialian Wu, name: Ximeng Sun, name: Ze Wang, name: Jiang Liu, name: Yusheng Su, name: Xiaodong Yu, name: Hao Chen, name: Jiebo Luo, name: Zicheng Liu, name: Emad Barsoum
Published: 2025-06-05 17:59:04

Recent long-form video-language understanding benchmarks have driven progress
in video large multimodal models (Video-LMMs). However, the scarcity of
well-annotated long videos has left the training of hour-long Video-LLMs
underexplored. To close this gap, we present VideoMarathon, a large-scale
hour-long video instruction-following dataset. This dataset includes around
9,700 hours of long videos sourced from diverse domains, ranging from 3 to 60
minutes per video. Specifically, it contains 3.3M high-quality QA pairs,
spanning six fundamental topics: temporality, spatiality, object, action,
scene, and event. Compared to existing video instruction datasets,
VideoMarathon significantly extends training video durations up to 1 hour, and
supports 22 diverse tasks requiring both short- and long-term video
comprehension. Building on VideoMarathon, we propose Hour-LLaVA, a powerful and
efficient Video-LMM for hour-scale video-language modeling. It enables
hour-long video training and inference at 1-FPS sampling by leveraging a memory
augmentation module, which adaptively integrates user question-relevant and
spatiotemporal-informative semantics from a cached full video context. In our
experiments, Hour-LLaVA achieves the best performance on multiple long
video-language benchmarks, demonstrating the high quality of the VideoMarathon
dataset and the superiority of the Hour-LLaVA model.
CapSpeech: Enabling Downstream Applications in Style-Captioned
  Text-to-Speech
Authors: name: Helin Wang, name: Jiarui Hai, name: Dading Chong, name: Karan Thakkar, name: Tiantian Feng, name: Dongchao Yang, name: Junhyeok Lee, name: Laureano Moro Velazquez, name: Jesus Villalba, name: Zengyi Qin, name: Shrikanth Narayanan, name: Mounya Elhiali, name: Najim Dehak
Published: 2025-06-03 13:28:55

Recent advancements in generative artificial intelligence have significantly
transformed the field of style-captioned text-to-speech synthesis (CapTTS).
However, adapting CapTTS to real-world applications remains challenging due to
the lack of standardized, comprehensive datasets and limited research on
downstream tasks built upon CapTTS. To address these gaps, we introduce
CapSpeech, a new benchmark designed for a series of CapTTS-related tasks,
including style-captioned text-to-speech synthesis with sound events
(CapTTS-SE), accent-captioned TTS (AccCapTTS), emotion-captioned TTS
(EmoCapTTS), and text-to-speech synthesis for chat agent (AgentTTS). CapSpeech
comprises over 10 million machine-annotated audio-caption pairs and nearly 0.36
million human-annotated audio-caption pairs. In addition, we introduce two new
datasets collected and recorded by a professional voice actor and experienced
audio engineers, specifically for the AgentTTS and CapTTS-SE tasks. Alongside
the datasets, we conduct comprehensive experiments using both autoregressive
and non-autoregressive models on CapSpeech. Our results demonstrate
high-fidelity and highly intelligible speech synthesis across a diverse range
of speaking styles. To the best of our knowledge, CapSpeech is the largest
available dataset offering comprehensive annotations for CapTTS-related tasks.
The experiments and findings further provide valuable insights into the
challenges of developing CapTTS systems.
POSS: Position Specialist Generates Better Draft for Speculative
  Decoding
Authors: name: Langlin Huang, name: Chengsong Huang, name: Jixuan Leng, name: Di Huang, name: Jiaxin Huang
Published: 2025-06-04 04:30:30

Speculative decoding accelerates Large Language Model (LLM) inference by
using a small draft model to predict multiple tokens, and a large target model
to verify these tokens in parallel. Recent studies leverage the hidden state of
the target model to enhance draft model prediction accuracy. However, existing
methods suffer from the degrading quality of draft token predictions at later
positions, due to error accumulation in draft model generated features. In this
paper, we propose Position Specialists (PosS), which consist of multiple
position-specialized draft layers to generate tokens at assigned position(s).
Position specialists greatly improve token acceptance rate at later positions
per drafting round, as each specialist only needs to focus on handling a
certain level of draft model feature deviation. Experiment results on
Llama-3-8B-Instruct and Llama-2-13B-chat across six datasets demonstrate that
PosS effectively improves over baselines on average acceptance length and
speed-up ratio. Our codebase is available at https://github.com/shrango/PosS.
OpenThoughts: Data Recipes for Reasoning Models
Authors: name: Etash Guha, name: Ryan Marten, name: Sedrick Keh, name: Negin Raoof, name: Georgios Smyrnis, name: Hritik Bansal, name: Marianna Nezhurina, name: Jean Mercat, name: Trung Vu, name: Zayne Sprague, name: Ashima Suvarna, name: Benjamin Feuer, name: Liangyu Chen, name: Zaid Khan, name: Eric Frankel, name: Sachin Grover, name: Caroline Choi, name: Niklas Muennighoff, name: Shiye Su, name: Wanjia Zhao, name: John Yang, name: Shreyas Pimpalgaonkar, name: Kartik Sharma, name: Charlie Cheng-Jie Ji, name: Yichuan Deng, name: Sarah Pratt, name: Vivek Ramanujan, name: Jon Saad-Falcon, name: Jeffrey Li, name: Achal Dave, name: Alon Albalak, name: Kushal Arora, name: Blake Wulfe, name: Chinmay Hegde, name: Greg Durrett, name: Sewoong Oh, name: Mohit Bansal, name: Saadia Gabriel, name: Aditya Grover, name: Kai-Wei Chang, name: Vaishaal Shankar, name: Aaron Gokaslan, name: Mike A. Merrill, name: Tatsunori Hashimoto, name: Yejin Choi, name: Jenia Jitsev, name: Reinhard Heckel, name: Maheswaran Sathiamoorthy, name: Alexandros G. Dimakis, name: Ludwig Schmidt
Published: 2025-06-04 17:25:39

Reasoning models have made rapid progress on many benchmarks involving math,
code, and science. Yet, there are still many open questions about the best
training recipes for reasoning since state-of-the-art models often rely on
proprietary datasets with little to no public information available. To address
this, the goal of the OpenThoughts project is to create open-source datasets
for training reasoning models. After initial explorations, our OpenThoughts2-1M
dataset led to OpenThinker2-32B, the first model trained on public reasoning
data to match DeepSeek-R1-Distill-32B on standard reasoning benchmarks such as
AIME and LiveCodeBench. We then improve our dataset further by systematically
investigating each step of our data generation pipeline with 1,000+ controlled
experiments, which led to OpenThoughts3. Scaling the pipeline to 1.2M examples
and using QwQ-32B as teacher yields our OpenThinker3-7B model, which achieves
state-of-the-art results: 53% on AIME 2025, 51% on LiveCodeBench 06/24-01/25,
and 54% on GPQA Diamond. All of our datasets and models are available on
https://openthoughts.ai.
Small Language Models are the Future of Agentic AI
Authors: name: Peter Belcak, name: Greg Heinrich, name: Shizhe Diao, name: Yonggan Fu, name: Xin Dong, name: Saurav Muralidharan, name: Yingyan Celine Lin, name: Pavlo Molchanov
Published: 2025-06-02 18:35:16

Large language models (LLMs) are often praised for exhibiting near-human
performance on a wide range of tasks and valued for their ability to hold a
general conversation. The rise of agentic AI systems is, however, ushering in a
mass of applications in which language models perform a small number of
specialized tasks repetitively and with little variation.
  Here we lay out the position that small language models (SLMs) are
sufficiently powerful, inherently more suitable, and necessarily more
economical for many invocations in agentic systems, and are therefore the
future of agentic AI. Our argumentation is grounded in the current level of
capabilities exhibited by SLMs, the common architectures of agentic systems,
and the economy of LM deployment. We further argue that in situations where
general-purpose conversational abilities are essential, heterogeneous agentic
systems (i.e., agents invoking multiple different models) are the natural
choice. We discuss the potential barriers for the adoption of SLMs in agentic
systems and outline a general LLM-to-SLM agent conversion algorithm.
  Our position, formulated as a value statement, highlights the significance of
the operational and economic impact even a partial shift from LLMs to SLMs is
to have on the AI agent industry. We aim to stimulate the discussion on the
effective use of AI resources and hope to advance the efforts to lower the
costs of AI of the present day. Calling for both contributions to and critique
of our position, we commit to publishing all such correspondence at
https://research.nvidia.com/labs/lpr/slm-agents.
RiOSWorld: Benchmarking the Risk of Multimodal Compter-Use Agents
Authors: name: Jingyi Yang, name: Shuai Shao, name: Dongrui Liu, name: Jing Shao
Published: 2025-05-31 16:04:59

With the rapid development of multimodal large language models (MLLMs), they
are increasingly deployed as autonomous computer-use agents capable of
accomplishing complex computer tasks. However, a pressing issue arises: Can the
safety risk principles designed and aligned for general MLLMs in dialogue
scenarios be effectively transferred to real-world computer-use scenarios?
Existing research on evaluating the safety risks of MLLM-based computer-use
agents suffers from several limitations: it either lacks realistic interactive
environments, or narrowly focuses on one or a few specific risk types. These
limitations ignore the complexity, variability, and diversity of real-world
environments, thereby restricting comprehensive risk evaluation for
computer-use agents. To this end, we introduce RiOSWorld, a benchmark
designed to evaluate the potential risks of MLLM-based agents during real-world
computer manipulations. Our benchmark includes 492 risky tasks spanning various
computer applications, involving web, social media, multimedia, os, email, and
office software. We categorize these risks into two major classes based on
their risk source: (i) User-originated risks and (ii) Environmental risks. For
the evaluation, we evaluate safety risks from two perspectives: (i) Risk goal
intention and (ii) Risk goal completion. Extensive experiments with multimodal
agents on RiOSWorld demonstrate that current computer-use agents
confront significant safety risks in real-world scenarios. Our findings
highlight the necessity and urgency of safety alignment for computer-use agents
in real-world computer manipulation, providing valuable insights for developing
trustworthy computer-use agents. Our benchmark is publicly available at
https://yjyddq.github.io/RiOSWorld.github.io/.
CRAWLDoc: A Dataset for Robust Ranking of Bibliographic Documents
Authors: name: Fabian Karl, name: Ansgar Scherp
Published: 2025-06-04 10:52:55

Publication databases rely on accurate metadata extraction from diverse web
sources, yet variations in web layouts and data formats present challenges for
metadata providers. This paper introduces CRAWLDoc, a new method for contextual
ranking of linked web documents. Starting with a publication's URL, such as a
digital object identifier, CRAWLDoc retrieves the landing page and all linked
web resources, including PDFs, ORCID profiles, and supplementary materials. It
embeds these resources, along with anchor texts and the URLs, into a unified
representation. For evaluating CRAWLDoc, we have created a new, manually
labeled dataset of 600 publications from six top publishers in computer
science. Our method CRAWLDoc demonstrates a robust and layout-independent
ranking of relevant documents across publishers and data formats. It lays the
foundation for improved metadata extraction from web documents with various
layouts and formats. Our source code and dataset can be accessed at
https://github.com/FKarl/CRAWLDoc.
Sounding that Object: Interactive Object-Aware Image to Audio Generation
Authors: name: Tingle Li, name: Baihe Huang, name: Xiaobin Zhuang, name: Dongya Jia, name: Jiawei Chen, name: Yuping Wang, name: Zhuo Chen, name: Gopala Anumanchipalli, name: Yuxuan Wang
Published: 2025-06-04 17:57:26

Generating accurate sounds for complex audio-visual scenes is challenging,
especially in the presence of multiple objects and sound sources. In this
paper, we propose an {\em interactive object-aware audio generation} model that
grounds sound generation in user-selected visual objects within images. Our
method integrates object-centric learning into a conditional latent diffusion
model, which learns to associate image regions with their corresponding sounds
through multi-modal attention. At test time, our model employs image
segmentation to allow users to interactively generate sounds at the {\em
object} level. We theoretically validate that our attention mechanism
functionally approximates test-time segmentation masks, ensuring the generated
audio aligns with selected objects. Quantitative and qualitative evaluations
show that our model outperforms baselines, achieving better alignment between
objects and their associated sounds. Project page:
https://tinglok.netlify.app/files/avobject/
Segment Policy Optimization: Effective Segment-Level Credit Assignment
  in RL for Large Language Models
Authors: name: Yiran Guo, name: Lijie Xu, name: Jie Liu, name: Dan Ye, name: Shuang Qiu
Published: 2025-05-29 15:38:19

Enhancing the reasoning capabilities of large language models effectively
using reinforcement learning (RL) remains a crucial challenge. Existing
approaches primarily adopt two contrasting advantage estimation granularities:
Token-level methods (e.g., PPO) aim to provide the fine-grained advantage
signals but suffer from inaccurate estimation due to difficulties in training
an accurate critic model. On the other extreme, trajectory-level methods (e.g.,
GRPO) solely rely on a coarse-grained advantage signal from the final reward,
leading to imprecise credit assignment. To address these limitations, we
propose Segment Policy Optimization (SPO), a novel RL framework that leverages
segment-level advantage estimation at an intermediate granularity, achieving a
better balance by offering more precise credit assignment than trajectory-level
methods and requiring fewer estimation points than token-level methods,
enabling accurate advantage estimation based on Monte Carlo (MC) without a
critic model. SPO features three components with novel strategies: (1) flexible
segment partition; (2) accurate segment advantage estimation; and (3) policy
optimization using segment advantages, including a novel probability-mask
strategy. We further instantiate SPO for two specific scenarios: (1) SPO-chain
for short chain-of-thought (CoT), featuring novel cutpoint-based partition and
chain-based advantage estimation, achieving 6-12 percentage point
improvements in accuracy over PPO and GRPO on GSM8K. (2) SPO-tree for long CoT,
featuring novel tree-based advantage estimation, which significantly reduces
the cost of MC estimation, achieving 7-11 percentage point improvements
over GRPO on MATH500 under 2K and 4K context evaluation. We make our code
publicly available at https://github.com/AIFrameResearch/SPO.
Video-Skill-CoT: Skill-based Chain-of-Thoughts for Domain-Adaptive Video
  Reasoning
Authors: name: Daeun Lee, name: Jaehong Yoon, name: Jaemin Cho, name: Mohit Bansal
Published: 2025-06-04 03:18:01

Recent advances in Chain-of-Thought (CoT) reasoning have improved complex
video understanding, but existing methods often struggle to adapt to
domain-specific skills (e.g., event detection, spatial relation understanding,
emotion understanding) over various video content. To address this, we propose
Video-Skill-CoT (a.k.a. Video-SKoT), a framework that automatically constructs
and leverages skill-aware CoT supervisions for domain-adaptive video reasoning.
First, we construct skill-based CoT annotations: we extract domain-relevant
reasoning skills from training questions, cluster them into a shared skill
taxonomy, and create detailed multi-step CoT rationale tailored to each
video-question pair for training. Second, we introduce a skill-specific expert
learning framework. Each expert module specializes in a subset of reasoning
skills and is trained with lightweight adapters using the collected CoT
supervision. We demonstrate the effectiveness of the proposed approach on three
video understanding benchmarks, where Video-SKoT consistently outperforms
strong baselines. We also provide in-depth analyses on comparing different CoT
annotation pipelines and learned skills over multiple video domains.
Orak: A Foundational Benchmark for Training and Evaluating LLM Agents on
  Diverse Video Games
Authors: name: Dongmin Park, name: Minkyu Kim, name: Beongjun Choi, name: Junhyuck Kim, name: Keon Lee, name: Jonghyun Lee, name: Inkyu Park, name: Byeong-Uk Lee, name: Jaeyoung Hwang, name: Jaewoo Ahn, name: Ameya S. Mahabaleshwarkar, name: Bilal Kartal, name: Pritam Biswas, name: Yoshi Suhara, name: Kangwook Lee, name: Jaewoong Cho
Published: 2025-06-04 06:40:33

Large Language Model (LLM) agents are reshaping the game industry,
particularly with more intelligent and human-preferable game characters.
However, existing game benchmarks fall short of practical needs: they lack
evaluations of diverse LLM capabilities across various game genres, studies of
agentic modules crucial for complex gameplay, and fine-tuning datasets for
aligning pre-trained LLMs into gaming agents. To fill these gaps, we present
\benchname{}, a foundational benchmark designed to train and evaluate
LLM agents across diverse real-world video games. Unlike existing benchmarks,
Orak includes 12 popular video games spanning all major genres, enabling
comprehensive studies of LLM capabilities and agentic modules essential for
intricate game scenarios. To support consistent evaluation of LLMs, we
introduce a plug-and-play interface based on Model Context Protocol (MCP) that
enables LLMs to seamlessly connect with games and manipulate agentic modules.
Additionally, we propose a fine-tuning dataset, consisting of LLM gameplay
trajectories across diverse game genres. Orak offers a comprehensive evaluation
framework, encompassing general game score leaderboards, LLM battle arenas, and
in-depth analyses of visual input state, agentic strategies, and fine-tuning
effects, establishing a foundation towards building generic gaming agents. Code
is available at https://github.com/krafton-ai/Orak.
HTSC-2025: A Benchmark Dataset of Ambient-Pressure High-Temperature
  Superconductors for AI-Driven Critical Temperature Prediction
Authors: name: Xiao-Qi Han, name: Ze-Feng Gao, name: Xin-De Wang, name: Zhenfeng Ouyang, name: Peng-Jie Guo, name: Zhong-Yi Lu
Published: 2025-06-04 11:14:00

The discovery of high-temperature superconducting materials holds great
significance for human industry and daily life. In recent years, research on
predicting superconducting transition temperatures using artificial
intelligence~(AI) has gained popularity, with most of these tools claiming to
achieve remarkable accuracy. However, the lack of widely accepted benchmark
datasets in this field has severely hindered fair comparisons between different
AI algorithms and impeded further advancement of these methods. In this work,
we present the HTSC-2025, an ambient-pressure high-temperature superconducting
benchmark dataset. This comprehensive compilation encompasses theoretically
predicted superconducting materials discovered by theoretical physicists from
2023 to 2025 based on BCS superconductivity theory, including the renowned
X_2YH_6 system, perovskite MXH_3 system, M_3XH_8 system, cage-like
BCN-doped metal atomic systems derived from LaH_{10} structural evolution,
and two-dimensional honeycomb-structured systems evolving from MgB_2. The
HTSC-2025 benchmark has been open-sourced at
https://github.com/xqh19970407/HTSC-2025 and will be continuously updated. This
benchmark holds significant importance for accelerating the discovery of
superconducting materials using AI-based methods.
Solving Inverse Problems with FLAIR
Authors: name: Julius Erbach, name: Dominik Narnhofer, name: Andreas Dombos, name: Bernt Schiele, name: Jan Eric Lenssen, name: Konrad Schindler
Published: 2025-06-03 09:29:47

Flow-based latent generative models such as Stable Diffusion 3 are able to
generate images with remarkable quality, even enabling photorealistic
text-to-image generation. Their impressive performance suggests that these
models should also constitute powerful priors for inverse imaging problems, but
that approach has not yet led to comparable fidelity. There are several key
obstacles: (i) the encoding into a lower-dimensional latent space makes the
underlying (forward) mapping non-linear; (ii) the data likelihood term is
usually intractable; and (iii) learned generative models struggle to recover
rare, atypical data modes during inference. We present FLAIR, a novel training
free variational framework that leverages flow-based generative models as a
prior for inverse problems. To that end, we introduce a variational objective
for flow matching that is agnostic to the type of degradation, and combine it
with deterministic trajectory adjustments to recover atypical modes. To enforce
exact consistency with the observed data, we decouple the optimization of the
data fidelity and regularization terms. Moreover, we introduce a time-dependent
calibration scheme in which the strength of the regularization is modulated
according to off-line accuracy estimates. Results on standard imaging
benchmarks demonstrate that FLAIR consistently outperforms existing diffusion-
and flow-based methods in terms of reconstruction quality and sample diversity.
IllumiCraft: Unified Geometry and Illumination Diffusion for
  Controllable Video Generation
Authors: name: Yuanze Lin, name: Yi-Wen Chen, name: Yi-Hsuan Tsai, name: Ronald Clark, name: Ming-Hsuan Yang
Published: 2025-06-03 17:59:52

Although diffusion-based models can generate high-quality and high-resolution
video sequences from textual or image inputs, they lack explicit integration of
geometric cues when controlling scene lighting and visual appearance across
frames. To address this limitation, we propose IllumiCraft, an end-to-end
diffusion framework accepting three complementary inputs: (1)
high-dynamic-range (HDR) video maps for detailed lighting control; (2)
synthetically relit frames with randomized illumination changes (optionally
paired with a static background reference image) to provide appearance cues;
and (3) 3D point tracks that capture precise 3D geometry information. By
integrating the lighting, appearance, and geometry cues within a unified
diffusion architecture, IllumiCraft generates temporally coherent videos
aligned with user-defined prompts. It supports background-conditioned and
text-conditioned video relighting and provides better fidelity than existing
controllable video generation methods. Project Page:
https://yuanze-lin.me/IllumiCraft_page