Orionnnn/hub-stats huggingface dataset:
Scaling Language-Free Visual Representation Learning
Authors: name: David Fan, name: Shengbang Tong, name: Jiachen Zhu, name: Koustuv Sinha, name: Zhuang Liu, name: Xinlei Chen, name: Michael Rabbat, name: Nicolas Ballas, name: Yann LeCun, name: Amir Bar, name: Saining Xie
Published: 2025-04-01 17:59:15

Visual Self-Supervised Learning (SSL) currently underperforms Contrastive
Language-Image Pretraining (CLIP) in multimodal settings such as Visual
Question Answering (VQA). This multimodal gap is often attributed to the
semantics introduced by language supervision, even though visual SSL and CLIP
models are often trained on different data. In this work, we ask the question:
"Do visual self-supervised approaches lag behind CLIP due to the lack of
language supervision, or differences in the training data?" We study this
question by training both visual SSL and CLIP models on the same MetaCLIP data,
and leveraging VQA as a diverse testbed for vision encoders. In this controlled
setup, visual SSL models scale better than CLIP models in terms of data and
model capacity, and visual SSL performance does not saturate even after scaling
up to 7B parameters. Consequently, we observe visual SSL methods achieve
CLIP-level performance on a wide range of VQA and classic vision benchmarks.
These findings demonstrate that pure visual SSL can match language-supervised
visual pretraining at scale, opening new opportunities for vision-centric
representation learning.
DiET-GS: Diffusion Prior and Event Stream-Assisted Motion Deblurring 3D
  Gaussian Splatting
Authors: name: Seungjun Lee, name: Gim Hee Lee
Published: 2025-03-31 15:27:07

Reconstructing sharp 3D representations from blurry multi-view images are
long-standing problem in computer vision. Recent works attempt to enhance
high-quality novel view synthesis from the motion blur by leveraging
event-based cameras, benefiting from high dynamic range and microsecond
temporal resolution. However, they often reach sub-optimal visual quality in
either restoring inaccurate color or losing fine-grained details. In this
paper, we present DiET-GS, a diffusion prior and event stream-assisted motion
deblurring 3DGS. Our framework effectively leverages both blur-free event
streams and diffusion prior in a two-stage training strategy. Specifically, we
introduce the novel framework to constraint 3DGS with event double integral,
achieving both accurate color and well-defined details. Additionally, we
propose a simple technique to leverage diffusion prior to further enhance the
edge details. Qualitative and quantitative results on both synthetic and
real-world data demonstrate that our DiET-GS is capable of producing
significantly better quality of novel views compared to the existing baselines.
Our project page is https://diet-gs.github.io
MixerMDM: Learnable Composition of Human Motion Diffusion Models
Authors: name: Pablo Ruiz-Ponce, name: German Barquero, name: Cristina Palmero, name: Sergio Escalera, name: José García-Rodríguez
Published: 2025-04-01 17:59:44

Generating human motion guided by conditions such as textual descriptions is
challenging due to the need for datasets with pairs of high-quality motion and
their corresponding conditions. The difficulty increases when aiming for finer
control in the generation. To that end, prior works have proposed to combine
several motion diffusion models pre-trained on datasets with different types of
conditions, thus allowing control with multiple conditions. However, the
proposed merging strategies overlook that the optimal way to combine the
generation processes might depend on the particularities of each pre-trained
generative model and also the specific textual descriptions. In this context,
we introduce MixerMDM, the first learnable model composition technique for
combining pre-trained text-conditioned human motion diffusion models. Unlike
previous approaches, MixerMDM provides a dynamic mixing strategy that is
trained in an adversarial fashion to learn to combine the denoising process of
each model depending on the set of conditions driving the generation. By using
MixerMDM to combine single- and multi-person motion diffusion models, we
achieve fine-grained control on the dynamics of every person individually, and
also on the overall interaction. Furthermore, we propose a new evaluation
technique that, for the first time in this task, measures the interaction and
individual quality by computing the alignment between the mixed generated
motions and their conditions as well as the capabilities of MixerMDM to adapt
the mixing throughout the denoising process depending on the motions to mix.
JudgeLRM: Large Reasoning Models as a Judge
Authors: name: Nuo Chen, name: Zhiyuan Hu, name: Qingyun Zou, name: Jiaying Wu, name: Qian Wang, name: Bryan Hooi, name: Bingsheng He
Published: 2025-03-31 02:18:51

The rise of Large Language Models (LLMs) as evaluators offers a scalable
alternative to human annotation, yet existing Supervised Fine-Tuning (SFT) for
judges approaches often fall short in domains requiring complex reasoning. In
this work, we investigate whether LLM judges truly benefit from enhanced
reasoning capabilities. Through a detailed analysis of reasoning requirements
across evaluation tasks, we reveal a negative correlation between SFT
performance gains and the proportion of reasoning-demanding samples -
highlighting the limitations of SFT in such scenarios. To address this, we
introduce JudgeLRM, a family of judgment-oriented LLMs trained using
reinforcement learning (RL) with judge-wise, outcome-driven rewards. JudgeLRM
models consistently outperform both SFT-tuned and state-of-the-art reasoning
models. Notably, JudgeLRM-3B surpasses GPT-4, and JudgeLRM-7B outperforms
DeepSeek-R1 by 2.79% in F1 score, particularly excelling in judge tasks
requiring deep reasoning.
CodeARC: Benchmarking Reasoning Capabilities of LLM Agents for Inductive
  Program Synthesis
Authors: name: Anjiang Wei, name: Tarun Suresh, name: Jiannan Cao, name: Naveen Kannan, name: Yuheng Wu, name: Kai Yan, name: Thiago S. F. X. Teixeira, name: Ke Wang, name: Alex Aiken
Published: 2025-03-29 16:50:39

Inductive program synthesis, or programming by example, requires synthesizing
functions from input-output examples that generalize to unseen inputs. While
large language model agents have shown promise in programming tasks guided by
natural language, their ability to perform inductive program synthesis is
underexplored. Existing evaluation protocols rely on static sets of examples
and held-out tests, offering no feedback when synthesized functions are
incorrect and failing to reflect real-world scenarios such as reverse
engineering. We propose CodeARC, the Code Abstraction and Reasoning Challenge,
a new evaluation framework where agents interact with a hidden target function
by querying it with new inputs, synthesizing candidate functions, and
iteratively refining their solutions using a differential testing oracle. This
interactive setting encourages agents to perform function calls and
self-correction based on feedback. We construct the first large-scale benchmark
for general-purpose inductive program synthesis, featuring 1114 functions.
Among 18 models evaluated, o3-mini performs best with a success rate of 52.7%,
highlighting the difficulty of this task. Fine-tuning LLaMA-3.1-8B-Instruct on
curated synthesis traces yields up to a 31% relative performance gain. CodeARC
provides a more realistic and challenging testbed for evaluating LLM-based
program synthesis and inductive reasoning.
Chapter-Llama: Efficient Chaptering in Hour-Long Videos with LLMs
Authors: name: Lucas Ventura, name: Antoine Yang, name: Cordelia Schmid, name: Gül Varol
Published: 2025-03-31 17:41:29

We address the task of video chaptering, i.e., partitioning a long video
timeline into semantic units and generating corresponding chapter titles. While
relatively underexplored, automatic chaptering has the potential to enable
efficient navigation and content retrieval in long-form videos. In this paper,
we achieve strong chaptering performance on hour-long videos by efficiently
addressing the problem in the text domain with our 'Chapter-Llama' framework.
Specifically, we leverage a pretrained large language model (LLM) with large
context window, and feed as input (i) speech transcripts and (ii) captions
describing video frames, along with their respective timestamps. Given the
inefficiency of exhaustively captioning all frames, we propose a lightweight
speech-guided frame selection strategy based on speech transcript content, and
experimentally demonstrate remarkable advantages. We train the LLM to output
timestamps for the chapter boundaries, as well as free-form chapter titles.
This simple yet powerful approach scales to processing one-hour long videos in
a single forward pass. Our results demonstrate substantial improvements (e.g.,
45.3 vs 26.7 F1 score) over the state of the art on the recent VidChapters-7M
benchmark. To promote further research, we release our code and models at our
project page.
Multi-Token Attention
Authors: name: Olga Golovneva, name: Tianlu Wang, name: Jason Weston, name: Sainbayar Sukhbaatar
Published: 2025-04-01 15:59:32

Soft attention is a critical mechanism powering LLMs to locate relevant parts
within a given context. However, individual attention weights are determined by
the similarity of only a single query and key token vector. This "single token
attention" bottlenecks the amount of information used in distinguishing a
relevant part from the rest of the context. To address this issue, we propose a
new attention method, Multi-Token Attention (MTA), which allows LLMs to
condition their attention weights on multiple query and key vectors
simultaneously. This is achieved by applying convolution operations over
queries, keys and heads, allowing nearby queries and keys to affect each
other's attention weights for more precise attention. As a result, our method
can locate relevant context using richer, more nuanced information that can
exceed a single vector's capacity. Through extensive evaluations, we
demonstrate that MTA achieves enhanced performance on a range of popular
benchmarks. Notably, it outperforms Transformer baseline models on standard
language modeling tasks, and on tasks that require searching for information
within long contexts, where our method's ability to leverage richer information
proves particularly beneficial.
m1: Unleash the Potential of Test-Time Scaling for Medical Reasoning
  with Large Language Models
Authors: name: Xiaoke Huang, name: Juncheng Wu, name: Hui Liu, name: Xianfeng Tang, name: Yuyin Zhou
Published: 2025-04-01 14:57:43

Test-time scaling has emerged as a powerful technique for enhancing the
reasoning capabilities of large language models. However, its effectiveness in
medical reasoning remains uncertain, as the medical domain fundamentally
differs from mathematical tasks in terms of knowledge representation and
decision-making processes. In this paper, we provide the first comprehensive
investigation of test-time scaling for medical reasoning and present m1, a
simple yet effective approach that increases a model's medical reasoning
capability at inference. Our evaluation across diverse medical tasks
demonstrates that test-time scaling consistently enhances medical reasoning,
enabling lightweight fine-tuned models under 10B parameters to establish new
state-of-the-art performance, while our 32B model rivals previous 70B-scale
medical LLMs. However, we identify an optimal reasoning token budget of
approximately 4K, beyond which performance may degrade due to overthinking.
Budget forcing, which extends test-time computation through iterative prompts,
helps models double-check answers but does not necessarily improve the overall
medical QA performance and, in some cases, even introduces errors into
previously correct responses. Our case-by-case analysis identifies insufficient
medical knowledge as a key bottleneck that prevents further performance gains
through test-time scaling. We find that increasing data scale, improving data
quality, and expanding model capacity consistently enhance medical knowledge
grounding, enabling continued performance improvements, particularly on
challenging medical benchmarks where smaller models reach saturation. These
findings underscore fundamental differences between medical and mathematical
reasoning in LLMs, highlighting that enriched medical knowledge, other than
increased reasoning depth alone, is essential for realizing the benefits of
test-time scaling.
Open-Qwen2VL: Compute-Efficient Pre-Training of Fully-Open Multimodal
  LLMs on Academic Resources
Authors: name: Weizhi Wang, name: Yu Tian, name: Linjie Yang, name: Heng Wang, name: Xifeng Yan
Published: 2025-04-01 09:54:00

The reproduction of state-of-the-art multimodal LLM pre-training faces
barriers at every stage of the pipeline, including high-quality data filtering,
multimodal data mixture strategies, sequence packing techniques, and training
frameworks. We introduce Open-Qwen2VL, a fully open-source 2B-parameter
Multimodal Large Language Model pre-trained efficiently on 29M image-text pairs
using only 442 A100-40G GPU hours. Our approach employs low-to-high dynamic
image resolution and multimodal sequence packing to significantly enhance
pre-training efficiency. The training dataset was carefully curated using both
MLLM-based filtering techniques (e.g., MLM-Filter) and conventional CLIP-based
filtering methods, substantially improving data quality and training
efficiency. The Open-Qwen2VL pre-training is conducted on academic level
8xA100-40G GPUs at UCSB on 5B packed multimodal tokens, which is 0.36\% of 1.4T
multimodal pre-training tokens of Qwen2-VL. The final instruction-tuned
Open-Qwen2VL outperforms partially-open state-of-the-art MLLM Qwen2-VL-2B on
various multimodal benchmarks of MMBench, SEEDBench, MMstar, and MathVista,
indicating the remarkable training efficiency of Open-Qwen2VL. We open-source
all aspects of our work, including compute-efficient and data-efficient
training details, data filtering methods, sequence packing scripts,
pre-training data in WebDataset format, FSDP-based training codebase, and both
base and instruction-tuned model checkpoints. We redefine "fully open" for
multimodal LLMs as the complete release of: 1) the training codebase, 2)
detailed data filtering techniques, and 3) all pre-training and supervised
fine-tuning data used to develop the model.
Z1: Efficient Test-time Scaling with Code
Authors: name: Zhaojian Yu, name: Yinghao Wu, name: Yilun Zhao, name: Arman Cohan, name: Xiao-Ping Zhang
Published: 2025-04-01 14:01:50

Large Language Models (LLMs) can achieve enhanced complex problem-solving
through test-time computing scaling, yet this often entails longer contexts and
numerous reasoning token costs. In this paper, we propose an efficient
test-time scaling method that trains LLMs on code-related reasoning
trajectories, facilitating their reduction of excess thinking tokens while
maintaining performance. First, we create Z1-Code-Reasoning-107K, a curated
dataset of simple and complex coding problems paired with their short and long
solution trajectories. Second, we present a novel Shifted Thinking Window to
mitigate overthinking overhead by removing context-delimiting tags (e.g.,
<think>. . . </think>) and capping reasoning tokens. Trained with long and
short trajectory data and equipped with Shifted Thinking Window, our model,
Z1-7B, demonstrates the ability to adjust its reasoning level as the complexity
of problems and exhibits efficient test-time scaling across different reasoning
tasks that matches R1-Distill-Qwen-7B performance with about 30% of its average
thinking tokens. Notably, fine-tuned with only code trajectories, Z1-7B
demonstrates generalization to broader reasoning tasks (47.5% on GPQA Diamond).
Our analysis of efficient reasoning elicitation also provides valuable insights
for future research.
Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on
  Elementary School-Level Reasoning Problems?
Authors: name: Kai Yan, name: Yufei Xu, name: Zhengyin Du, name: Xuesong Yao, name: Zheyu Wang, name: Xiaowen Guo, name: Jiecao Chen
Published: 2025-04-01 07:57:58

The rapid escalation from elementary school-level to frontier problems of the
difficulty for LLM benchmarks in recent years have weaved a miracle for
researchers that we are only inches away from surpassing human intelligence.
However, is the LLMs' remarkable reasoning ability indeed comes from true
intelligence by human standards, or are they simply reciting solutions
witnessed during training at an Internet level? To study this problem, we
propose RoR-Bench, a novel, multi-modal benchmark for detecting LLM's
recitation behavior when asked simple reasoning problems but with conditions
subtly shifted, and conduct empirical analysis on our benchmark. Surprisingly,
we found existing cutting-edge LLMs unanimously exhibits extremely severe
recitation behavior; by changing one phrase in the condition, top models such
as OpenAI-o1 and DeepSeek-R1 can suffer 60% performance loss on elementary
school-level arithmetic and reasoning problems. Such findings are a wake-up
call to the LLM community that compels us to re-evaluate the true intelligence
level of cutting-edge LLMs.
Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies
  Ahead
Authors: name: Vidhisha Balachandran, name: Jingya Chen, name: Lingjiao Chen, name: Shivam Garg, name: Neel Joshi, name: Yash Lara, name: John Langford, name: Besmira Nushi, name: Vibhav Vineet, name: Yue Wu, name: Safoora Yousefi
Published: 2025-03-31 23:40:28

Inference-time scaling can enhance the reasoning capabilities of large
language models (LLMs) on complex problems that benefit from step-by-step
problem solving. Although lengthening generated scratchpads has proven
effective for mathematical tasks, the broader impact of this approach on other
tasks remains less clear. In this work, we investigate the benefits and
limitations of scaling methods across nine state-of-the-art models and eight
challenging tasks, including math and STEM reasoning, calendar planning,
NP-hard problems, navigation, and spatial reasoning. We compare conventional
models (e.g., GPT-4o) with models fine-tuned for inference-time scaling (e.g.,
o1) through evaluation protocols that involve repeated model calls, either
independently or sequentially with feedback. These evaluations approximate
lower and upper performance bounds and potential for future performance
improvements for each model, whether through enhanced training or multi-model
inference systems. Our extensive empirical analysis reveals that the advantages
of inference-time scaling vary across tasks and diminish as problem complexity
increases. In addition, simply using more tokens does not necessarily translate
to higher accuracy in these challenging regimes. Results from multiple
independent runs with conventional models using perfect verifiers show that,
for some tasks, these models can achieve performance close to the average
performance of today's most advanced reasoning models. However, for other
tasks, a significant performance gap remains, even in very high scaling
regimes. Encouragingly, all models demonstrate significant gains when inference
is further scaled with perfect verifiers or strong feedback, suggesting ample
potential for future improvements.
Efficient LLaMA-3.2-Vision by Trimming Cross-attended Visual Features
Authors: name: Jewon Lee, name: Ki-Ung Song, name: Seungmin Yang, name: Donguk Lim, name: Jaeyeon Kim, name: Wooksu Shin, name: Bo-Kyeong Kim, name: Yong Jae Lee, name: Tae-Ho Kim
Published: 2025-04-01 09:10:32

Visual token reduction lowers inference costs caused by extensive image
features in large vision-language models (LVLMs). Unlike relevant studies that
prune tokens in self-attention-only LVLMs, our work uniquely addresses
cross-attention-based models, which achieve superior performance. We identify
that the key-value (KV) cache size for image tokens in cross-attention layers
significantly exceeds that of text tokens in self-attention layers, posing a
major compute bottleneck. To mitigate this issue, we exploit the sparse nature
in cross-attention maps to selectively prune redundant visual features. Our
Trimmed Llama effectively reduces KV cache demands without requiring additional
training. By benefiting from 50%-reduced visual features, our model can reduce
inference latency and memory usage while achieving benchmark parity.
Any2Caption:Interpreting Any Condition to Caption for Controllable Video
  Generation
Authors: name: Shengqiong Wu, name: Weicai Ye, name: Jiahao Wang, name: Quande Liu, name: Xintao Wang, name: Pengfei Wan, name: Di Zhang, name: Kun Gai, name: Shuicheng Yan, name: Hao Fei, name: Tat-Seng Chua
Published: 2025-03-31 17:59:01

To address the bottleneck of accurate user intent interpretation within the
current video generation community, we present Any2Caption, a novel framework
for controllable video generation under any condition. The key idea is to
decouple various condition interpretation steps from the video synthesis step.
By leveraging modern multimodal large language models (MLLMs), Any2Caption
interprets diverse inputs--text, images, videos, and specialized cues such as
region, motion, and camera poses--into dense, structured captions that offer
backbone video generators with better guidance. We also introduce Any2CapIns, a
large-scale dataset with 337K instances and 407K conditions for
any-condition-to-caption instruction tuning. Comprehensive evaluations
demonstrate significant improvements of our system in controllability and video
quality across various aspects of existing video generation models. Project
Page: https://sqwu.top/Any2Cap/
Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for
  Large Language Models
Authors: name: Rui Wang, name: Hongru Wang, name: Boyang Xue, name: Jianhui Pang, name: Shudong Liu, name: Yi Chen, name: Jiahao Qiu, name: Derek Fai Wong, name: Heng Ji, name: Kam-Fai Wong
Published: 2025-03-31 17:58:07

Recent advancements in Large Language Models (LLMs) have significantly
enhanced their ability to perform complex reasoning tasks, transitioning from
fast and intuitive thinking (System 1) to slow and deep reasoning (System 2).
While System 2 reasoning improves task accuracy, it often incurs substantial
computational costs due to its slow thinking nature and inefficient or
unnecessary reasoning behaviors. In contrast, System 1 reasoning is
computationally efficient but leads to suboptimal performance. Consequently, it
is critical to balance the trade-off between performance (benefits) and
computational costs (budgets), giving rise to the concept of reasoning economy.
In this survey, we provide a comprehensive analysis of reasoning economy in
both the post-training and test-time inference stages of LLMs, encompassing i)
the cause of reasoning inefficiency, ii) behavior analysis of different
reasoning patterns, and iii) potential solutions to achieve reasoning economy.
By offering actionable insights and highlighting open challenges, we aim to
shed light on strategies for improving the reasoning economy of LLMs, thereby
serving as a valuable resource for advancing research in this evolving area. We
also provide a public repository to continually track developments in this
fast-evolving field.