Orionnnn/hub-stats huggingface dataset:
Sigma: Differential Rescaling of Query, Key and Value for Efficient
  Language Models
Authors: name: Zhenghao Lin, name: Zihao Tang, name: Xiao Liu, name: Yeyun Gong, name: Yi Cheng, name: Qi Chen, name: Hang Li, name: Ying Xin, name: Ziyue Yang, name: Kailai Yang, name: Yu Yan, name: Xiao Liang, name: Shuai Lu, name: Yiming Huang, name: Zheheng Luo, name: Lei Qu, name: Xuan Feng, name: Yaoxiang Wang, name: Yuqing Xia, name: Feiyang Chen, name: Yuting Jiang, name: Yasen Hu, name: Hao Ni, name: Binyang Li, name: Guoshuai Zhao, name: Jui-Hao Chiang, name: Zhongxin Guo, name: Chen Lin, name: Kun Kuang, name: Wenjie Li, name: Yelong Shen, name: Jian Jiao, name: Peng Cheng, name: Mao Yang
Published: 2025-01-23 12:58:14

We introduce Sigma, an efficient large language model specialized for the
system domain, empowered by a novel architecture including DiffQKV attention,
and pre-trained on our meticulously collected system domain data. DiffQKV
attention significantly enhances the inference efficiency of Sigma by
optimizing the Query (Q), Key (K), and Value (V) components in the attention
mechanism differentially, based on their varying impacts on the model
performance and efficiency indicators. Specifically, we (1) conduct extensive
experiments that demonstrate the model's varying sensitivity to the compression
of K and V components, leading to the development of differentially compressed
KV, and (2) propose augmented Q to expand the Q head dimension, which enhances
the model's representation capacity with minimal impacts on the inference
speed. Rigorous theoretical and empirical analyses reveal that DiffQKV
attention significantly enhances efficiency, achieving up to a 33.36%
improvement in inference speed over the conventional grouped-query attention
(GQA) in long-context scenarios. We pre-train Sigma on 6T tokens from various
sources, including 19.5B system domain data that we carefully collect and 1T
tokens of synthesized and rewritten data. In general domains, Sigma achieves
comparable performance to other state-of-arts models. In the system domain, we
introduce the first comprehensive benchmark AIMicius, where Sigma demonstrates
remarkable performance across all tasks, significantly outperforming GPT-4 with
an absolute improvement up to 52.5%.
Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary
  Feedback
Authors: name: Yen-Ting Lin, name: Di Jin, name: Tengyu Xu, name: Tianhao Wu, name: Sainbayar Sukhbaatar, name: Chen Zhu, name: Yun He, name: Yun-Nung Chen, name: Jason Weston, name: Yuandong Tian, name: Arash Rahnama, name: Sinong Wang, name: Hao Ma, name: Han Fang
Published: 2025-01-18 15:38:03

Large language models (LLMs) have recently demonstrated remarkable success in
mathematical reasoning. Despite progress in methods like chain-of-thought
prompting and self-consistency sampling, these advances often focus on final
correctness without ensuring that the underlying reasoning process is coherent
and reliable. This paper introduces Step-KTO, a training framework that
combines process-level and outcome-level binary feedback to guide LLMs toward
more trustworthy reasoning trajectories. By providing binary evaluations for
both the intermediate reasoning steps and the final answer, Step-KTO encourages
the model to adhere to logical progressions rather than relying on superficial
shortcuts. Our experiments on challenging mathematical benchmarks show that
Step-KTO significantly improves both final answer accuracy and the quality of
intermediate reasoning steps. For example, on the MATH-500 dataset, Step-KTO
achieves a notable improvement in Pass@1 accuracy over strong baselines. These
results highlight the promise of integrating stepwise process feedback into LLM
training, paving the way toward more interpretable and dependable reasoning
capabilities.
Can We Generate Images with CoT? Let's Verify and Reinforce Image
  Generation Step by Step
Authors: name: Ziyu Guo, name: Renrui Zhang, name: Chengzhuo Tong, name: Zhizheng Zhao, name: Peng Gao, name: Hongsheng Li, name: Pheng-Ann Heng
Published: 2025-01-23 18:59:43

Chain-of-Thought (CoT) reasoning has been extensively explored in large
models to tackle complex understanding tasks. However, it still remains an open
question whether such strategies can be applied to verifying and reinforcing
image generation scenarios. In this paper, we provide the first comprehensive
investigation of the potential of CoT reasoning to enhance autoregressive image
generation. We focus on three techniques: scaling test-time computation for
verification, aligning model preferences with Direct Preference Optimization
(DPO), and integrating these techniques for complementary effects. Our results
demonstrate that these approaches can be effectively adapted and combined to
significantly improve image generation performance. Furthermore, given the
pivotal role of reward models in our findings, we propose the Potential
Assessment Reward Model (PARM) and PARM++, specialized for autoregressive image
generation. PARM adaptively assesses each generation step through a potential
assessment approach, merging the strengths of existing reward models, and
PARM++ further introduces a reflection mechanism to self-correct the generated
unsatisfactory image. Using our investigated reasoning strategies, we enhance a
baseline model, Show-o, to achieve superior results, with a significant +24%
improvement on the GenEval benchmark, surpassing Stable Diffusion 3 by +15%. We
hope our study provides unique insights and paves a new path for integrating
CoT reasoning with autoregressive image generation. Code and models are
released at https://github.com/ZiyuGuo99/Image-Generation-CoT
Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass
Authors: name: Jianing Yang, name: Alexander Sax, name: Kevin J. Liang, name: Mikael Henaff, name: Hao Tang, name: Ang Cao, name: Joyce Chai, name: Franziska Meier, name: Matt Feiszli
Published: 2025-01-23 18:59:55

Multi-view 3D reconstruction remains a core challenge in computer vision,
particularly in applications requiring accurate and scalable representations
across diverse perspectives. Current leading methods such as DUSt3R employ a
fundamentally pairwise approach, processing images in pairs and necessitating
costly global alignment procedures to reconstruct from multiple views. In this
work, we propose Fast 3D Reconstruction (Fast3R), a novel multi-view
generalization to DUSt3R that achieves efficient and scalable 3D reconstruction
by processing many views in parallel. Fast3R's Transformer-based architecture
forwards N images in a single forward pass, bypassing the need for iterative
alignment. Through extensive experiments on camera pose estimation and 3D
reconstruction, Fast3R demonstrates state-of-the-art performance, with
significant improvements in inference speed and reduced error accumulation.
These results establish Fast3R as a robust alternative for multi-view
applications, offering enhanced scalability without compromising reconstruction
accuracy.
Pairwise RM: Perform Best-of-N Sampling with Knockout Tournament
Authors: name: Yantao Liu, name: Zijun Yao, name: Rui Min, name: Yixin Cao, name: Lei Hou, name: Juanzi Li
Published: 2025-01-22 16:49:37

Best-of-N (BoN) sampling, a common strategy for test-time scaling of Large
Language Models (LLMs), relies on reward models to select the best candidate
solution from multiple generations. However, traditional reward models often
assign arbitrary and inconsistent scores, limiting their effectiveness. To
address this, we propose a Pairwise Reward Model (Pairwise RM) combined with a
knockout tournament for BoN sampling. Instead of assigning absolute scores,
given one math problem, Pairwise RM evaluates two candidate solutions'
correctness simultaneously. This approach eliminates the need for arbitrary
scoring and enables cross-validation of solutions through parallel comparison.
In the knockout tournament, Pairwise RM conducts pairwise comparisons between
candidate solutions and eliminates the incorrect ones iteratively. We construct
\ourdataset, a large-scale dataset of 443K pairwise comparisons derived from
NumiaMath and annotated using gemini-1.5-flash, and train the Pairwise
RM via supervised fine-tuning. Experiments on MATH-500 and the Olympiad Bench
demonstrate significant improvements over traditional discriminative reward
models. And a 40\% to 60\% relative improvement is achieved on the top 50\%
challenging problems.
Test-Time Preference Optimization: On-the-Fly Alignment via Iterative
  Textual Feedback
Authors: name: Yafu Li, name: Xuyang Hu, name: Xiaoye Qu, name: Linjie Li, name: Yu Cheng
Published: 2025-01-22 14:15:46

Large language models (LLMs) demonstrate impressive performance but lack the
flexibility to adapt to human preferences quickly without retraining. In this
work, we introduce Test-time Preference Optimization (TPO), a framework that
aligns LLM outputs with human preferences during inference, removing the need
to update model parameters. Rather than relying on purely numerical rewards,
TPO translates reward signals into textual critiques and uses them as textual
rewards to iteratively refine its response. Evaluations on benchmarks covering
instruction following, preference alignment, safety, and mathematics reveal
that TPO progressively improves alignment with human preferences. Notably,
after only a few TPO steps, the initially unaligned Llama-3.1-70B-SFT model can
surpass the aligned counterpart, Llama-3.1-70B-Instruct. Furthermore, TPO
scales efficiently with both the search width and depth during inference.
Through case studies, we illustrate how TPO exploits the innate capacity of LLM
to interpret and act upon reward signals. Our findings establish TPO as a
practical, lightweight alternative for test-time preference optimization,
achieving alignment on the fly. Our code is publicly available at
https://github.com/yafuly/TPO.
VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video
  Understanding
Authors: name: Boqiang Zhang, name: Kehan Li, name: Zesen Cheng, name: Zhiqiang Hu, name: Yuqian Yuan, name: Guanzheng Chen, name: Sicong Leng, name: Yuming Jiang, name: Hang Zhang, name: Xin Li, name: Peng Jin, name: Wenqi Zhang, name: Fan Wang, name: Lidong Bing, name: Deli Zhao
Published: 2025-01-22 18:59:46

In this paper, we propose VideoLLaMA3, a more advanced multimodal foundation
model for image and video understanding. The core design philosophy of
VideoLLaMA3 is vision-centric. The meaning of "vision-centric" is two-fold: the
vision-centric training paradigm and vision-centric framework design. The key
insight of our vision-centric training paradigm is that high-quality image-text
data is crucial for both image and video understanding. Instead of preparing
massive video-text datasets, we focus on constructing large-scale and
high-quality image-text datasets. VideoLLaMA3 has four training stages: 1)
vision-centric alignment stage, which warms up the vision encoder and
projector; 2) vision-language pretraining stage, which jointly tunes the vision
encoder, projector, and LLM with large-scale image-text data covering multiple
types (including scene images, documents, charts) as well as text-only data. 3)
multi-task fine-tuning stage, which incorporates image-text SFT data for
downstream tasks and video-text data to establish a foundation for video
understanding. 4) video-centric fine-tuning, which further improves the model's
capability in video understanding. As for the framework design, to better
capture fine-grained details in images, the pretrained vision encoder is
adapted to encode images of varying sizes into vision tokens with corresponding
numbers, rather than a fixed number of tokens. For video inputs, we reduce the
number of vision tokens according to their similarity so that the
representation of videos will be more precise and compact. Benefit from
vision-centric designs, VideoLLaMA3 achieves compelling performances in both
image and video understanding benchmarks.
IntellAgent: A Multi-Agent Framework for Evaluating Conversational AI
  Systems
Authors: name: Elad Levi, name: Ilan Kadar
Published: 2025-01-19 14:58:35

Large Language Models (LLMs) are transforming artificial intelligence,
evolving into task-oriented systems capable of autonomous planning and
execution. One of the primary applications of LLMs is conversational AI
systems, which must navigate multi-turn dialogues, integrate domain-specific
APIs, and adhere to strict policy constraints. However, evaluating these agents
remains a significant challenge, as traditional methods fail to capture the
complexity and variability of real-world interactions. We introduce
IntellAgent, a scalable, open-source multi-agent framework designed to evaluate
conversational AI systems comprehensively. IntellAgent automates the creation
of diverse, synthetic benchmarks by combining policy-driven graph modeling,
realistic event generation, and interactive user-agent simulations. This
innovative approach provides fine-grained diagnostics, addressing the
limitations of static and manually curated benchmarks with coarse-grained
metrics. IntellAgent represents a paradigm shift in evaluating conversational
AI. By simulating realistic, multi-policy scenarios across varying levels of
complexity, IntellAgent captures the nuanced interplay of agent capabilities
and policy constraints. Unlike traditional methods, it employs a graph-based
policy model to represent relationships, likelihoods, and complexities of
policy interactions, enabling highly detailed diagnostics. IntellAgent also
identifies critical performance gaps, offering actionable insights for targeted
optimization. Its modular, open-source design supports seamless integration of
new domains, policies, and APIs, fostering reproducibility and community
collaboration. Our findings demonstrate that IntellAgent serves as an effective
framework for advancing conversational AI by addressing challenges in bridging
research and deployment. The framework is available at
https://github.com/plurai-ai/intellagent
Autonomy-of-Experts Models
Authors: name: Ang Lv, name: Ruobing Xie, name: Yining Qian, name: Songhao Wu, name: Xingwu Sun, name: Zhanhui Kang, name: Di Wang, name: Rui Yan
Published: 2025-01-22 18:37:08

Mixture-of-Experts (MoE) models mostly use a router to assign tokens to
specific expert modules, activating only partial parameters and often
outperforming dense models. We argue that the separation between the router's
decision-making and the experts' execution is a critical yet overlooked issue,
leading to suboptimal expert selection and ineffective learning. To address
this, we propose Autonomy-of-Experts (AoE), a novel MoE paradigm in which
experts autonomously select themselves to process inputs. AoE is based on the
insight that an expert is aware of its own capacity to effectively process a
token, an awareness reflected in the scale of its internal activations. In AoE,
routers are removed; instead, experts pre-compute internal activations for
inputs and are ranked based on their activation norms. Only the top-ranking
experts proceed with the forward pass, while the others abort. The overhead of
pre-computing activations is reduced through a low-rank weight factorization.
This self-evaluating-then-partner-comparing approach ensures improved expert
selection and effective learning. We pre-train language models having 700M up
to 4B parameters, demonstrating that AoE outperforms traditional MoE models
with comparable efficiency.
FilmAgent: A Multi-Agent Framework for End-to-End Film Automation in
  Virtual 3D Spaces
Authors: name: Zhenran Xu, name: Longyue Wang, name: Jifang Wang, name: Zhouyi Li, name: Senbao Shi, name: Xue Yang, name: Yiyu Wang, name: Baotian Hu, name: Jun Yu, name: Min Zhang
Published: 2025-01-22 14:36:30

Virtual film production requires intricate decision-making processes,
including scriptwriting, virtual cinematography, and precise actor positioning
and actions. Motivated by recent advances in automated decision-making with
language agent-based societies, this paper introduces FilmAgent, a novel
LLM-based multi-agent collaborative framework for end-to-end film automation in
our constructed 3D virtual spaces. FilmAgent simulates various crew roles,
including directors, screenwriters, actors, and cinematographers, and covers
key stages of a film production workflow: (1) idea development transforms
brainstormed ideas into structured story outlines; (2) scriptwriting elaborates
on dialogue and character actions for each scene; (3) cinematography determines
the camera setups for each shot. A team of agents collaborates through
iterative feedback and revisions, thereby verifying intermediate scripts and
reducing hallucinations. We evaluate the generated videos on 15 ideas and 4 key
aspects. Human evaluation shows that FilmAgent outperforms all baselines across
all aspects and scores 3.98 out of 5 on average, showing the feasibility of
multi-agent collaboration in filmmaking. Further analysis reveals that
FilmAgent, despite using the less advanced GPT-4o model, surpasses the
single-agent o1, showing the advantage of a well-coordinated multi-agent
system. Lastly, we discuss the complementary strengths and weaknesses of
OpenAI's text-to-video model Sora and our FilmAgent in filmmaking.
O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning
Authors: name: Haotian Luo, name: Li Shen, name: Haiying He, name: Yibo Wang, name: Shiwei Liu, name: Wei Li, name: Naiqiang Tan, name: Xiaochun Cao, name: Dacheng Tao
Published: 2025-01-22 01:35:11

Recently, long-thought reasoning LLMs, such as OpenAI's O1, adopt extended
reasoning processes similar to how humans ponder over complex problems. This
reasoning paradigm significantly enhances the model's problem-solving abilities
and has achieved promising results. However, long-thought reasoning process
leads to a substantial increase in inference time. A pressing challenge is
reducing the inference overhead of long-thought LLMs while ensuring accuracy.
In this paper, we experimentally demonstrate that long-thought reasoning models
struggle to effectively allocate token budgets based on problem difficulty and
reasoning redundancies. To address this, we propose Length-Harmonizing
Fine-Tuning (O1-Pruner), aiming at minimizing reasoning overhead while
maintaining accuracy. This effective fine-tuning method first estimates the
LLM's baseline performance through pre-sampling and then uses RL-style
fine-tuning to encourage the model to generate shorter reasoning processes
under accuracy constraints. This allows the model to achieve efficient
reasoning with lower redundancy while maintaining accuracy. Experiments on
various mathematical reasoning benchmarks show that O1-Pruner not only
significantly reduces inference overhead but also achieves higher accuracy,
providing a novel and promising solution to this challenge. Our code is coming
soon at https://github.com/StarDewXXX/O1-Pruner
Taming Teacher Forcing for Masked Autoregressive Video Generation
Authors: name: Deyu Zhou, name: Quan Sun, name: Yuang Peng, name: Kun Yan, name: Runpei Dong, name: Duomin Wang, name: Zheng Ge, name: Nan Duan, name: Xiangyu Zhang, name: Lionel M. Ni, name: Heung-Yeung Shum
Published: 2025-01-21 18:59:31

We introduce MAGI, a hybrid video generation framework that combines masked
modeling for intra-frame generation with causal modeling for next-frame
generation. Our key innovation, Complete Teacher Forcing (CTF), conditions
masked frames on complete observation frames rather than masked ones (namely
Masked Teacher Forcing, MTF), enabling a smooth transition from token-level
(patch-level) to frame-level autoregressive generation. CTF significantly
outperforms MTF, achieving a +23% improvement in FVD scores on first-frame
conditioned video prediction. To address issues like exposure bias, we employ
targeted training strategies, setting a new benchmark in autoregressive video
generation. Experiments show that MAGI can generate long, coherent video
sequences exceeding 100 frames, even when trained on as few as 16 frames,
highlighting its potential for scalable, high-quality video generation.
Fixing Imbalanced Attention to Mitigate In-Context Hallucination of
  Large Vision-Language Model
Authors: name: Kazi Hasan Ibn Arif, name: Sajib Acharjee Dip, name: Khizar Hussain, name: Lang Zhang, name: Chris Thomas
Published: 2025-01-21 15:22:31

Large Vision Language Models (LVLMs) have demonstrated remarkable
capabilities in understanding and describing visual content, achieving
state-of-the-art performance across various vision-language tasks. However,
these models frequently exhibit hallucination behavior, where they generate
descriptions containing objects or details absent in the input image. Our work
investigates this phenomenon by analyzing attention patterns across transformer
layers and heads, revealing that hallucinations often stem from progressive
degradation of visual grounding in deeper layers. We propose a novel attention
modification approach that combines selective token emphasis and head-specific
modulation to maintain visual grounding throughout the generation process. Our
method introduces two key components: (1) a dual-stream token selection
mechanism that identifies and prioritizes both locally informative and
spatially significant visual tokens, and (2) an attention head-specific
modulation strategy that differentially amplifies visual information processing
based on measured visual sensitivity of individual attention heads. Through
extensive experimentation on the MSCOCO dataset, we demonstrate that our
approach reduces hallucination rates by up to 62.3\% compared to baseline
models while maintaining comparable task performance. Our analysis reveals that
selectively modulating tokens across attention heads with varying levels of
visual sensitivity can significantly improve visual grounding without requiring
model retraining.
The Geometry of Tokens in Internal Representations of Large Language
  Models
Authors: name: Karthik Viswanathan, name: Yuri Gardinazzi, name: Giada Panerai, name: Alberto Cazzaniga, name: Matteo Biagetti
Published: 2025-01-17 22:02:17

We investigate the relationship between the geometry of token embeddings and
their role in the next token prediction within transformer models. An important
aspect of this connection uses the notion of empirical measure, which encodes
the distribution of token point clouds across transformer layers and drives the
evolution of token representations in the mean-field interacting picture. We
use metrics such as intrinsic dimension, neighborhood overlap, and cosine
similarity to observationally probe these empirical measures across layers. To
validate our approach, we compare these metrics to a dataset where the tokens
are shuffled, which disrupts the syntactic and semantic structure. Our findings
reveal a correlation between the geometric properties of token embeddings and
the cross-entropy loss of next token predictions, implying that prompts with
higher loss values have tokens represented in higher-dimensional spaces.
Panoramic Interests: Stylistic-Content Aware Personalized Headline
  Generation
Authors: name: Junhong Lian, name: Xiang Ao, name: Xinyu Liu, name: Yang Liu, name: Qing He
Published: 2025-01-21 05:30:20

Personalized news headline generation aims to provide users with
attention-grabbing headlines that are tailored to their preferences. Prevailing
methods focus on user-oriented content preferences, but most of them overlook
the fact that diverse stylistic preferences are integral to users' panoramic
interests, leading to suboptimal personalization. In view of this, we propose a
novel Stylistic-Content Aware Personalized Headline Generation (SCAPE)
framework. SCAPE extracts both content and stylistic features from headlines
with the aid of large language model (LLM) collaboration. It further adaptively
integrates users' long- and short-term interests through a contrastive
learning-based hierarchical fusion network. By incorporating the panoramic
interests into the headline generator, SCAPE reflects users' stylistic-content
preferences during the generation process. Extensive experiments on the
real-world dataset PENS demonstrate the superiority of SCAPE over baselines.