Orionnnn/hub-stats huggingface dataset:
URSA: Understanding and Verifying Chain-of-thought Reasoning in
  Multimodal Mathematics
Authors: name: Ruilin Luo, name: Zhuofan Zheng, name: Yifan Wang, name: Yiyao Yu, name: Xinzhe Ni, name: Zicheng Lin, name: Jin Zeng, name: Yujiu Yang
Published: 2025-01-08 18:49:41

Chain-of-thought (CoT) reasoning has been widely applied in the mathematical
reasoning of Large Language Models (LLMs). Recently, the introduction of
derivative process supervision on CoT trajectories has sparked discussions on
enhancing scaling capabilities during test time, thereby boosting the potential
of these models. However, in multimodal mathematical reasoning, the scarcity of
high-quality CoT training data has hindered existing models from achieving
high-precision CoT reasoning and has limited the realization of reasoning
potential during test time. In this work, we propose a three-module synthesis
strategy that integrates CoT distillation, trajectory-format rewriting, and
format unification. It results in a high-quality CoT reasoning instruction
fine-tuning dataset in multimodal mathematics, MMathCoT-1M. We comprehensively
validate the state-of-the-art (SOTA) performance of the trained URSA-7B model
on multiple multimodal mathematical benchmarks. For test-time scaling, we
introduce a data synthesis strategy that automatically generates process
annotation datasets, known as DualMath-1.1M, focusing on both interpretation
and logic. By further training URSA-7B on DualMath-1.1M, we transition from CoT
reasoning capabilities to robust supervision abilities. The trained URSA-RM-7B
acts as a verifier, effectively enhancing the performance of URSA-7B at test
time. URSA-RM-7B also demonstrates excellent out-of-distribution (OOD)
verifying capabilities, showcasing its generalization. Model weights, training
data and code will be open-sourced.
InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning
  and Reflection
Authors: name: Yuhang Liu, name: Pengxiang Li, name: Zishu Wei, name: Congkai Xie, name: Xueyu Hu, name: Xinchen Xu, name: Shengyu Zhang, name: Xiaotian Han, name: Hongxia Yang, name: Fei Wu
Published: 2025-01-08 15:45:21

Graphical User Interface (GUI) Agents, powered by multimodal large language
models (MLLMs), have shown great potential for task automation on computing
devices such as computers and mobile phones. However, existing agents face
challenges in multi-step reasoning and reliance on textual annotations,
limiting their effectiveness. We introduce InfiGUIAgent, an MLLM-based
GUI Agent trained with a two-stage supervised fine-tuning pipeline. Stage 1
enhances fundamental skills such as GUI understanding and grounding, while
Stage 2 integrates hierarchical reasoning and expectation-reflection reasoning
skills using synthesized data to enable native reasoning abilities of the
agents. InfiGUIAgent achieves competitive performance on several GUI
benchmarks, highlighting the impact of native reasoning skills in enhancing GUI
interaction for automation tasks. Resources are available at
https://github.com/Reallm-Labs/InfiGUIAgent.
Towards System 2 Reasoning in LLMs: Learning How to Think With Meta
  Chain-of-Though
Authors: name: Violet Xiang, name: Charlie Snell, name: Kanishk Gandhi, name: Alon Albalak, name: Anikait Singh, name: Chase Blagden, name: Duy Phung, name: Rafael Rafailov, name: Nathan Lile, name: Dakota Mahan, name: Louis Castricato, name: Jan-Philipp Franken, name: Nick Haber, name: Chelsea Finn
Published: 2025-01-08 18:42:48

We propose a novel framework, Meta Chain-of-Thought (Meta-CoT), which extends
traditional Chain-of-Thought (CoT) by explicitly modeling the underlying
reasoning required to arrive at a particular CoT. We present empirical evidence
from state-of-the-art models exhibiting behaviors consistent with in-context
search, and explore methods for producing Meta-CoT via process supervision,
synthetic data generation, and search algorithms. Finally, we outline a
concrete pipeline for training a model to produce Meta-CoTs, incorporating
instruction tuning with linearized search traces and reinforcement learning
post-training. Finally, we discuss open research questions, including scaling
laws, verifier roles, and the potential for discovering novel reasoning
algorithms. This work provides a theoretical and practical roadmap to enable
Meta-CoT in LLMs, paving the way for more powerful and human-like reasoning in
artificial intelligence.
Agent Laboratory: Using LLM Agents as Research Assistants
Authors: name: Samuel Schmidgall, name: Yusheng Su, name: Ze Wang, name: Ximeng Sun, name: Jialian Wu, name: Xiaodong Yu, name: Jiang Liu, name: Zicheng Liu, name: Emad Barsoum
Published: 2025-01-08 01:58:42

Historically, scientific discovery has been a lengthy and costly process,
demanding substantial time and resources from initial conception to final
results. To accelerate scientific discovery, reduce research costs, and improve
research quality, we introduce Agent Laboratory, an autonomous LLM-based
framework capable of completing the entire research process. This framework
accepts a human-provided research idea and progresses through three
stages--literature review, experimentation, and report writing to produce
comprehensive research outputs, including a code repository and a research
report, while enabling users to provide feedback and guidance at each stage. We
deploy Agent Laboratory with various state-of-the-art LLMs and invite multiple
researchers to assess its quality by participating in a survey, providing human
feedback to guide the research process, and then evaluate the final paper. We
found that: (1) Agent Laboratory driven by o1-preview generates the best
research outcomes; (2) The generated machine learning code is able to achieve
state-of-the-art performance compared to existing methods; (3) Human
involvement, providing feedback at each stage, significantly improves the
overall quality of research; (4) Agent Laboratory significantly reduces
research expenses, achieving an 84% decrease compared to previous autonomous
research methods. We hope Agent Laboratory enables researchers to allocate more
effort toward creative ideation rather than low-level coding and writing,
ultimately accelerating scientific discovery.
rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep
  Thinking
Authors: name: Xinyu Guan, name: Li Lyna Zhang, name: Yifei Liu, name: Ning Shang, name: Youran Sun, name: Yi Zhu, name: Fan Yang, name: Mao Yang
Published: 2025-01-08 14:12:57

We present rStar-Math to demonstrate that small language models (SLMs) can
rival or even surpass the math reasoning capability of OpenAI o1, without
distillation from superior models. rStar-Math achieves this by exercising "deep
thinking" through Monte Carlo Tree Search (MCTS), where a math policy SLM
performs test-time search guided by an SLM-based process reward model.
rStar-Math introduces three innovations to tackle the challenges in training
the two SLMs: (1) a novel code-augmented CoT data sythesis method, which
performs extensive MCTS rollouts to generate step-by-step verified reasoning
trajectories used to train the policy SLM; (2) a novel process reward model
training method that avoids na\"ive step-level score annotation, yielding a
more effective process preference model (PPM); (3) a self-evolution recipe in
which the policy SLM and PPM are built from scratch and iteratively evolved to
improve reasoning capabilities. Through 4 rounds of self-evolution with
millions of synthesized solutions for 747k math problems, rStar-Math boosts
SLMs' math reasoning to state-of-the-art levels. On the MATH benchmark, it
improves Qwen2.5-Math-7B from 58.8% to 90.0% and Phi3-mini-3.8B from 41.4% to
86.4%, surpassing o1-preview by +4.5% and +0.9%. On the USA Math Olympiad
(AIME), rStar-Math solves an average of 53.3% (8/15) of problems, ranking among
the top 20% the brightest high school math students. Code and data will be
available at https://github.com/microsoft/rStar.
GeAR: Generation Augmented Retrieval
Authors: name: Haoyu Liu, name: Shaohan Huang, name: Jianfeng Liu, name: Yuefeng Zhan, name: Hao Sun, name: Weiwei Deng, name: Feng Sun, name: Furu Wei, name: Qi Zhang
Published: 2025-01-06 05:29:00

Document retrieval techniques form the foundation for the development of
large-scale information systems. The prevailing methodology is to construct a
bi-encoder and compute the semantic similarity. However, such scalar similarity
is difficult to reflect enough information and impedes our comprehension of the
retrieval results. In addition, this computational process mainly emphasizes
the global semantics and ignores the fine-grained semantic relationship between
the query and the complex text in the document. In this paper, we propose a new
method called Generation Augmented Retrieval
(GeAR) that incorporates well-designed fusion and decoding modules.
This enables GeAR to generate the relevant text from documents based on the
fused representation of the query and the document, thus learning to "focus on"
the fine-grained information. Also when used as a retriever, GeAR does not add
any computational burden over bi-encoders. To support the training of the new
framework, we have introduced a pipeline to efficiently synthesize high-quality
data by utilizing large language models. GeAR exhibits competitive retrieval
and localization performance across diverse scenarios and datasets. Moreover,
the qualitative analysis and the results generated by GeAR provide novel
insights into the interpretation of retrieval results. The code, data, and
models will be released after completing technical review to facilitate future
research.
OpenOmni: Large Language Models Pivot Zero-shot Omnimodal Alignment
  across Language with Real-time Self-Aware Emotional Speech Synthesis
Authors: name: Run Luo, name: Ting-En Lin, name: Haonan Zhang, name: Yuchuan Wu, name: Xiong Liu, name: Min Yang, name: Yongbin Li, name: Longze Chen, name: Jiaming Li, name: Lei Zhang, name: Yangyi Chen, name: Hamid Alinejad-Rokny, name: Fei Huang
Published: 2025-01-08 15:18:09

Recent advancements in omnimodal learning have been achieved in understanding
and generation across images, text, and speech, though mainly within
proprietary models. Limited omnimodal datasets and the inherent challenges
associated with real-time emotional speech generation have hindered open-source
progress. To address these issues, we propose openomni, a two-stage training
method combining omnimodal alignment and speech generation to develop a
state-of-the-art omnimodal large language model. In the alignment phase, a
pre-trained speech model is further trained on text-image tasks to generalize
from vision to speech in a (near) zero-shot manner, outperforming models
trained on tri-modal datasets. In the speech generation phase, a lightweight
decoder facilitates real-time emotional speech through training on speech tasks
and preference learning. Experiments demonstrate that openomni consistently
improves across omnimodal, vision-language, and speech-language evaluations,
enabling natural, emotion-rich dialogues and real-time emotional speech
generation.
Generalizable Origin Identification for Text-Guided Image-to-Image
  Diffusion Models
Authors: name: Wenhao Wang, name: Yifan Sun, name: Zongxin Yang, name: Zhentao Tan, name: Zhengdong Hu, name: Yi Yang
Published: 2025-01-04 20:34:53

Text-guided image-to-image diffusion models excel in translating images based
on textual prompts, allowing for precise and creative visual modifications.
However, such a powerful technique can be misused for spreading misinformation,
infringing on copyrights, and evading content tracing. This motivates us to
introduce the task of origin IDentification for text-guided Image-to-image
Diffusion models (ID^2), aiming to retrieve the original image of a given
translated query. A straightforward solution to ID^2 involves training a
specialized deep embedding model to extract and compare features from both
query and reference images. However, due to visual discrepancy across
generations produced by different diffusion models, this similarity-based
approach fails when training on images from one model and testing on those from
another, limiting its effectiveness in real-world applications. To solve this
challenge of the proposed ID^2 task, we contribute the first dataset and a
theoretically guaranteed method, both emphasizing generalizability. The curated
dataset, OriPID, contains abundant Origins and guided Prompts, which can be
used to train and test potential IDentification models across various diffusion
models. In the method section, we first prove the existence of a linear
transformation that minimizes the distance between the pre-trained Variational
Autoencoder (VAE) embeddings of generated samples and their origins.
Subsequently, it is demonstrated that such a simple linear transformation can
be generalized across different diffusion models. Experimental results show
that the proposed method achieves satisfying generalization performance,
significantly surpassing similarity-based methods (+31.6% mAP), even those
with generalization designs.
Graph-Aware Isomorphic Attention for Adaptive Dynamics in Transformers
Authors: name: Markus J. Buehler
Published: 2025-01-04 22:30:21

We present an approach to modifying Transformer architectures by integrating
graph-aware relational reasoning into the attention mechanism, merging concepts
from graph neural networks and language modeling. Building on the inherent
connection between attention and graph theory, we reformulate the Transformer's
attention mechanism as a graph operation and propose Graph-Aware Isomorphic
Attention. This method leverages advanced graph modeling strategies, including
Graph Isomorphism Networks (GIN) and Principal Neighborhood Aggregation (PNA),
to enrich the representation of relational structures. Our approach captures
complex dependencies and generalizes across tasks, as evidenced by a reduced
generalization gap and improved learning performance. Additionally, we expand
the concept of graph-aware attention to introduce Sparse GIN-Attention, a
fine-tuning approach that employs sparse GINs. By interpreting attention
matrices as sparse adjacency graphs, this technique enhances the adaptability
of pre-trained foundational models with minimal computational overhead,
endowing them with graph-aware capabilities. Sparse GIN-Attention fine-tuning
achieves improved training dynamics and better generalization compared to
alternative methods like low-rank adaption (LoRA). We discuss latent graph-like
structures within traditional attention mechanisms, offering a new lens through
which Transformers can be understood. By evolving Transformers as hierarchical
GIN models for relational reasoning. This perspective suggests profound
implications for foundational model development, enabling the design of
architectures that dynamically adapt to both local and global dependencies.
Applications in bioinformatics, materials science, language modeling, and
beyond could benefit from this synthesis of relational and sequential data
modeling, setting the stage for interpretable and generalizable modeling
strategies.
Segmenting Text and Learning Their Rewards for Improved RLHF in Language
  Model
Authors: name: Yueqin Yin, name: Shentao Yang, name: Yujia Xie, name: Ziyi Yang, name: Yuting Sun, name: Hany Awadalla, name: Weizhu Chen, name: Mingyuan Zhou
Published: 2025-01-06 06:17:56

Reinforcement learning from human feedback (RLHF) has been widely adopted to
align language models (LMs) with human preference. Prior RLHF works typically
take a bandit formulation, which, though intuitive, ignores the sequential
nature of LM generation and can suffer from the sparse reward issue. While
recent works propose dense token-level RLHF, treating each token as an action
may be oversubtle to proper reward assignment. In this paper, we seek to get
the best of both by training and utilizing a segment-level reward model, which
assigns a reward to each semantically complete text segment that spans over a
short sequence of tokens. For reward learning, our method allows dynamic text
segmentation and compatibility with standard sequence-preference datasets. For
effective RL-based LM training against segment reward, we generalize the
classical scalar bandit reward normalizers into location-aware normalizer
functions and interpolate the segment reward for further densification. With
these designs, our method performs competitively on three popular RLHF
benchmarks for LM policy: AlpacaEval 2.0, Arena-Hard, and MT-Bench. Ablation
studies are conducted to further demonstrate our method.
PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slides
Authors: name: Hao Zheng, name: Xinyan Guan, name: Hao Kong, name: Jia Zheng, name: Hongyu Lin, name: Yaojie Lu, name: Ben He, name: Xianpei Han, name: Le Sun
Published: 2025-01-07 16:53:01

Automatically generating presentations from documents is a challenging task
that requires balancing content quality, visual design, and structural
coherence. Existing methods primarily focus on improving and evaluating the
content quality in isolation, often overlooking visual design and structural
coherence, which limits their practical applicability. To address these
limitations, we propose PPTAgent, which comprehensively improves presentation
generation through a two-stage, edit-based approach inspired by human
workflows. PPTAgent first analyzes reference presentations to understand their
structural patterns and content schemas, then drafts outlines and generates
slides through code actions to ensure consistency and alignment. To
comprehensively evaluate the quality of generated presentations, we further
introduce PPTEval, an evaluation framework that assesses presentations across
three dimensions: Content, Design, and Coherence. Experiments show that
PPTAgent significantly outperforms traditional automatic presentation
generation methods across all three dimensions. The code and data are available
at https://github.com/icip-cas/PPTAgent.
MoDec-GS: Global-to-Local Motion Decomposition and Temporal Interval
  Adjustment for Compact Dynamic 3D Gaussian Splatting
Authors: name: Sangwoon Kwak, name: Joonsoo Kim, name: Jun Young Jeong, name: Won-Sik Cheong, name: Jihyong Oh, name: Munchurl Kim
Published: 2025-01-07 11:43:13

3D Gaussian Splatting (3DGS) has made significant strides in scene
representation and neural rendering, with intense efforts focused on adapting
it for dynamic scenes. Despite delivering remarkable rendering quality and
speed, existing methods struggle with storage demands and representing complex
real-world motions. To tackle these issues, we propose MoDecGS, a
memory-efficient Gaussian splatting framework designed for reconstructing novel
views in challenging scenarios with complex motions. We introduce GlobaltoLocal
Motion Decomposition (GLMD) to effectively capture dynamic motions in a
coarsetofine manner. This approach leverages Global Canonical Scaffolds (Global
CS) and Local Canonical Scaffolds (Local CS), extending static Scaffold
representation to dynamic video reconstruction. For Global CS, we propose
Global Anchor Deformation (GAD) to efficiently represent global dynamics along
complex motions, by directly deforming the implicit Scaffold attributes which
are anchor position, offset, and local context features. Next, we finely adjust
local motions via the Local Gaussian Deformation (LGD) of Local CS explicitly.
Additionally, we introduce Temporal Interval Adjustment (TIA) to automatically
control the temporal coverage of each Local CS during training, allowing
MoDecGS to find optimal interval assignments based on the specified number of
temporal segments. Extensive evaluations demonstrate that MoDecGS achieves an
average 70% reduction in model size over stateoftheart methods for dynamic 3D
Gaussians from realworld dynamic videos while maintaining or even improving
rendering quality.
Dolphin: Closed-loop Open-ended Auto-research through Thinking,
  Practice, and Feedback
Authors: name: Jiakang Yuan, name: Xiangchao Yan, name: Botian Shi, name: Tao Chen, name: Wanli Ouyang, name: Bo Zhang, name: Lei Bai, name: Yu Qiao, name: Bowen Zhou
Published: 2025-01-07 16:31:10

The scientific research paradigm is undergoing a profound transformation
owing to the development of Artificial Intelligence (AI). Recent works
demonstrate that various AI-assisted research methods can largely improve
research efficiency by improving data analysis, accelerating computation, and
fostering novel idea generation. To further move towards the ultimate goal
(i.e., automatic scientific research), in this paper, we propose Dolphin, the
first closed-loop open-ended auto-research framework to further build the
entire process of human scientific research. Dolphin can generate research
ideas, perform experiments, and get feedback from experimental results to
generate higher-quality ideas. More specifically, Dolphin first generates novel
ideas based on relevant papers which are ranked by the topic and task
attributes. Then, the codes are automatically generated and debugged with the
exception-traceback-guided local code structure. Finally, Dolphin automatically
analyzes the results of each idea and feeds the results back to the next round
of idea generation. Experiments are conducted on the benchmark datasets of
different topics and results show that Dolphin can generate novel ideas
continuously and complete the experiment in a loop. We highlight that Dolphin
can automatically propose methods that are comparable to the state-of-the-art
in some tasks such as 2D image classification and 3D point classification.
Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of
  Images and Videos
Authors: name: Haobo Yuan, name: Xiangtai Li, name: Tao Zhang, name: Zilong Huang, name: Shilin Xu, name: Shunping Ji, name: Yunhai Tong, name: Lu Qi, name: Jiashi Feng, name: Ming-Hsuan Yang
Published: 2025-01-07 18:58:54

This work presents Sa2VA, the first unified model for dense grounded
understanding of both images and videos. Unlike existing multi-modal large
language models, which are often limited to specific modalities and tasks,
Sa2VA supports a wide range of image and video tasks, including referring
segmentation and conversation, with minimal one-shot instruction tuning. Sa2VA
combines SAM-2, a foundation video segmentation model, with LLaVA, an advanced
vision-language model, and unifies text, image, and video into a shared LLM
token space. Using the LLM, Sa2VA generates instruction tokens that guide SAM-2
in producing precise masks, enabling a grounded, multi-modal understanding of
both static and dynamic visual content. Additionally, we introduce Ref-SAV, an
auto-labeled dataset containing over 72k object expressions in complex video
scenes, designed to boost model performance. We also manually validate 2k video
objects in the Ref-SAV datasets to benchmark referring video object
segmentation in complex environments. Experiments show that Sa2VA achieves
state-of-the-art across multiple tasks, particularly in referring video object
segmentation, highlighting its potential for complex real-world applications.
REINFORCE++: A Simple and Efficient Approach for Aligning Large Language
  Models
Authors: name: Jian Hu
Published: 2025-01-04 02:08:06

Reinforcement Learning from Human Feedback (RLHF) has emerged as a critical
approach for aligning large language models with human preferences, witnessing
rapid algorithmic evolution through methods such as Proximal Policy
Optimization (PPO), Direct Preference Optimization (DPO), REINFORCE Leave
One-Out (RLOO), ReMax, and Group Relative Policy Optimization (GRPO). We
present REINFORCE++, an enhanced variant of the classical REINFORCE algorithm
that incorporates key optimization techniques from PPO while eliminating the
need for a critic network. REINFORCE++ achieves three primary objectives: (1)
simplicity (2) enhanced training stability, and (3) reduced computational
overhead. Through extensive empirical evaluation, we demonstrate that
REINFORCE++ exhibits superior stability compared to GRPO and achieves greater
computational efficiency than PPO while maintaining comparable performance. The
implementation is available at https://github.com/OpenRLHF/OpenRLHF.