Orionnnn/hub-stats huggingface dataset:
V-MAGE: A Game Evaluation Framework for Assessing Visual-Centric
  Capabilities in Multimodal Large Language Models
Authors: name: Xiangxi Zheng, name: Linjie Li, name: Zhengyuan Yang, name: Ping Yu, name: Alex Jinpeng Wang, name: Rui Yan, name: Yuan Yao, name: Lijuan Wang
Published: 2025-04-08 15:43:01

Recent advancements in Multimodal Large Language Models (MLLMs) have led to
significant improvements across various multimodal benchmarks. However, as
evaluations shift from static datasets to open-world, dynamic environments,
current game-based benchmarks remain inadequate because they lack
visual-centric tasks and fail to assess the diverse reasoning skills required
for real-world decision-making. To address this, we introduce Visual-centric
Multiple Abilities Game Evaluation (V-MAGE), a game-based evaluation framework
designed to assess visual reasoning capabilities of MLLMs. V-MAGE features five
diverse games with 30+ handcrafted levels, testing models on core visual skills
such as positioning, trajectory tracking, timing, and visual memory, alongside
higher-level reasoning like long-term planning and deliberation. We use V-MAGE
to evaluate leading MLLMs, revealing significant challenges in their visual
perception and reasoning. In all game environments, the top-performing MLLMs,
as determined by Elo rating comparisons, exhibit a substantial performance gap
compared to humans. Our findings highlight critical limitations, including
various types of perceptual errors made by the models, and suggest potential
avenues for improvement from an agent-centric perspective, such as refining
agent strategies and addressing perceptual inaccuracies. Code is available at
https://github.com/CSU-JPG/V-MAGE.
Accelerate Parallelizable Reasoning via Parallel Decoding within One
  Sequence
Authors: name: Yijiong Yu
Published: 2025-03-26 13:28:57

Recent advances in reasoning models have demonstrated significant
improvements in accuracy, particularly for complex tasks such as mathematical
reasoning, by employing detailed and comprehensive reasoning processes.
However, generating these lengthy reasoning sequences is computationally
expensive and time-consuming. To address this inefficiency, we leverage the
inherent parallelizability of certain tasks to accelerate the reasoning
process. Specifically, when multiple parallel reasoning branches exist, we
decode multiple tokens per step using a specialized attention mask, processing
them within a single sequence, avoiding additional memory usage. Experimental
results show that our method achieves over 100% speedup in decoding time while
maintaining the answer quality.
Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought
Authors: name: Yi Peng, name: Chris, name: Xiaokun Wang, name: Yichen Wei, name: Jiangbo Pei, name: Weijie Qiu, name: Ai Jian, name: Yunzhuo Hao, name: Jiachun Pan, name: Tianyidan Xie, name: Li Ge, name: Rongxian Zhuang, name: Xuchen Song, name: Yang Liu, name: Yahui Zhou
Published: 2025-04-08 01:19:20

We introduce Skywork R1V, a multimodal reasoning model extending the an
R1-series Large language models (LLM) to visual modalities via an efficient
multimodal transfer method. Leveraging a lightweight visual projector, Skywork
R1V facilitates seamless multimodal adaptation without necessitating retraining
of either the foundational language model or the vision encoder. To strengthen
visual-text alignment, we propose a hybrid optimization strategy that combines
Iterative Supervised Fine-Tuning (SFT) with Group Relative Policy Optimization
(GRPO), significantly enhancing cross-modal integration efficiency.
Additionally, we introduce an adaptive-length Chain-of-Thought distillation
approach for reasoning data generation. This approach dynamically optimizes
reasoning chain lengths, thereby enhancing inference efficiency and preventing
excessive reasoning overthinking. Empirical evaluations demonstrate that
Skywork R1V, with only 38B parameters, delivers competitive performance,
achieving a score of 69.0 on the MMMU benchmark and 67.5 on MathVista.
Meanwhile, it maintains robust textual reasoning performance, evidenced by
impressive scores of 72.0 on AIME and 94.0 on MATH500. The Skywork R1V model
weights have been publicly released to promote openness and reproducibility.
Hogwild! Inference: Parallel LLM Generation via Concurrent Attention
Authors: name: Gleb Rodionov, name: Roman Garipov, name: Alina Shutova, name: George Yakushev, name: Vage Egiazarian, name: Anton Sinitsin, name: Denis Kuznedelev, name: Dan Alistarh
Published: 2025-04-08 17:59:41

Large Language Models (LLMs) have demonstrated the ability to tackle
increasingly complex tasks through advanced reasoning, long-form content
generation, and tool use. Solving these tasks often involves long
inference-time computations. In human problem solving, a common strategy to
expedite work is collaboration: by dividing the problem into sub-tasks,
exploring different strategies concurrently, etc. Recent research has shown
that LLMs can also operate in parallel by implementing explicit cooperation
frameworks, such as voting mechanisms or the explicit creation of independent
sub-tasks that can be executed in parallel. However, each of these frameworks
may not be suitable for all types of tasks, which can hinder their
applicability. In this work, we propose a different design approach: we run LLM
"workers" in parallel , allowing them to synchronize via a concurrently-updated
attention cache and prompt these workers to decide how best to collaborate. Our
approach allows the instances to come up with their own collaboration strategy
for the problem at hand, all the while "seeing" each other's partial progress
in the concurrent cache. We implement this approach via Hogwild! Inference: a
parallel LLM inference engine where multiple instances of the same LLM run in
parallel with the same attention cache, with "instant" access to each other's
generated tokens. Hogwild! inference takes advantage of Rotary Position
Embeddings (RoPE) to avoid recomputation while improving parallel hardware
utilization. We find that modern reasoning-capable LLMs can perform inference
with shared Key-Value cache out of the box, without additional fine-tuning.
Less-to-More Generalization: Unlocking More Controllability by
  In-Context Generation
Authors: name: Shaojin Wu, name: Mengqi Huang, name: Wenxu Wu, name: Yufeng Cheng, name: Fei Ding, name: Qian He
Published: 2025-04-02 22:20:21

Although subject-driven generation has been extensively explored in image
generation due to its wide applications, it still has challenges in data
scalability and subject expansibility. For the first challenge, moving from
curating single-subject datasets to multiple-subject ones and scaling them is
particularly difficult. For the second, most recent methods center on
single-subject generation, making it hard to apply when dealing with
multi-subject scenarios. In this study, we propose a highly-consistent data
synthesis pipeline to tackle this challenge. This pipeline harnesses the
intrinsic in-context generation capabilities of diffusion transformers and
generates high-consistency multi-subject paired data. Additionally, we
introduce UNO, which consists of progressive cross-modal alignment and
universal rotary position embedding. It is a multi-image conditioned
subject-to-image model iteratively trained from a text-to-image model.
Extensive experiments show that our method can achieve high consistency while
ensuring controllability in both single-subject and multi-subject driven
generation.
OmniSVG: A Unified Scalable Vector Graphics Generation Model
Authors: name: Yiying Yang, name: Wei Cheng, name: Sijin Chen, name: Xianfang Zeng, name: Jiaxu Zhang, name: Liao Wang, name: Gang Yu, name: Xingjun Ma, name: Yu-Gang Jiang
Published: 2025-04-08 17:59:49

Scalable Vector Graphics (SVG) is an important image format widely adopted in
graphic design because of their resolution independence and editability. The
study of generating high-quality SVG has continuously drawn attention from both
designers and researchers in the AIGC community. However, existing methods
either produces unstructured outputs with huge computational cost or is limited
to generating monochrome icons of over-simplified structures. To produce
high-quality and complex SVG, we propose OmniSVG, a unified framework that
leverages pre-trained Vision-Language Models (VLMs) for end-to-end multimodal
SVG generation. By parameterizing SVG commands and coordinates into discrete
tokens, OmniSVG decouples structural logic from low-level geometry for
efficient training while maintaining the expressiveness of complex SVG
structure. To further advance the development of SVG synthesis, we introduce
MMSVG-2M, a multimodal dataset with two million richly annotated SVG assets,
along with a standardized evaluation protocol for conditional SVG generation
tasks. Extensive experiments show that OmniSVG outperforms existing methods and
demonstrates its potential for integration into professional SVG design
workflows.
Generative Evaluation of Complex Reasoning in Large Language Models
Authors: name: Haowei Lin, name: Xiangyu Wang, name: Ruilin Yan, name: Baizhou Huang, name: Haotian Ye, name: Jianhua Zhu, name: Zihao Wang, name: James Zou, name: Jianzhu Ma, name: Yitao Liang
Published: 2025-04-03 17:54:18

With powerful large language models (LLMs) demonstrating superhuman reasoning
capabilities, a critical question arises: Do LLMs genuinely reason, or do they
merely recall answers from their extensive, web-scraped training datasets?
Publicly released benchmarks inevitably become contaminated once incorporated
into subsequent LLM training sets, undermining their reliability as faithful
assessments. To address this, we introduce KUMO, a generative evaluation
framework designed specifically for assessing reasoning in LLMs. KUMO
synergistically combines LLMs with symbolic engines to dynamically produce
diverse, multi-turn reasoning tasks that are partially observable and
adjustable in difficulty. Through an automated pipeline, KUMO continuously
generates novel tasks across open-ended domains, compelling models to
demonstrate genuine generalization rather than memorization. We evaluated 23
state-of-the-art LLMs on 5,000 tasks across 100 domains created by KUMO,
benchmarking their reasoning abilities against university students. Our
findings reveal that many LLMs have outperformed university-level performance
on easy reasoning tasks, and reasoning-scaled LLMs reach university-level
performance on complex reasoning challenges. Moreover, LLM performance on KUMO
tasks correlates strongly with results on newly released real-world reasoning
benchmarks, underscoring KUMO's value as a robust, enduring assessment tool for
genuine LLM reasoning capabilities.
Tuning-Free Image Editing with Fidelity and Editability via Unified
  Latent Diffusion Model
Authors: name: Qi Mao, name: Lan Chen, name: Yuchao Gu, name: Mike Zheng Shou, name: Ming-Hsuan Yang
Published: 2025-04-08 01:02:50

Balancing fidelity and editability is essential in text-based image editing
(TIE), where failures commonly lead to over- or under-editing issues. Existing
methods typically rely on attention injections for structure preservation and
leverage the inherent text alignment capabilities of pre-trained text-to-image
(T2I) models for editability, but they lack explicit and unified mechanisms to
properly balance these two objectives. In this work, we introduce UnifyEdit, a
tuning-free method that performs diffusion latent optimization to enable a
balanced integration of fidelity and editability within a unified framework.
Unlike direct attention injections, we develop two attention-based constraints:
a self-attention (SA) preservation constraint for structural fidelity, and a
cross-attention (CA) alignment constraint to enhance text alignment for
improved editability. However, simultaneously applying both constraints can
lead to gradient conflicts, where the dominance of one constraint results in
over- or under-editing. To address this challenge, we introduce an adaptive
time-step scheduler that dynamically adjusts the influence of these
constraints, guiding the diffusion latent toward an optimal balance. Extensive
quantitative and qualitative experiments validate the effectiveness of our
approach, demonstrating its superiority in achieving a robust balance between
structure preservation and text alignment across various editing tasks,
outperforming other state-of-the-art methods. The source code will be available
at https://github.com/CUC-MIPG/UnifyEdit.
CrossWordBench: Evaluating the Reasoning Capabilities of LLMs and LVLMs
  with Controllable Puzzle Generation
Authors: name: Jixuan Leng, name: Chengsong Huang, name: Langlin Huang, name: Bill Yuchen Lin, name: William W. Cohen, name: Haohan Wang, name: Jiaxin Huang
Published: 2025-03-30 20:03:36

Existing reasoning evaluation frameworks for Large Language Models (LLMs) and
Large Vision-Language Models (LVLMs) predominantly either assess text-based
reasoning or vision-language understanding capabilities, with limited dynamic
interplay between textual and visual constraints. To address this limitation,
we introduce CrossWordBench, a benchmark designed to evaluate the reasoning
capabilities of both LLMs and LVLMs through the medium of crossword puzzles-a
task requiring multimodal adherence to semantic constraints from text-based
clues and intersectional constraints from visual grid structures.
CrossWordBench leverages a controllable puzzle generation framework that
produces puzzles in multiple formats (text and image) and offers different
evaluation strategies ranging from direct puzzle solving to interactive modes.
Our extensive evaluation of over 20 models reveals that reasoning LLMs
outperform non-reasoning models substantially by effectively leveraging
crossing-letter constraints. We further demonstrate that LVLMs struggle with
the task, showing a strong correlation between their puzzle-solving performance
and grid-parsing accuracy. Our findings offer insights into the limitations of
the reasoning capabilities of current LLMs and LVLMs, and provide an effective
approach for creating multimodal constrained tasks for future evaluations.
An Empirical Study of GPT-4o Image Generation Capabilities
Authors: name: Sixiang Chen, name: Jinbin Bai, name: Zhuoran Zhao, name: Tian Ye, name: Qingyu Shi, name: Donghao Zhou, name: Wenhao Chai, name: Xin Lin, name: Jianzong Wu, name: Chao Tang, name: Shilin Xu, name: Tao Zhang, name: Haobo Yuan, name: Yikang Zhou, name: Wei Chow, name: Linfeng Li, name: Xiangtai Li, name: Lei Zhu, name: Lu Qi
Published: 2025-04-08 12:34:36

The landscape of image generation has rapidly evolved, from early GAN-based
approaches to diffusion models and, most recently, to unified generative
architectures that seek to bridge understanding and generation tasks. Recent
advances, especially the GPT-4o, have demonstrated the feasibility of
high-fidelity multimodal generation, their architectural design remains
mysterious and unpublished. This prompts the question of whether image and text
generation have already been successfully integrated into a unified framework
for those methods. In this work, we conduct an empirical study of GPT-4o's
image generation capabilities, benchmarking it against leading open-source and
commercial models. Our evaluation covers four main categories, including
text-to-image, image-to-image, image-to-3D, and image-to-X generation, with
more than 20 tasks. Our analysis highlights the strengths and limitations of
GPT-4o under various settings, and situates it within the broader evolution of
generative modeling. Through this investigation, we identify promising
directions for future unified generative models, emphasizing the role of
architectural design and data scaling.
3D Scene Understanding Through Local Random Access Sequence Modeling
Authors: name: Wanhee Lee, name: Klemen Kotar, name: Rahul Mysore Venkatesh, name: Jared Watrous, name: Honglin Chen, name: Khai Loong Aw, name: Daniel L. K. Yamins
Published: 2025-04-04 18:59:41

3D scene understanding from single images is a pivotal problem in computer
vision with numerous downstream applications in graphics, augmented reality,
and robotics. While diffusion-based modeling approaches have shown promise,
they often struggle to maintain object and scene consistency, especially in
complex real-world scenarios. To address these limitations, we propose an
autoregressive generative approach called Local Random Access Sequence (LRAS)
modeling, which uses local patch quantization and randomly ordered sequence
generation. By utilizing optical flow as an intermediate representation for 3D
scene editing, our experiments demonstrate that LRAS achieves state-of-the-art
novel view synthesis and 3D object manipulation capabilities. Furthermore, we
show that our framework naturally extends to self-supervised depth estimation
through a simple modification of the sequence design. By achieving strong
performance on multiple 3D scene understanding tasks, LRAS provides a unified
and effective framework for building the next generation of 3D vision models.
Distillation and Refinement of Reasoning in Small Language Models for
  Document Re-ranking
Authors: name: Chris Samarinas, name: Hamed Zamani
Published: 2025-04-04 21:27:48

We present a novel approach for training small language models for
reasoning-intensive document ranking that combines knowledge distillation with
reinforcement learning optimization. While existing methods often rely on
expensive human annotations or large black-box language models, our methodology
leverages web data and a teacher LLM to automatically generate high-quality
training examples with relevance explanations. By framing document ranking as a
reinforcement learning problem and incentivizing explicit reasoning
capabilities, we train a compact 3B parameter language model that achieves
state-of-the-art performance on the BRIGHT benchmark. Our model ranks third on
the leaderboard while using substantially fewer parameters than other
approaches, outperforming models that are over 20 times larger. Through
extensive experiments, we demonstrate that generating explanations during
inference, rather than directly predicting relevance scores, enables more
effective reasoning with smaller language models. The self-supervised nature of
our method offers a scalable and interpretable solution for modern information
retrieval systems.
Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning
  (v1)
Authors: name: Jing Bi, name: Susan Liang, name: Xiaofei Zhou, name: Pinxin Liu, name: Junjia Guo, name: Yunlong Tang, name: Luchuan Song, name: Chao Huang, name: Guangyu Sun, name: Jinxi He, name: Jiarui Wu, name: Shu Yang, name: Daoan Zhang, name: Chen Chen, name: Lianggong Bruce Wen, name: Zhang Liu, name: Jiebo Luo, name: Chenliang Xu
Published: 2025-04-04 04:04:56

Reasoning is central to human intelligence, enabling structured
problem-solving across diverse tasks. Recent advances in large language models
(LLMs) have greatly enhanced their reasoning abilities in arithmetic,
commonsense, and symbolic domains. However, effectively extending these
capabilities into multimodal contexts-where models must integrate both visual
and textual inputs-continues to be a significant challenge. Multimodal
reasoning introduces complexities, such as handling conflicting information
across modalities, which require models to adopt advanced interpretative
strategies. Addressing these challenges involves not only sophisticated
algorithms but also robust methodologies for evaluating reasoning accuracy and
coherence. This paper offers a concise yet insightful overview of reasoning
techniques in both textual and multimodal LLMs. Through a thorough and
up-to-date comparison, we clearly formulate core reasoning challenges and
opportunities, highlighting practical methods for post-training optimization
and test-time inference. Our work provides valuable insights and guidance,
bridging theoretical frameworks and practical implementations, and sets clear
directions for future research.
Sample, Don't Search: Rethinking Test-Time Alignment for Language Models
Authors: name: Gonçalo Faria, name: Noah A. Smith
Published: 2025-04-04 00:41:40

Increasing test-time computation has emerged as a promising direction for
improving language model performance, particularly in scenarios where model
finetuning is impractical or impossible due to computational constraints or
private model weights. However, existing test-time search methods using a
reward model (RM) often degrade in quality as compute scales, due to the
over-optimization of what are inherently imperfect reward proxies. We introduce
QAlign, a new test-time alignment approach. As we scale test-time compute,
QAlign converges to sampling from the optimal aligned distribution for each
individual prompt. By adopting recent advances in Markov chain Monte Carlo for
text generation, our method enables better-aligned outputs without modifying
the underlying model or even requiring logit access. We demonstrate the
effectiveness of QAlign on mathematical reasoning benchmarks (GSM8K and
GSM-Symbolic) using a task-specific RM, showing consistent improvements over
existing test-time compute methods like best-of-n and majority voting.
Furthermore, when applied with more realistic RMs trained on the Tulu 3
preference dataset, QAlign outperforms direct preference optimization (DPO),
best-of-n, majority voting, and weighted majority voting on a diverse range of
datasets (GSM8K, MATH500, IFEval, MMLU-Redux, and TruthfulQA). A practical
solution to aligning language models at test time using additional computation
without degradation, our approach expands the limits of the capability that can
be obtained from off-the-shelf language models without further training.
Rethinking Reflection in Pre-Training
Authors: name: Essential AI, name: Darsh J Shah, name: Peter Rushton, name: Somanshu Singla, name: Mohit Parmar, name: Kurt Smith, name: Yash Vanjani, name: Ashish Vaswani, name: Adarsh Chaluvaraju, name: Andrew Hojel, name: Andrew Ma, name: Anil Thomas, name: Anthony Polloreno, name: Ashish Tanwer, name: Burhan Drak Sibai, name: Divya S Mansingka, name: Divya Shivaprasad, name: Ishaan Shah, name: Karl Stratos, name: Khoi Nguyen, name: Michael Callahan, name: Michael Pust, name: Mrinal Iyer, name: Philip Monk, name: Platon Mazarakis, name: Ritvik Kapila, name: Saurabh Srivastava, name: Tim Romanski
Published: 2025-04-05 02:24:07

A language model's ability to reflect on its own reasoning provides a key
advantage for solving complex problems. While most recent research has focused
on how this ability develops during reinforcement learning, we show that it
actually begins to emerge much earlier - during the model's pre-training. To
study this, we introduce deliberate errors into chains-of-thought and test
whether the model can still arrive at the correct answer by recognizing and
correcting these mistakes. By tracking performance across different stages of
pre-training, we observe that this self-correcting ability appears early and
improves steadily over time. For instance, an OLMo2-7B model pre-trained on 4
trillion tokens displays self-correction on our six self-reflection tasks.