Orionnnn/hub-stats huggingface dataset:
SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation
Authors: name: Siqi Chen, name: Xinyu Dong, name: Haolei Xu, name: Xingyu Wu, name: Fei Tang, name: Hang Zhang, name: Yuchen Yan, name: Linjuan Wu, name: Wenqi Zhang, name: Guiyang Hou, name: Yongliang Shen, name: Weiming Lu, name: Yueting Zhuang
Published: 2025-06-03 17:58:57

Large Language Models (LLMs) and Multimodal LLMs have shown promising
capabilities for SVG processing, yet existing benchmarks suffer from limited
real-world coverage, lack of complexity stratification, and fragmented
evaluation paradigms. We introduce SVGenius, a comprehensive benchmark
comprising 2,377 queries across three progressive dimensions: understanding,
editing, and generation. Built on real-world data from 24 application domains
with systematic complexity stratification, SVGenius evaluates models through 8
task categories and 18 metrics. We assess 22 mainstream models spanning
different scales, architectures, training paradigms, and accessibility levels.
Our analysis reveals that while proprietary models significantly outperform
open-source counterparts, all models exhibit systematic performance degradation
with increasing complexity, indicating fundamental limitations in current
approaches; however, reasoning-enhanced training proves more effective than
pure scaling for overcoming these limitations, though style transfer remains
the most challenging capability across all model types. SVGenius establishes
the first systematic evaluation framework for SVG processing, providing crucial
insights for developing more capable vector graphics models and advancing
automated graphic design applications. Appendix and supplementary materials
(including all data and code) are available at
https://zju-real.github.io/SVGenius.
Ψ-Sampler: Initial Particle Sampling for SMC-Based Inference-Time
  Reward Alignment in Score Models
Authors: name: Taehoon Yoon, name: Yunhong Min, name: Kyeongmin Yeo, name: Minhyuk Sung
Published: 2025-06-02 05:02:33

We introduce Psi-Sampler, an SMC-based framework incorporating pCNL-based
initial particle sampling for effective inference-time reward alignment with a
score-based generative model. Inference-time reward alignment with score-based
generative models has recently gained significant traction, following a broader
paradigm shift from pre-training to post-training optimization. At the core of
this trend is the application of Sequential Monte Carlo (SMC) to the denoising
process. However, existing methods typically initialize particles from the
Gaussian prior, which inadequately captures reward-relevant regions and results
in reduced sampling efficiency. We demonstrate that initializing from the
reward-aware posterior significantly improves alignment performance. To enable
posterior sampling in high-dimensional latent spaces, we introduce the
preconditioned Crank-Nicolson Langevin (pCNL) algorithm, which combines
dimension-robust proposals with gradient-informed dynamics. This approach
enables efficient and scalable posterior sampling and consistently improves
performance across various reward alignment tasks, including layout-to-image
generation, quantity-aware generation, and aesthetic-preference generation, as
demonstrated in our experiments.
Unleashing the Reasoning Potential of Pre-trained LLMs by Critique
  Fine-Tuning on One Problem
Authors: name: Yubo Wang, name: Ping Nie, name: Kai Zou, name: Lijun Wu, name: Wenhu Chen
Published: 2025-06-03 18:35:52

We have witnessed that strong LLMs like Qwen-Math, MiMo, and Phi-4 possess
immense reasoning potential inherited from the pre-training stage. With
reinforcement learning (RL), these models can improve dramatically on reasoning
tasks. Recent studies have shown that even RL on a single problem can unleash
these models' reasoning capabilities. However, RL is not only expensive but
also unstable. Even one-shot RL requires hundreds of GPU hours. This raises a
critical question: Is there a more efficient way to unleash the reasoning
potential of these powerful base LLMs? In this work, we demonstrate that
Critique Fine-Tuning (CFT) on only one problem can effectively unleash the
reasoning potential of LLMs. Our method constructs critique data by collecting
diverse model-generated solutions to a single problem and using teacher LLMs to
provide detailed critiques. We fine-tune Qwen and Llama family models, ranging
from 1.5B to 14B parameters, on the CFT data and observe significant
performance gains across diverse reasoning tasks. For example, with just 5 GPU
hours of training, Qwen-Math-7B-CFT show an average improvement of 15% on six
math benchmarks and 16% on three logic reasoning benchmarks. These results are
comparable to or even surpass the results from RL with 20x less compute.
Ablation studies reveal the robustness of one-shot CFT across different prompt
problems. These results highlight one-shot CFT as a simple, general, and
compute-efficient approach to unleashing the reasoning capabilities of modern
LLMs.
Advancing Multimodal Reasoning: From Optimized Cold Start to Staged
  Reinforcement Learning
Authors: name: Shuang Chen, name: Yue Guo, name: Zhaochen Su, name: Yafu Li, name: Yulun Wu, name: Jiacheng Chen, name: Jiayu Chen, name: Weijie Wang, name: Xiaoye Qu, name: Yu Cheng
Published: 2025-06-04 17:51:08

Inspired by the remarkable reasoning capabilities of Deepseek-R1 in complex
textual tasks, many works attempt to incentivize similar capabilities in
Multimodal Large Language Models (MLLMs) by directly applying reinforcement
learning (RL). However, they still struggle to activate complex reasoning. In
this paper, rather than examining multimodal RL in isolation, we delve into
current training pipelines and identify three crucial phenomena: 1) Effective
cold start initialization is critical for enhancing MLLM reasoning.
Intriguingly, we find that initializing with carefully selected text data alone
can lead to performance surpassing many recent multimodal reasoning models,
even before multimodal RL. 2) Standard GRPO applied to multimodal RL suffers
from gradient stagnation, which degrades training stability and performance. 3)
Subsequent text-only RL training, following the multimodal RL phase, further
enhances multimodal reasoning. This staged training approach effectively
balances perceptual grounding and cognitive reasoning development. By
incorporating the above insights and addressing multimodal RL issues, we
introduce ReVisual-R1, achieving a new state-of-the-art among open-source 7B
MLLMs on challenging benchmarks including MathVerse, MathVision, WeMath,
LogicVista, DynaMath, and challenging AIME2024 and AIME2025.
A Controllable Examination for Long-Context Language Models
Authors: name: Yijun Yang, name: Zeyu Huang, name: Wenhao Zhu, name: Zihan Qiu, name: Fei Yuan, name: Jeff Z. Pan, name: Ivan Titov
Published: 2025-06-03 14:23:06

Existing frameworks for evaluating long-context language models (LCLM) can be
broadly categorized into real-world and synthetic tasks. Despite their utility,
both approaches are accompanied by certain intrinsic limitations. Real-world
tasks are too complex to interpret or characterize and are susceptible to data
contamination. In contrast, synthetic tasks often adopt the
needle-in-the-haystack (NIAH) format, wherein a lack of coherence between the
"needle" and the "haystack" compromises their validity as proxies for realistic
applications. In response to these challenges, we posit that an ideal
long-context evaluation framework should be characterized by three essential
features: seamless context, controllable setting, and
sound evaluation. This study introduces LongBioBench, a
novel benchmark that utilizes artificially generated biographies as a
controlled environment for assessing LCLMs across dimensions of
understanding, reasoning, and trustworthiness.
Our experimental evaluation, which includes 18 LCLMs in total,
demonstrates that most models still exhibit deficiencies in semantic
understanding and elementary reasoning over retrieved results and are less
trustworthy as context length increases. Our further analysis indicates some
design choices employed by existing synthetic benchmarks, such as contextual
non-coherence, numerical needles, and the absence of distractors, rendering
them vulnerable to test the model long-context capabilities. Moreover, we also
reveal that long-context continual pretraining primarily adjusts RoPE embedding
to accommodate extended context lengths. To sum up, compared to previous
synthetic benchmarks, LongBioBench achieves a better trade-off between
mirroring authentic language tasks and maintaining controllability, and is
highly interpretable and configurable.
Beyond the Surface: Measuring Self-Preference in LLM Judgments
Authors: name: Zhi-Yuan Chen, name: Hao Wang, name: Xinyu Zhang, name: Enrui Hu, name: Yankai Lin
Published: 2025-06-03 08:12:47

Recent studies show that large language models (LLMs) exhibit self-preference
bias when serving as judges, meaning they tend to favor their own responses
over those generated by other models. Existing methods typically measure this
bias by calculating the difference between the scores a judge model assigns to
its own responses and those it assigns to responses from other models. However,
this approach conflates self-preference bias with response quality, as
higher-quality responses from the judge model may also lead to positive score
differences, even in the absence of bias. To address this issue, we introduce
gold judgments as proxies for the actual quality of responses and propose the
DBG score, which measures self-preference bias as the difference between the
scores assigned by the judge model to its own responses and the corresponding
gold judgments. Since gold judgments reflect true response quality, the DBG
score mitigates the confounding effect of response quality on bias measurement.
Using the DBG score, we conduct comprehensive experiments to assess
self-preference bias across LLMs of varying versions, sizes, and reasoning
abilities. Additionally, we investigate two factors that influence and help
alleviate self-preference bias: response text style and the post-training data
of judge models. Finally, we explore potential underlying mechanisms of
self-preference bias from an attention-based perspective. Our code and data are
available at https://github.com/zhiyuanc2001/self-preference.
TalkingMachines: Real-Time Audio-Driven FaceTime-Style Video via
  Autoregressive Diffusion Models
Authors: name: Chetwin Low, name: Weimin Wang
Published: 2025-06-03 17:29:28

In this paper, we present TalkingMachines -- an efficient framework that
transforms pretrained video generation models into real-time, audio-driven
character animators. TalkingMachines enables natural conversational experiences
by integrating an audio large language model (LLM) with our video generation
foundation model. Our primary contributions include: (1) We adapt a pretrained
SOTA image-to-video DiT into an audio-driven avatar generation model of 18
billion parameters; (2) We enable infinite video streaming without error
accumulation through asymmetric knowledge distillation from a bidirectional
teacher model into a sparse causal, autoregressive student model; (3) We design
a high-throughput, low-latency inference pipeline incorporating several key
engineering optimizations such as: (a) disaggregation of the DiT and VAE
decoder across separate devices, (b) efficient overlap of inter-device
communication and computation using CUDA streams, (c) elimination of redundant
recomputations to maximize frame-generation throughput. Please see demo videos
here - https://aaxwaz.github.io/TalkingMachines/
RefEdit: A Benchmark and Method for Improving Instruction-based Image
  Editing Model on Referring Expressions
Authors: name: Bimsara Pathiraja, name: Maitreya Patel, name: Shivam Singh, name: Yezhou Yang, name: Chitta Baral
Published: 2025-06-03 23:20:24

Despite recent advances in inversion and instruction-based image editing,
existing approaches primarily excel at editing single, prominent objects but
significantly struggle when applied to complex scenes containing multiple
entities. To quantify this gap, we first introduce RefEdit-Bench, a rigorous
real-world benchmark rooted in RefCOCO, where even baselines trained on
millions of samples perform poorly. To overcome this limitation, we introduce
RefEdit -- an instruction-based editing model trained on our scalable synthetic
data generation pipeline. Our RefEdit, trained on only 20,000 editing triplets,
outperforms the Flux/SD3 model-based baselines trained on millions of data.
Extensive evaluations across various benchmarks demonstrate that our model not
only excels in referring expression tasks but also enhances performance on
traditional benchmarks, achieving state-of-the-art results comparable to
closed-source methods. We release data \& checkpoint for reproducibility.
Critique-GRPO: Advancing LLM Reasoning with Natural Language and
  Numerical Feedback
Authors: name: Xiaoying Zhang, name: Hao Sun, name: Yipeng Zhang, name: Kaituo Feng, name: Chaochao Lu, name: Chao Yang, name: Helen Meng
Published: 2025-06-03 17:39:02

Recent advances in reinforcement learning (RL) with numerical feedback, such
as scalar rewards, have significantly enhanced the complex reasoning
capabilities of large language models (LLMs). Despite this success, we identify
three key challenges encountered by RL with solely numerical feedback:
performance plateaus, limited effectiveness of self-reflection, and persistent
failures. We then demonstrate that RL-finetuned models, even after exhibiting
performance plateaus, can generate correct refinements on persistently failed
problems by leveraging natural language feedback in the form of critiques.
Building on this insight, we propose Critique-GRPO, an online RL framework that
integrates both natural language and numerical feedback for effective policy
optimization. Critique-GRPO enables LLMs to learn from initial responses and
critique-guided refinements simultaneously while maintaining exploration.
Extensive experiments using Qwen2.5-7B-Base and Qwen3-8B-Base show that
Critique-GRPO consistently outperforms supervised learning-based and RL-based
fine-tuning approaches across eight challenging mathematical, STEM, and general
reasoning tasks, improving average pass@1 scores by approximately 4.5% and 5%,
respectively. Notably, Critique-GRPO surpasses a strong baseline that
incorporates expert demonstrations within online RL. Further analysis reveals
two critical insights about policy exploration: (1) higher entropy does not
always guarantee efficient learning from exploration, and (2) longer responses
do not necessarily lead to more effective exploration.
Rectified Sparse Attention
Authors: name: Yutao Sun, name: Tianzhu Ye, name: Li Dong, name: Yuqing Xia, name: Jian Chen, name: Yizhao Gao, name: Shijie Cao, name: Jianyong Wang, name: Furu Wei
Published: 2025-06-04 16:01:48

Efficient long-sequence generation is a critical challenge for Large Language
Models. While recent sparse decoding methods improve efficiency, they suffer
from KV cache misalignment, where approximation errors accumulate and degrade
generation quality. In this work, we propose Rectified Sparse Attention (ReSA),
a simple yet effective method that combines block-sparse attention with
periodic dense rectification. By refreshing the KV cache at fixed intervals
using a dense forward pass, ReSA bounds error accumulation and preserves
alignment with the pretraining distribution. Experiments across math reasoning,
language modeling, and retrieval tasks demonstrate that ReSA achieves
near-lossless generation quality with significantly improved efficiency.
Notably, ReSA delivers up to 2.42times end-to-end speedup under decoding at
256K sequence length, making it a practical solution for scalable long-context
inference. Code is available at https://aka.ms/ReSA-LM.
LayerFlow: A Unified Model for Layer-aware Video Generation
Authors: name: Sihui Ji, name: Hao Luo, name: Xi Chen, name: Yuanpeng Tu, name: Yiyang Wang, name: Hengshuang Zhao
Published: 2025-06-04 17:59:58

We present LayerFlow, a unified solution for layer-aware video generation.
Given per-layer prompts, LayerFlow generates videos for the transparent
foreground, clean background, and blended scene. It also supports versatile
variants like decomposing a blended video or generating the background for the
given foreground and vice versa. Starting from a text-to-video diffusion
transformer, we organize the videos for different layers as sub-clips, and
leverage layer embeddings to distinguish each clip and the corresponding
layer-wise prompts. In this way, we seamlessly support the aforementioned
variants in one unified framework. For the lack of high-quality layer-wise
training videos, we design a multi-stage training strategy to accommodate
static images with high-quality layer annotations. Specifically, we first train
the model with low-quality video data. Then, we tune a motion LoRA to make the
model compatible with static frames. Afterward, we train the content LoRA on
the mixture of image data with high-quality layered images along with
copy-pasted video data. During inference, we remove the motion LoRA thus
generating smooth videos with desired layers.
SuperWriter: Reflection-Driven Long-Form Generation with Large Language
  Models
Authors: name: Yuhao Wu, name: Yushi Bai, name: Zhiqiang Hu, name: Juanzi Li, name: Roy Ka-Wei Lee
Published: 2025-06-04 17:27:42

Long-form text generation remains a significant challenge for large language
models (LLMs), particularly in maintaining coherence, ensuring logical
consistency, and preserving text quality as sequence length increases. To
address these limitations, we propose SuperWriter-Agent, an agent-based
framework designed to enhance the quality and consistency of long-form text
generation. SuperWriter-Agent introduces explicit structured thinking-through
planning and refinement stages into the generation pipeline, guiding the model
to follow a more deliberate and cognitively grounded process akin to that of a
professional writer. Based on this framework, we construct a supervised
fine-tuning dataset to train a 7B SuperWriter-LM. We further develop a
hierarchical Direct Preference Optimization (DPO) procedure that uses Monte
Carlo Tree Search (MCTS) to propagate final quality assessments and optimize
each generation step accordingly. Empirical results across diverse benchmarks
demonstrate that SuperWriter-LM achieves state-of-the-art performance,
surpassing even larger-scale baseline models in both automatic evaluation and
human evaluation. Furthermore, comprehensive ablation studies demonstrate the
effectiveness of hierarchical DPO and underscore the value of incorporating
structured thinking steps to improve the quality of long-form text generation.
Follow the Flow: Fine-grained Flowchart Attribution with Neurosymbolic
  Agents
Authors: name: Manan Suri, name: Puneet Mathur, name: Nedim Lipka, name: Franck Dernoncourt, name: Ryan A. Rossi, name: Vivek Gupta, name: Dinesh Manocha
Published: 2025-06-02 06:02:41

Flowcharts are a critical tool for visualizing decision-making processes.
However, their non-linear structure and complex visual-textual relationships
make it challenging to interpret them using LLMs, as vision-language models
frequently hallucinate nonexistent connections and decision paths when
analyzing these diagrams. This leads to compromised reliability for automated
flowchart processing in critical domains such as logistics, health, and
engineering. We introduce the task of Fine-grained Flowchart Attribution, which
traces specific components grounding a flowchart referring LLM response.
Flowchart Attribution ensures the verifiability of LLM predictions and improves
explainability by linking generated responses to the flowchart's structure. We
propose FlowPathAgent, a neurosymbolic agent that performs fine-grained post
hoc attribution through graph-based reasoning. It first segments the flowchart,
then converts it into a structured symbolic graph, and then employs an agentic
approach to dynamically interact with the graph, to generate attribution paths.
Additionally, we present FlowExplainBench, a novel benchmark for evaluating
flowchart attributions across diverse styles, domains, and question types.
Experimental results show that FlowPathAgent mitigates visual hallucinations in
LLM answers over flowchart QA, outperforming strong baselines by 10-14% on our
proposed FlowExplainBench dataset.
Quantitative LLM Judges
Authors: name: Aishwarya Sahoo, name: Jeevana Kruthi Karnuthala, name: Tushar Parmanand Budhwani, name: Pranchal Agarwal, name: Sankaran Vaidyanathan, name: Alexa Siu, name: Franck Dernoncourt, name: Jennifer Healey, name: Nedim Lipka, name: Ryan Rossi, name: Uttaran Bhattacharya, name: Branislav Kveton
Published: 2025-06-03 14:44:23

LLM-as-a-judge is a framework in which a large language model (LLM)
automatically evaluates the output of another LLM. We propose quantitative LLM
judges, which align evaluation scores of existing LLM judges to human scores in
a given domain using regression models. The models are trained to improve the
score of the original judge by using the judge's textual evaluation and score.
We present four quantitative judges for different types of absolute and
relative feedback, which showcases the generality and versatility of our
framework. Our framework is more computationally efficient than supervised
fine-tuning and can be more statistically efficient when human feedback is
limited, which is expected in most applications of our work. We validate these
claims empirically on four datasets using two base judges. Our experiments show
that quantitative judges can effectively improve the predictive power of
existing judges through post-hoc modeling.
MiMo-VL Technical Report
Authors: name: Xiaomi LLM-Core Team, name: Zihao Yue, name: Zhenru Lin, name: Yifan Song, name: Weikun Wang, name: Shuhuai Ren, name: Shuhao Gu, name: Shicheng Li, name: Peidian Li, name: Liang Zhao, name: Lei Li, name: Kainan Bao, name: Hao Tian, name: Hailin Zhang, name: Gang Wang, name: Dawei Zhu, name: Cici, name: Chenhong He, name: Bowen Ye, name: Bowen Shen, name: Zihan Zhang, name: Zihan Jiang, name: Zhixian Zheng, name: Zhichao Song, name: Zhenbo Luo, name: Yue Yu, name: Yudong Wang, name: Yuanyuan Tian, name: Yu Tu, name: Yihan Yan, name: Yi Huang, name: Xu Wang, name: Xinzhe Xu, name: Xingchen Song, name: Xing Zhang, name: Xing Yong, name: Xin Zhang, name: Xiangwei Deng, name: Wenyu Yang, name: Wenhan Ma, name: Weiwei Lv, name: Weiji Zhuang, name: Wei Liu, name: Sirui Deng, name: Shuo Liu, name: Shimao Chen, name: Shihua Yu, name: Shaohui Liu, name: Shande Wang, name: Rui Ma, name: Qiantong Wang, name: Peng Wang, name: Nuo Chen, name: Menghang Zhu, name: Kangyang Zhou, name: Kang Zhou, name: Kai Fang, name: Jun Shi, name: Jinhao Dong, name: Jiebao Xiao, name: Jiaming Xu, name: Huaqiu Liu, name: Hongshen Xu, name: Heng Qu, name: Haochen Zhao, name: Hanglong Lv, name: Guoan Wang, name: Duo Zhang, name: Dong Zhang, name: Di Zhang, name: Chong Ma, name: Chang Liu, name: Can Cai, name: Bingquan Xia
Published: 2025-06-04 04:32:54

We open-source MiMo-VL-7B-SFT and MiMo-VL-7B-RL, two powerful vision-language
models delivering state-of-the-art performance in both general visual
understanding and multimodal reasoning. MiMo-VL-7B-RL outperforms Qwen2.5-VL-7B
on 35 out of 40 evaluated tasks, and scores 59.4 on OlympiadBench, surpassing
models with up to 78B parameters. For GUI grounding applications, it sets a new
standard with 56.1 on OSWorld-G, even outperforming specialized models such as
UI-TARS. Our training combines four-stage pre-training (2.4 trillion tokens)
with Mixed On-policy Reinforcement Learning (MORL) integrating diverse reward
signals. We identify the importance of incorporating high-quality reasoning
data with long Chain-of-Thought into pre-training stages, and the benefits of
mixed RL despite challenges in simultaneous multi-domain optimization. We also
contribute a comprehensive evaluation suite covering 50+ tasks to promote
reproducibility and advance the field. The model checkpoints and full
evaluation suite are available at https://github.com/XiaomiMiMo/MiMo-VL.