Orionnnn/hub-stats huggingface dataset:
One Missing Piece for Open-Source Reasoning Models: A Dataset to
  Mitigate Cold-Starting Short CoT LLMs in RL
Authors: name: Hyungjoo Chae, name: Dongjin Kang, name: Jihyuk Kim, name: Beong-woo Kwak, name: Sunghyun Park, name: Haeju Park, name: Jinyoung Yeo, name: Moontae Lee, name: Kyungjae Lee
Published: 2025-06-03 00:29:15

With the release of R1, a publicly available large reasoning model (LRM),
researchers commonly train new LRMs by training language models on R1's long
chain-of-thought (CoT) inferences. While prior works show that LRMs'
capabilities can be reproduced through direct distillation, the continued
reliance on the existing models (e.g., R1) remains a critical limitation in
advancing the field. As a first step toward independent LRM development, this
paper explores the possibility of constructing a long CoT dataset with LLMs
that are not trained for inference-time scaling. To this end, we present the
Long CoT Collection, a dataset of 100K CoT rationales annotated using existing
short CoT LLMs. We develop a pipeline that induces o1's novel reasoning
strategies into short CoT LLMs, enabling them to think longer and introducing
controllability over the thought budget to better manage the overthinking
problem. Our extensive analyses validate that our dataset achieves quality
comparable to--or slightly below--R1. Furthermore, our experiments demonstrate
that training on our dataset not only strengthens general reasoning skills, but
also provides a strong foundation for reinforcement learning--models
initialized on our data achieve 2-3x larger gains with RLVR.
Datasheets Aren't Enough: DataRubrics for Automated Quality Metrics and
  Accountability
Authors: name: Genta Indra Winata, name: David Anugraha, name: Emmy Liu, name: Alham Fikri Aji, name: Shou-Yi Hung, name: Aditya Parashar, name: Patrick Amadeus Irawan, name: Ruochen Zhang, name: Zheng-Xin Yong, name: Jan Christian Blaise Cruz, name: Niklas Muennighoff, name: Seungone Kim, name: Hanyang Zhao, name: Sudipta Kar, name: Kezia Erina Suryoraharjo, name: M. Farid Adilazuarda, name: En-Shiun Annie Lee, name: Ayu Purwarianti, name: Derry Tanti Wijaya, name: Monojit Choudhury
Published: 2025-06-02 15:31:52

High-quality datasets are fundamental to training and evaluating machine
learning models, yet their creation-especially with accurate human
annotations-remains a significant challenge. Many dataset paper submissions
lack originality, diversity, or rigorous quality control, and these
shortcomings are often overlooked during peer review. Submissions also
frequently omit essential details about dataset construction and properties.
While existing tools such as datasheets aim to promote transparency, they are
largely descriptive and do not provide standardized, measurable methods for
evaluating data quality. Similarly, metadata requirements at conferences
promote accountability but are inconsistently enforced. To address these
limitations, this position paper advocates for the integration of systematic,
rubric-based evaluation metrics into the dataset review process-particularly as
submission volumes continue to grow. We also explore scalable, cost-effective
methods for synthetic data generation, including dedicated tools and
LLM-as-a-judge approaches, to support more efficient evaluation. As a call to
action, we introduce DataRubrics, a structured framework for assessing the
quality of both human- and model-generated datasets. Leveraging recent advances
in LLM-based evaluation, DataRubrics offers a reproducible, scalable, and
actionable solution for dataset quality assessment, enabling both authors and
reviewers to uphold higher standards in data-centric research. We also release
code to support reproducibility of LLM-based evaluations at
https://github.com/datarubrics/datarubrics.
RelationAdapter: Learning and Transferring Visual Relation with
  Diffusion Transformers
Authors: name: Yan Gong, name: Yiren Song, name: Yicheng Li, name: Chenglin Li, name: Yin Zhang
Published: 2025-06-03 07:06:35

Inspired by the in-context learning mechanism of large language models
(LLMs), a new paradigm of generalizable visual prompt-based image editing is
emerging. Existing single-reference methods typically focus on style or
appearance adjustments and struggle with non-rigid transformations. To address
these limitations, we propose leveraging source-target image pairs to extract
and transfer content-aware editing intent to novel query images. To this end,
we introduce RelationAdapter, a lightweight module that enables Diffusion
Transformer (DiT) based models to effectively capture and apply visual
transformations from minimal examples. We also introduce Relation252K, a
comprehensive dataset comprising 218 diverse editing tasks, to evaluate model
generalization and adaptability in visual prompt-driven scenarios. Experiments
on Relation252K show that RelationAdapter significantly improves the model's
ability to understand and transfer editing intent, leading to notable gains in
generation quality and overall editing performance.
Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports
  From Scratch with Agentic Framework
Authors: name: Zhaorui Yang, name: Bo Pan, name: Han Wang, name: Yiyao Wang, name: Xingyu Liu, name: Minfeng Zhu, name: Bo Zhang, name: Wei Chen
Published: 2025-06-03 05:18:19

Visualizations play a crucial part in effective communication of concepts and
information. Recent advances in reasoning and retrieval augmented generation
have enabled Large Language Models (LLMs) to perform deep research and generate
comprehensive reports. Despite its progress, existing deep research frameworks
primarily focus on generating text-only content, leaving the automated
generation of interleaved texts and visualizations underexplored. This novel
task poses key challenges in designing informative visualizations and
effectively integrating them with text reports. To address these challenges, we
propose Formal Description of Visualization (FDV), a structured textual
representation of charts that enables LLMs to learn from and generate diverse,
high-quality visualizations. Building on this representation, we introduce
Multimodal DeepResearcher, an agentic framework that decomposes the task into
four stages: (1) researching, (2) exemplar report textualization, (3) planning,
and (4) multimodal report generation. For the evaluation of generated
multimodal reports, we develop MultimodalReportBench, which contains 100
diverse topics served as inputs along with 5 dedicated metrics. Extensive
experiments across models and evaluation methods demonstrate the effectiveness
of Multimodal DeepResearcher. Notably, utilizing the same Claude 3.7 Sonnet
model, Multimodal DeepResearcher achieves an 82\% overall win rate over the
baseline method.
R^2ec: Towards Large Recommender Models with Reasoning
Authors: name: Runyang You, name: Yongqi Li, name: Xinyu Lin, name: Xin Zhang, name: Wenjie Wang, name: Wenjie Li, name: Liqiang Nie
Published: 2025-05-22 17:55:43

Large recommender models have extended LLMs as powerful recommenders via
encoding or item generation, and recent breakthroughs in LLM reasoning
synchronously motivate the exploration of reasoning in recommendation. Current
studies usually position LLMs as external reasoning modules to yield auxiliary
thought for augmenting conventional recommendation pipelines. However, such
decoupled designs are limited in significant resource cost and suboptimal joint
optimization. To address these issues, we propose \name, a unified large
recommender model with intrinsic reasoning capabilities. Initially, we
reconceptualize the model architecture to facilitate interleaved reasoning and
recommendation in the autoregressive process. Subsequently, we propose RecPO, a
corresponding reinforcement learning framework that optimizes \name\ both the
reasoning and recommendation capabilities simultaneously in a single policy
update; RecPO introduces a fused reward scheme that solely leverages
recommendation labels to simulate the reasoning capability, eliminating
dependency on specialized reasoning annotations. Experiments on three datasets
with various baselines verify the effectiveness of \name, showing relative
improvements of 68.67\% in Hit@5 and 45.21\% in NDCG@20. Code available at
https://github.com/YRYangang/RRec.
Native-Resolution Image Synthesis
Authors: name: Zidong Wang, name: Lei Bai, name: Xiangyu Yue, name: Wanli Ouyang, name: Yiyuan Zhang
Published: 2025-06-03 17:57:33

We introduce native-resolution image synthesis, a novel generative modeling
paradigm that enables the synthesis of images at arbitrary resolutions and
aspect ratios. This approach overcomes the limitations of conventional
fixed-resolution, square-image methods by natively handling variable-length
visual tokens, a core challenge for traditional techniques. To this end, we
introduce the Native-resolution diffusion Transformer (NiT), an architecture
designed to explicitly model varying resolutions and aspect ratios within its
denoising process. Free from the constraints of fixed formats, NiT learns
intrinsic visual distributions from images spanning a broad range of
resolutions and aspect ratios. Notably, a single NiT model simultaneously
achieves the state-of-the-art performance on both ImageNet-256x256 and 512x512
benchmarks. Surprisingly, akin to the robust zero-shot capabilities seen in
advanced large language models, NiT, trained solely on ImageNet, demonstrates
excellent zero-shot generalization performance. It successfully generates
high-fidelity images at previously unseen high resolutions (e.g., 1536 x 1536)
and diverse aspect ratios (e.g., 16:9, 3:1, 4:3), as shown in Figure 1. These
findings indicate the significant potential of native-resolution modeling as a
bridge between visual generative modeling and advanced LLM methodologies.
FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation
Authors: name: Junyu Luo, name: Zhizhuo Kou, name: Liming Yang, name: Xiao Luo, name: Jinsheng Huang, name: Zhiping Xiao, name: Jingshu Peng, name: Chengzhong Liu, name: Jiaming Ji, name: Xuanzhe Liu, name: Sirui Han, name: Ming Zhang, name: Yike Guo
Published: 2025-05-30 15:36:19

Multimodal Large Language Models (MLLMs) have experienced rapid development
in recent years. However, in the financial domain, there is a notable lack of
effective and specialized multimodal evaluation datasets. To advance the
development of MLLMs in the finance domain, we introduce FinMME, encompassing
more than 11,000 high-quality financial research samples across 18 financial
domains and 6 asset classes, featuring 10 major chart types and 21 subtypes. We
ensure data quality through 20 annotators and carefully designed validation
mechanisms. Additionally, we develop FinScore, an evaluation system
incorporating hallucination penalties and multi-dimensional capability
assessment to provide an unbiased evaluation. Extensive experimental results
demonstrate that even state-of-the-art models like GPT-4o exhibit
unsatisfactory performance on FinMME, highlighting its challenging nature. The
benchmark exhibits high robustness with prediction variations under different
prompts remaining below 1%, demonstrating superior reliability compared to
existing datasets. Our dataset and evaluation protocol are available at
https://huggingface.co/datasets/luojunyu/FinMME and
https://github.com/luo-junyu/FinMME.
VS-Bench: Evaluating VLMs for Strategic Reasoning and Decision-Making in
  Multi-Agent Environments
Authors: name: Zelai Xu, name: Zhexuan Xu, name: Xiangmin Yi, name: Huining Yuan, name: Xinlei Chen, name: Yi Wu, name: Chao Yu, name: Yu Wang
Published: 2025-06-03 02:57:38

Recent advancements in Vision Language Models (VLMs) have expanded their
capabilities to interactive agent tasks, yet existing benchmarks remain limited
to single-agent or text-only environments. In contrast, real-world scenarios
often involve multiple agents interacting within rich visual and linguistic
contexts, posing challenges with both multimodal observations and strategic
interactions. To bridge this gap, we introduce Visual Strategic Bench
(VS-Bench), a multimodal benchmark that evaluates VLMs for strategic reasoning
and decision-making in multi-agent environments. VS-Bench comprises eight
vision-grounded environments spanning cooperative, competitive, and
mixed-motive interactions, designed to assess agents' ability to predict
others' future moves and optimize for long-term objectives. We consider two
complementary evaluation dimensions, including offline evaluation of strategic
reasoning by next-action prediction accuracy and online evaluation of
decision-making by normalized episode return. Extensive experiments of fourteen
leading VLMs reveal a significant gap between current models and optimal
performance, with the best models attaining 47.8% prediction accuracy and 24.3%
normalized return. We further conduct in-depth analyses on multimodal
observations, test-time scaling, social behaviors, and failure cases of VLM
agents. By standardizing the evaluation and highlighting the limitations of
existing models, we envision VS-Bench as a foundation for future research on
strategic multimodal agents. Code and data are available at
https://vs-bench.github.io.
UniWorld: High-Resolution Semantic Encoders for Unified Visual
  Understanding and Generation
Authors: name: Bin Lin, name: Zongjian Li, name: Xinhua Cheng, name: Yuwei Niu, name: Yang Ye, name: Xianyi He, name: Shenghai Yuan, name: Wangbo Yu, name: Shaodong Wang, name: Yunyang Ge, name: Yatian Pang, name: Li Yuan
Published: 2025-06-03 17:59:33

Although existing unified models deliver strong performance on
vision-language understanding and text-to-image generation, their models are
limited in exploring image perception and manipulation tasks, which are
urgently desired by users for wide applications. Recently, OpenAI released
their powerful GPT-4o-Image model for comprehensive image perception and
manipulation, achieving expressive capability and attracting community
interests. By observing the performance of GPT-4o-Image in our carefully
constructed experiments, we infer that GPT-4o-Image leverages features
extracted by semantic encoders instead of VAE, while VAEs are considered
essential components in many image manipulation models. Motivated by such
inspiring observations, we present a unified generative framework named
UniWorld based on semantic features provided by powerful visual-language models
and contrastive semantic encoders. As a result, we build a strong unified model
using only 1% amount of BAGEL's data, which consistently outperforms BAGEL on
image editing benchmarks. UniWorld also maintains competitive image
understanding and generation capabilities, achieving strong performance across
multiple image perception tasks. We fully open-source our models, including
model weights, training and evaluation scripts, and datasets.
SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis
Authors: name: Zijian Wu, name: Jinjie Ni, name: Xiangyan Liu, name: Zichen Liu, name: Hang Yan, name: Michael Qizhe Shieh
Published: 2025-06-02 17:45:16

Vision-language models (VLMs) trained via reinforcement learning with
verifiable reward (RLVR) have shown notable progress in scaling test-time
compute effectively. In this work, we investigate how synthesized RL data can
further improve RLVR. To this end, we propose SynthRL-a scalable and
guaranteed pipeline for automatic data scaling in reasoning-oriented RL
training. SynthRL comprises three key stages: (1) selecting seed questions with
appropriate distribution, (2) augmenting them into more challenging variants
while preserving the original answers, and (3) a guaranteed verification stage
that ensures near-perfect correctness and difficulty enhancement. Our empirical
experiments demonstrate SynthRL's scalability and effectiveness. When applied
to the MMK12 dataset, SynthRL synthesizes over 3.3K additional verifiable,
challenging questions from approximately 8K seed samples. Models trained with
our synthesized data achieve consistent gains across five out-of-domain visual
math reasoning benchmarks, with a significant improvement over baseline models
trained on seed data alone. Notably, detailed analysis reveals that the gains
are more pronounced on the most challenging evaluation samples, highlighting
SynthRL's effectiveness in eliciting deeper and more complex reasoning
patterns.
Co-Evolving LLM Coder and Unit Tester via Reinforcement Learning
Authors: name: Yinjie Wang, name: Ling Yang, name: Ye Tian, name: Ke Shen, name: Mengdi Wang
Published: 2025-06-03 17:58:42

We propose CURE, a novel reinforcement learning framework with a dedicated
reward design that co-evolves coding and unit test generation capabilities
based on their interaction outcomes, without any ground-truth code as
supervision. This approach enables flexible and scalable training and allows
the unit tester to learn directly from the coder's mistakes. Our derived
ReasonFlux-Coder-7B and 14B models improve code generation accuracy by 5.3% and
Best-of-N accuracy by 9.0% after optimization on Qwen2.5-Instruct models,
outperforming similarly sized Qwen-Coder, DeepSeek-Coder, and Seed-Coder. They
naturally extend to downstream tasks such as test-time scaling and agentic
coding-achieving a 8.1% improvement over the base model. For the long-CoT
model, our ReasonFlux-Coder-4B consistently outperforms Qwen3-4B while
achieving 64.8% inference efficiency in unit test generation. Notably, we also
find that our model can serve as an effective reward model for reinforcement
learning on base models. Project: https://github.com/Gen-Verse/CURE
M^3FinMeeting: A Multilingual, Multi-Sector, and Multi-Task Financial
  Meeting Understanding Evaluation Dataset
Authors: name: Jie Zhu, name: Junhui Li, name: Yalong Wen, name: Xiandong Li, name: Lifan Guo, name: Feng Chen
Published: 2025-06-03 06:41:09

Recent breakthroughs in large language models (LLMs) have led to the
development of new benchmarks for evaluating their performance in the financial
domain. However, current financial benchmarks often rely on news articles,
earnings reports, or announcements, making it challenging to capture the
real-world dynamics of financial meetings. To address this gap, we propose a
novel benchmark called M^3FinMeeting, which is a multilingual,
multi-sector, and multi-task dataset designed for financial meeting
understanding. First, M^3FinMeeting supports English, Chinese, and
Japanese, enhancing comprehension of financial discussions in diverse
linguistic contexts. Second, it encompasses various industry sectors defined by
the Global Industry Classification Standard (GICS), ensuring that the benchmark
spans a broad range of financial activities. Finally,
M^3FinMeeting includes three tasks: summarization, question-answer
(QA) pair extraction, and question answering, facilitating a more realistic and
comprehensive evaluation of understanding. Experimental results with seven
popular LLMs reveal that even the most advanced long-context models have
significant room for improvement, demonstrating the effectiveness of
M^3FinMeeting as a benchmark for assessing LLMs' financial meeting
comprehension skills.
MotionSight: Boosting Fine-Grained Motion Understanding in Multimodal
  LLMs
Authors: name: Yipeng Du, name: Tiehan Fan, name: Kepan Nan, name: Rui Xie, name: Penghao Zhou, name: Xiang Li, name: Jian Yang, name: Zhenheng Yang, name: Ying Tai
Published: 2025-06-02 13:44:56

Despite advancements in Multimodal Large Language Models (MLLMs), their
proficiency in fine-grained video motion understanding remains critically
limited. They often lack inter-frame differencing and tend to average or ignore
subtle visual cues. Furthermore, while visual prompting has shown potential in
static images, its application to video's temporal complexities, particularly
for fine-grained motion understanding, remains largely unexplored. We
investigate whether inherent capability can be unlocked and boost MLLMs' motion
perception and enable distinct visual signatures tailored to decouple object
and camera motion cues. In this study, we introduce MotionSight, a novel
zero-shot method pioneering object-centric visual spotlight and motion blur as
visual prompts to effectively improve fine-grained motion understanding without
training. To convert this into valuable data assets, we curated MotionVid-QA,
the first large-scale dataset for fine-grained video motion understanding, with
hierarchical annotations including SFT and preference data, {\Theta}(40K) video
clips and {\Theta}(87K) QAs. Experiments show MotionSight achieves
state-of-the-art open-source performance and competitiveness with commercial
models. In particular, for fine-grained motion understanding we present a novel
zero-shot technique and a large-scale, high-quality dataset. All the code and
annotations will be publicly available.
Plan and Budget: Effective and Efficient Test-Time Scaling on Large
  Language Model Reasoning
Authors: name: Junhong Lin, name: Xinyue Zeng, name: Jie Zhu, name: Song Wang, name: Julian Shun, name: Jun Wu, name: Dawei Zhou
Published: 2025-05-22 01:56:29

Large Language Models (LLMs) have achieved remarkable success in complex
reasoning tasks, but their inference remains computationally inefficient. We
observe a common failure mode in many prevalent LLMs, overthinking, where
models generate verbose and tangential reasoning traces even for simple
queries. Recent works have tried to mitigate this by enforcing fixed token
budgets, however, this can lead to underthinking, especially on harder
problems. Through empirical analysis, we identify that this inefficiency often
stems from unclear problem-solving strategies. To formalize this, we develop a
theoretical model, BBAM (Bayesian Budget Allocation Model), which models
reasoning as a sequence of sub-questions with varying uncertainty, and
introduce the E^3 metric to capture the trade-off between correctness and
computation efficiency. Building on theoretical results from BBAM, we propose
Plan-and-Budget, a model-agnostic, test-time framework that decomposes complex
queries into sub-questions and allocates token budgets based on estimated
complexity using adaptive scheduling. Plan-and-Budget improves reasoning
efficiency across a range of tasks and models, achieving up to +70% accuracy
gains, -39% token reduction, and +187.5% improvement in E^3. Notably, it
elevates a smaller model (DS-Qwen-32B) to match the efficiency of a larger
model (DS-LLaMA-70B)-demonstrating Plan-and-Budget's ability to close
performance gaps without retraining. Our code is available at
anonymous.4open.science/r/P-and-B-6513/.
Pixels Versus Priors: Controlling Knowledge Priors in Vision-Language
  Models through Visual Counterfacts
Authors: name: Michal Golovanevsky, name: William Rudman, name: Michael Lepori, name: Amir Bar, name: Ritambhara Singh, name: Carsten Eickhoff
Published: 2025-05-21 22:56:55

Multimodal Large Language Models (MLLMs) perform well on tasks such as visual
question answering, but it remains unclear whether their reasoning relies more
on memorized world knowledge or on the visual information present in the input
image. To investigate this, we introduce Visual CounterFact, a new dataset of
visually-realistic counterfactuals that put world knowledge priors (e.g, red
strawberry) into direct conflict with visual input (e.g, blue strawberry).
Using Visual CounterFact, we show that model predictions initially reflect
memorized priors, but shift toward visual evidence in mid-to-late layers. This
dynamic reveals a competition between the two modalities, with visual input
ultimately overriding priors during evaluation. To control this behavior, we
propose Pixels Versus Priors (PvP) steering vectors, a mechanism for
controlling model outputs toward either world knowledge or visual input through
activation-level interventions. On average, PvP successfully shifts 92.5% of
color and 74.6% of size predictions from priors to counterfactuals. Together,
these findings offer new tools for interpreting and controlling factual
behavior in multimodal models.