Orionnnn/hub-stats huggingface dataset:
BPMN Assistant: An LLM-Based Approach to Business Process Modeling
Authors: name: Josip Tomo Licardo, name: Nikola Tankovic, name: Darko Etinger
Published: 2025-09-29 10:56:08

This paper presents BPMN Assistant, a tool that leverages Large Language
Models (LLMs) for natural language-based creation and editing of BPMN diagrams.
A specialized JSON-based representation is introduced as a structured
alternative to the direct handling of XML to enhance the accuracy of process
modifications. Process generation quality is evaluated using Graph Edit
Distance (GED) and Relative Graph Edit Distance (RGED), while editing
performance is evaluated with a binary success metric. Results show that JSON
and XML achieve similar similarity scores in generation, but JSON offers
greater reliability, faster processing, and significantly higher editing
success rates. We discuss key trade-offs, limitations, and future improvements.
The implementation is available at https://github.com/jtlicardo/bpmn-assistant.
Taming Masked Diffusion Language Models via Consistency Trajectory
  Reinforcement Learning with Fewer Decoding Step
Authors: name: Jingyi Yang, name: Guanxu Chen, name: Xuhao Hu, name: Jing Shao
Published: 2025-09-28 15:01:15

Masked diffusion language models (MDLMs) have recently emerged as a promising
alternative to autoregressive (AR) language models, offering properties such as
parallel decoding, flexible generation orders, and the potential for fewer
inference steps. Despite these advantages, decoding strategies and
reinforcement learning (RL) algorithms tailored for MDLMs remain underexplored.
A naive approach is to directly transfer techniques well-established for AR
models to MDLMs. However, this raises an immediate question: Is such a naive
transfer truly optimal? For example, 1) Block-wise and semi-AR decoding
strategies are not employed during the training of MDLMs, so why do they
outperform full diffusion-style decoding during inference? 2) Applying RL
algorithms designed for AR models directly to MDLMs exhibits a
training-inference inconsistency, since MDLM decoding are non-causal
(parallel). This results in inconsistencies between the rollout trajectory and
the optimization trajectory. To address these challenges, we propose EOS Early
Rejection (EOSER) and Ascending Step-Size (ASS) decoding scheduler, which
unlock the potential of MDLMs to perform full diffusion-style decoding,
achieving competitive performance with fewer decoding steps. Additionally, we
introduce Consistency Trajectory Group Relative Policy Optimization (CJ-GRPO)
for taming MDLMs, which emphasizes the consistency between rollout trajectory
and optimization trajectory, and reduces the optimization errors caused by
skip-step optimization. We conduct extensive experiments on reasoning tasks,
such as mathematical and planning benchmarks, using LLaDA-8B-Instruct. The
results demonstrate that the proposed EOSER and ASS mechanisms, together with
CJ-GRPO, hold significant promise for effectively and efficiently taming MDLMs.
Code: https://github.com/yjyddq/EOSER-ASS-RL.
SCI-Verifier: Scientific Verifier with Thinking
Authors: name: Shenghe Zheng, name: Chenyu Huang, name: Fangchen Yu, name: Junchi Yao, name: Jingqi Ye, name: Tao Chen, name: Yun Luo, name: Ning Ding, name: LEI BAI, name: Ganqu Cui, name: Peng Ye
Published: 2025-09-29 04:58:43

As large language models (LLMs) are increasingly applied to scientific
reasoning, the complexity of answer formats and the diversity of equivalent
expressions make answer verification a critical yet challenging task. Existing
verification studies in scientific domains suffer from two major limitations:
(a) the absence of systematic evaluation standards and insufficient
disciplinary coverage, which hinders their comprehensive assessment; and (b)
heavy reliance on cumbersome rule design or prompt engineering, which reduces
their effectiveness in complex reasoning scenarios or limits their
cross-disciplinary generalization. To address these challenges, we propose
solutions at both the data and model levels. On the data side, we construct
SCI-VerifyBench, a cross-disciplinary benchmark covering mathematics, physics,
biology, chemistry, and general scientific QA. The benchmark is built from real
LLM responses and enhanced with domain-specific equivalence transformations
that generate challenging and realistic data. Model-based and expert
annotations ensure both quality and diversity, enabling rigorous evaluation of
verification ability. On the model side, we emphasize the importance of
reasoning for verification and introduce SCI-Verifier, a unified
reasoning-augmented verifier for scientific domains. Through post-training,
SCI-Verifier demonstrates strong logical reasoning and equivalence judgment
capabilities while maintaining concise and stable outputs. Together,
SCI-VerifyBench and SCI-Verifier provide a principled framework for scientific
verification, offering both systematic evaluation and practical pathways to
enhance the reliability and applicability of LLMs in scientific domains.
PARROT: A Benchmark for Evaluating LLMs in Cross-System SQL Translation
Authors: name: Wei Zhou, name: Guoliang Li, name: Haoyu Wang, name: Yuxing Han, name: Xufei Wu, name: Fan Wu, name: Xuanhe Zhou
Published: 2025-09-27 14:41:13

Large language models (LLMS) have shown increasing effectiveness in
Text-to-SQL tasks. However, another closely related problem, Cross-System SQL
Translation (a.k.a., SQL-to-SQL), which adapts a query written for one database
system (e.g., MySQL) into its equivalent one for another system (e.g.,
ClickHouse), is of great practical importance but remains underexplored.
Existing SQL benchmarks are not well-suited for SQL-to-SQL evaluation, which
(1) focus on a limited set of database systems (often just SQLite) and (2)
cannot capture many system-specific SQL dialects (e.g., customized functions,
data types, and syntax rules). Thus, in this paper, we introduce PARROT, a
Practical And Realistic BenchmaRk for CrOss-System SQL Translation. PARROT
comprises 598 translation pairs from 38 open-source benchmarks and real-world
business services, specifically prepared to challenge system-specific SQL
understanding (e.g., LLMS achieve lower than 38.53% accuracy on average). We
also provide multiple benchmark variants, including PARROT-Diverse with 28,003
translations (for extensive syntax testing) and PARROT-Simple with 5,306
representative samples (for focused stress testing), covering 22
production-grade database systems. To promote future research, we release a
public leaderboard and source code at: https://code4db.github.io/parrot-bench/.
BOE-XSUM: Extreme Summarization in Clear Language of Spanish Legal
  Decrees and Notifications
Authors: name: Andrés Fernández García, name: Javier de la Rosa, name: Julio Gonzalo, name: Roser Morante, name: Enrique Amigó, name: Alejandro Benito-Santos, name: Jorge Carrillo-de-Albornoz, name: Víctor Fresno, name: Adrian Ghajari, name: Guillermo Marco, name: Laura Plaza, name: Eva Sánchez Salido
Published: 2025-09-29 15:15:17

The ability to summarize long documents succinctly is increasingly important
in daily life due to information overload, yet there is a notable lack of such
summaries for Spanish documents in general, and in the legal domain in
particular. In this work, we present BOE-XSUM, a curated dataset comprising
3,648 concise, plain-language summaries of documents sourced from Spain's
``Bolet\'{\i}n Oficial del Estado'' (BOE), the State Official Gazette. Each
entry in the dataset includes a short summary, the original text, and its
document type label. We evaluate the performance of medium-sized large language
models (LLMs) fine-tuned on BOE-XSUM, comparing them to general-purpose
generative models in a zero-shot setting. Results show that fine-tuned models
significantly outperform their non-specialized counterparts. Notably, the
best-performing model -- BERTIN GPT-J 6B (32-bit precision) -- achieves a 24\%
performance gain over the top zero-shot model, DeepSeek-R1 (accuracies of
41.6\% vs.\ 33.5\%).
Scaling Generalist Data-Analytic Agents
Authors: name: Shuofei Qiao, name: Yanqiu Zhao, name: Zhisong Qiu, name: Xiaobin Wang, name: Jintian Zhang, name: Zhao Bin, name: Ningyu Zhang, name: Yong Jiang, name: Pengjun Xie, name: Fei Huang, name: Huajun Chen
Published: 2025-09-29 17:23:08

Data-analytic agents are emerging as a key catalyst for automated scientific
discovery and for the vision of Innovating AI. Current approaches, however,
rely heavily on prompt engineering over proprietary models, while open-source
models struggle to face diverse-format, large-scale data files and
long-horizon, multi-step reasoning that real-world analytics demands. This
paper introduces DataMind, a scalable data synthesis and agent training recipe
designed to build generalist data-analytic agents. DataMind tackles three key
challenges in building open-source data-analytic agents, including insufficient
data resources, improper training strategy, and unstable code-based multi-turn
rollout. Concretely, DataMind applies 1) a fine-grained task taxonomy and a
recursive easy-to-hard task composition mechanism to increase the diversity and
difficulty of synthesized queries; 2) a knowledge-augmented trajectory sampling
strategy followed by model-based and rule-based filtering; 3) a dynamically
adjustable training objective combining both SFT and RL losses; 4) a
memory-frugal and stable code-based multi-turn rollout framework. Built on
DataMind, we curate DataMind-12K, a high-quality trajectory set spanning
diverse domains, task categories, and data file formats for data-analytic
tasks. Trained on DataMind-12K, our DataMind-14B achieves state-of-the-art with
an average score of 71.16% on multiple data analysis benchmarks, outperforming
the strongest proprietary baselines DeepSeek-V3.1 and GPT-5. Our DataMind-7B
also performs best among all open-source models with a score of 68.10%. We also
incorporate some empirical insights gained from our exploratory trials into the
analysis experiments, aiming to provide actionable insights about agentic
training for the community. We will release DataMind-12K and DataMind-7B,14B
for the community's future research.
Towards Personalized Deep Research: Benchmarks and Evaluations
Authors: name: Yuan Liang, name: Jiaxian Li, name: Yuqing Wang, name: Piaohong Wang, name: Motong Tian, name: Pai Liu, name: Shuofei Qiao, name: Runnan Fang, name: He Zhu, name: Ge Zhang, name: Minghao Liu, name: Yuchen Eleanor Jiang, name: Ningyu Zhang, name: Wangchunshu Zhou
Published: 2025-09-29 17:39:17

Deep Research Agents (DRAs) can autonomously conduct complex investigations
and generate comprehensive reports, demonstrating strong real-world potential.
However, existing evaluations mostly rely on close-ended benchmarks, while
open-ended deep research benchmarks remain scarce and typically neglect
personalized scenarios. To bridge this gap, we introduce Personalized Deep
Research Bench, the first benchmark for evaluating personalization in DRAs. It
pairs 50 diverse research tasks across 10 domains with 25 authentic user
profiles that combine structured persona attributes with dynamic real-world
contexts, yielding 250 realistic user-task queries. To assess system
performance, we propose the PQR Evaluation Framework, which jointly measures
(P) Personalization Alignment, (Q) Content Quality, and (R) Factual
Reliability. Our experiments on a range of systems highlight current
capabilities and limitations in handling personalized deep research. This work
establishes a rigorous foundation for developing and evaluating the next
generation of truly personalized AI research assistants.
Random Policy Valuation is Enough for LLM Reasoning with Verifiable
  Rewards
Authors: name: Haoran He, name: Yuxiao Ye, name: Qingpeng Cai, name: Chen Hu, name: Binxing Jiao, name: Daxin Jiang, name: Ling Pan
Published: 2025-09-29 16:09:07

RL with Verifiable Rewards (RLVR) has emerged as a promising paradigm for
improving the reasoning abilities of large language models (LLMs). Current
methods rely primarily on policy optimization frameworks like PPO and GRPO,
which follow generalized policy iteration that alternates between evaluating
the current policy's value and improving the policy based on evaluation. While
effective, they often suffer from training instability and diversity collapse,
requiring complex heuristic tricks and careful tuning. We observe that standard
RLVR in math reasoning can be formalized as a specialized finite-horizon Markov
Decision Process with deterministic state transitions, tree-structured
dynamics, and binary terminal rewards. Though large in scale, the underlying
structure is simpler than general-purpose control settings for which popular RL
algorithms (e.g., PPO) were developed, suggesting that several sophisticated
techniques in existing methods may be reduced or even omitted. Based on this
insight, we prove a surprising result: the optimal action can be recovered from
the Q-function of a fixed uniformly random policy, thereby bypassing the
generalized policy iteration loop and its associated heuristics. We introduce
Random Policy Valuation for Diverse Reasoning (ROVER) to translate this
principle into a practical and scalable algorithm for LLM math reasoning, a
minimalist yet highly effective RL method that samples actions from a softmax
over these uniform-policy Q-values. ROVER preserves diversity throughout
training, allowing sustained exploration of multiple valid pathways. Across
multiple base models and standard math reasoning benchmarks, ROVER demonstrates
superior performance in both quality (+8.2 on pass@1,
+16.8 on pass@256) and diversity (+17.6\%), despite
its radical simplification compared to strong, complicated existing methods.
Advancing Reference-free Evaluation of Video Captions with Factual
  Analysis
Authors: name: Shubhashis Roy Dipta, name: Tz-Ying Wu, name: Subarna Tripathi
Published: 2025-09-20 05:04:41

Video captions offer concise snapshots of actors, objects, and actions within
a video, serving as valuable assets for applications such as question answering
and event localization. However, acquiring human annotations for video captions
is costly or even impractical, especially when dealing with diverse video
domains. Existing models trained on supervised datasets face challenges in
evaluating performance across different domains due to the reliance on
reference-based evaluation protocols, which necessitate ground truth captions.
This assumption is unrealistic for evaluating videos in the wild. To address
these limitations, we propose a reference-free evaluation framework that does
not require ground truth captions, focusing on factual grounding to ensure
accurate assessment of caption quality. We introduce VC-Inspector, a novel
caption quality evaluator that is both reference-free and factually grounded.
Utilizing large language models, we generate pseudo captions of varying quality
based on supervised data, which are subsequently used to train a multimodal
model (i.e., Qwen2.5-VL) as the evaluator. Our approach demonstrates superior
alignment with human judgments on the VATEX-Eval dataset, outperforming
existing methods. The performance also generalizes to image caption datasets,
Flickr8K-Expert and Flickr8K-CF, when viewing images as 1-frame videos.
Overall, VC-Inspector offers a scalable and generalizable solution for
evaluating the factual accuracy of video captions, paving the way for more
effective and objective assessment methodologies in diverse video domains.
Beyond the Exploration-Exploitation Trade-off: A Hidden State Approach
  for LLM Reasoning in RLVR
Authors: name: Fanding Huang, name: Guanbo Huang, name: Xiao Fan, name: Yi He, name: Xiao Liang, name: Xiao Chen, name: Qinting Jiang, name: Faisal Nadeem Khan, name: Jingyan Jiang, name: Zhi Wang
Published: 2025-09-28 11:14:58

A prevailing view in Reinforcement Learning for Verifiable Rewards (RLVR)
interprets recent progress through the lens of an exploration-exploitation
trade-off, a perspective largely shaped by token-level metrics. We re-examine
this perspective, proposing that this perceived trade-off may not be a
fundamental constraint but rather an artifact of the measurement level. To
investigate this, we shift the analysis to the semantically rich hidden-state
space, adopting Effective Rank (ER) to quantify exploration and proposing its
novel first- and second-order derivatives, named Effective Rank Velocity (ERV)
and Effective Rank Acceleration (ERA), to capture exploitation dynamics. Our
analysis reveals that at the hidden-state level, exploration and exploitation
could be decoupled (Sec. 4). This finding reveals an opportunity to enhance
both capacities simultaneously. This insight motivates our method,
Velocity-Exploiting Rank-Learning (VERL), the first to operationalize the
principle of synergistic exploration-exploitation enhancement by directly
shaping the RL advantage function. The key innovation is leveraging the
theoretically stable ERA as a predictive meta-controller to create a
synergistic, dual-channel incentive structure. Instead of forcing a trade-off,
VERL prospectively amplifies rewards for exploration to preempt overconfidence
and reinforces exploitative gains to consolidate reasoning. Experiments across
diverse LLMs and reasoning benchmarks show consistent gains, including up to
21.4% absolute accuracy improvement on the challenging Gaokao 2024 dataset.
From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by
  Composing Old Ones
Authors: name: Lifan Yuan, name: Weize Chen, name: Yuchen Zhang, name: Ganqu Cui, name: Hanbin Wang, name: Ziming You, name: Ning Ding, name: Zhiyuan Liu, name: Maosong Sun, name: Hao Peng
Published: 2025-09-29 17:44:27

Does RL teach LLMs genuinely new skills, or does it merely activate existing
ones? This question lies at the core of ongoing debates about the role of RL in
LLM post-training. On one side, strong empirical results can be achieved with
RL even without preceding supervised finetuning; on the other, critics argue
that RL contributes little beyond reweighting existing reasoning strategies.
This work provides concrete evidence that LLMs can acquire genuinely new skills
during RL by composing existing ones, mirroring one of the central mechanisms
by which humans acquire new cognitive skills. To mitigate data contamination
and other confounding factors, and to allow precise control over task
complexity, we develop a synthetic framework for our investigation.
Specifically, we define a skill as the ability to infer the output of a string
transformation function f(x) given x. When an LLM has already learned f and g
prior to RL, our experiments reveal that RL enables it to learn unseen
compositions of them h(x)=g(f(x)). Further, this compositional ability
generalizes to more difficult problems such as compositions of >2 functions
unseen during RL training. Surprisingly, our experiments show that
compositional skill acquired on a source task transfers to a different target
task. This transfer happens even without compositional training on the target,
requiring only prior knowledge of the target's atomic skills. Our qualitative
analysis shows that RL fundamentally changes the reasoning behaviors of the
models. In contrast, next-token training with the same data yields none of
these findings. Our systematic experiments provide fresh insights into LLM
learning, suggesting the value of first building base models with basic skills,
then using RL to incentivize advanced, generalizable skills for complex
problems.
VGGT-X: When VGGT Meets Dense Novel View Synthesis
Authors: name: Yang Liu, name: Chuanchen Luo, name: Zimo Tang, name: Junran Peng, name: Zhaoxiang Zhang
Published: 2025-09-29 17:59:59

We study the problem of applying 3D Foundation Models (3DFMs) to dense Novel
View Synthesis (NVS). Despite significant progress in Novel View Synthesis
powered by NeRF and 3DGS, current approaches remain reliant on accurate 3D
attributes (e.g., camera poses and point clouds) acquired from
Structure-from-Motion (SfM), which is often slow and fragile in low-texture or
low-overlap captures. Recent 3DFMs showcase orders of magnitude speedup over
the traditional pipeline and great potential for online NVS. But most of the
validation and conclusions are confined to sparse-view settings. Our study
reveals that naively scaling 3DFMs to dense views encounters two fundamental
barriers: dramatically increasing VRAM burden and imperfect outputs that
degrade initialization-sensitive 3D training. To address these barriers, we
introduce VGGT-X, incorporating a memory-efficient VGGT implementation that
scales to 1,000+ images, an adaptive global alignment for VGGT output
enhancement, and robust 3DGS training practices. Extensive experiments show
that these measures substantially close the fidelity gap with
COLMAP-initialized pipelines, achieving state-of-the-art results in dense
COLMAP-free NVS and pose estimation. Additionally, we analyze the causes of
remaining gaps with COLMAP-initialized rendering, providing insights for the
future development of 3D foundation models and dense NVS. Our project page is
available at https://dekuliutesla.github.io/vggt-x.github.io/
BRIDGE - Building Reinforcement-Learning Depth-to-Image Data Generation
  Engine for Monocular Depth Estimation
Authors: name: Dingning Liu, name: Haoyu Guo, name: Jingyi Zhou, name: Tong He
Published: 2025-09-29 17:19:45

Monocular Depth Estimation (MDE) is a foundational task for computer vision.
Traditional methods are limited by data scarcity and quality, hindering their
robustness. To overcome this, we propose BRIDGE, an RL-optimized depth-to-image
(D2I) generation framework that synthesizes over 20M realistic and
geometrically accurate RGB images, each intrinsically paired with its ground
truth depth, from diverse source depth maps. Then we train our depth estimation
model on this dataset, employing a hybrid supervision strategy that integrates
teacher pseudo-labels with ground truth depth for comprehensive and robust
training. This innovative data generation and training paradigm enables BRIDGE
to achieve breakthroughs in scale and domain diversity, consistently
outperforming existing state-of-the-art approaches quantitatively and in
complex scene detail capture, thereby fostering general and robust depth
features. Code and models are available at
https://dingning-liu.github.io/bridge.github.io/.