Orionnnn/hub-stats huggingface dataset:
TL;DR: Too Long, Do Re-weighting for Effcient LLM Reasoning Compression
Authors: name: Zhong-Zhi Li, name: Xiao Liang, name: Zihao Tang, name: Lei Ji, name: Peijie Wang, name: Haotian Xu, name: Xing W, name: Haizhen Huang, name: Weiwei Deng, name: Ying Nian Wu, name: Yeyun Gong, name: Zhijiang Guo, name: Xiao Liu, name: Fei Yin, name: Cheng-Lin Liu
Published: 2025-06-03 09:23:41

Large Language Models (LLMs) have recently achieved remarkable progress by
leveraging Reinforcement Learning and extended Chain-of-Thought (CoT)
techniques. However, the challenge of performing efficient language
reasoning--especially during inference with extremely long outputs--has drawn
increasing attention from the research community. In this work, we propose a
dynamic ratio-based training pipeline that does not rely on sophisticated data
annotations or interpolation between multiple models. We continuously balance
the weights between the model's System-1 and System-2 data to eliminate
redundant reasoning processes while preserving the model's reasoning
capability. We validate our approach across models on DeepSeek-R1-Distill-7B
and DeepSeek-R1-Distill-14B and on a diverse set of benchmarks with varying
difficulty levels. Our method significantly reduces the number of output tokens
by nearly 40% while maintaining the accuracy of the reasoning. Our code and
data will be available soon.
Beyond In-Context Learning: Aligning Long-form Generation of Large
  Language Models via Task-Inherent Attribute Guidelines
Authors: name: Do Xuan Long, name: Duong Ngoc Yen, name: Do Xuan Trong, name: Luu Anh Tuan, name: Kenji Kawaguchi, name: Shafiq Joty, name: Min-Yen Kan, name: Nancy F. Chen
Published: 2025-06-02 02:35:24

In-context learning (ICL) is an important yet not fully understood ability of
pre-trained large language models (LLMs). It can greatly enhance task
performance using a few examples, termed demonstrations, without fine-tuning.
Although effective in question answering, ICL often underperforms in long-form
generation tasks such as summarization. Under appropriately realistic
assumptions, we empirically and theoretically show that ICL demonstrations
alone are insufficient to teach LLMs the task language and format distributions
for generation. We argue for explicit exposure to the task distributions and
hypothesize that defining them by prompting enhances model performance. To this
end, we present LongGuide, which efficiently generates two parallel streams of
guidelines capturing task language and format properties: (i) Metric Guidelines
(MGs) that instruct models to optimize self-evaluated metrics; and (ii) Output
Constraint Guidelines (OCGs) that constrain generation at both token and
sentence levels. LongGuide automatically selects the best combination of
guidelines, improving both strong open- and closed-source LLMs by over 5% in
both zero- and few-shot settings. We show that LongGuide is generalizable,
learnable by weak models to enhance strong ones, and integrates synergistically
with automatic prompt optimizers.
Self-Challenging Language Model Agents
Authors: name: Yifei Zhou, name: Sergey Levine, name: Jason Weston, name: Xian Li, name: Sainbayar Sukhbaatar
Published: 2025-06-02 14:23:33

Large language models are quickly becoming the foundation for intelligent
agents that are capable of using tools. However, training such agents is
challenging because it requires human creation and annotation of a diverse set
of tasks, tools, and evaluation criteria. In this paper, we propose the
Self-Challenging framework for training an agent on high-quality tasks that are
generated by itself. The agent first plays the role of challenger and generates
a task after interacting with the given tools. The tasks take the form of a
novel general class of problems termed Code-as-Task, which are defined by an
instruction, a verification function and solution and failure cases which serve
as tests, allowing to filter only for high-quality tasks. The agent then takes
an executor role and trains on those tasks with reinforcement learning using
the evaluation feedback as a reward. Evaluation on two existing multi-turn
tool-use agent benchmarks, M3ToolEval and TauBench, shows the Self-Challenging
framework achieves over a two-fold improvement in Llama-3.1-8B-Instruct,
despite using only self-generated training data.
DCM: Dual-Expert Consistency Model for Efficient and High-Quality Video
  Generation
Authors: name: Zhengyao Lv, name: Chenyang Si, name: Tianlin Pan, name: Zhaoxi Chen, name: Kwan-Yee K. Wong, name: Yu Qiao, name: Ziwei Liu
Published: 2025-06-03 17:55:04

Diffusion Models have achieved remarkable results in video synthesis but
require iterative denoising steps, leading to substantial computational
overhead. Consistency Models have made significant progress in accelerating
diffusion models. However, directly applying them to video diffusion models
often results in severe degradation of temporal consistency and appearance
details. In this paper, by analyzing the training dynamics of Consistency
Models, we identify a key conflicting learning dynamics during the distillation
process: there is a significant discrepancy in the optimization gradients and
loss contributions across different timesteps. This discrepancy prevents the
distilled student model from achieving an optimal state, leading to compromised
temporal consistency and degraded appearance details. To address this issue, we
propose a parameter-efficient Dual-Expert Consistency Model~(DCM),
where a semantic expert focuses on learning semantic layout and motion, while a
detail expert specializes in fine detail refinement. Furthermore, we introduce
Temporal Coherence Loss to improve motion consistency for the semantic expert
and apply GAN and Feature Matching Loss to enhance the synthesis quality of the
detail expert.Our approach achieves state-of-the-art visual quality with
significantly reduced sampling steps, demonstrating the effectiveness of expert
specialization in video diffusion model distillation. Our code and models are
available at
https://github.com/Vchitect/DCM{https://github.com/Vchitect/DCM}.
SHARE: An SLM-based Hierarchical Action CorREction Assistant for
  Text-to-SQL
Authors: name: Ge Qu, name: Jinyang Li, name: Bowen Qin, name: Xiaolong Li, name: Nan Huo, name: Chenhao Ma, name: Reynold Cheng
Published: 2025-05-31 04:51:12

Current self-correction approaches in text-to-SQL face two critical
limitations: 1) Conventional self-correction methods rely on recursive
self-calls of LLMs, resulting in multiplicative computational overhead, and 2)
LLMs struggle to implement effective error detection and correction for
declarative SQL queries, as they fail to demonstrate the underlying reasoning
path. In this work, we propose SHARE, an SLM-based Hierarchical Action
corREction assistant that enables LLMs to perform more precise error
localization and efficient correction. SHARE orchestrates three specialized
Small Language Models (SLMs) in a sequential pipeline, where it first
transforms declarative SQL queries into stepwise action trajectories that
reveal underlying reasoning, followed by a two-phase granular refinement. We
further propose a novel hierarchical self-evolution strategy for data-efficient
training. Experimental results demonstrate that SHARE effectively enhances
self-correction capabilities while proving robust across various LLMs.
Furthermore, our comprehensive analysis shows that SHARE maintains strong
performance even in low-resource training settings, which is particularly
valuable for text-to-SQL applications with data privacy constraints.
OThink-R1: Intrinsic Fast/Slow Thinking Mode Switching for
  Over-Reasoning Mitigation
Authors: name: Shengjia Zhang, name: Junjie Wu, name: Jiawei Chen, name: Changwang Zhang, name: Xingyu Lou, name: Wangchunshu Zhou, name: Sheng Zhou, name: Can Wang, name: Jun Wang
Published: 2025-06-03 03:31:30

Recent advanced large reasoning models (LRMs) leverage extended
chain-of-thought (CoT) reasoning to solve complex tasks, achieving
state-of-the-art performance. Despite their success, we identify a critical
issue: a substantial portion of simple tasks solved by LRMs can also be
addressed by non-reasoning LLMs using significantly fewer tokens, indicating
the complex reasoning may not always be necessary. To address this, we
systematically analyze the reasoning trajectories of LRMs and present a method
utilizing identified paradigms and LLM-Judge to classify these trajectories as
either Redundant Reasoning or Essential Reasoning. And we introduce OThink-R1,
a method that prunes redundant reasoning steps while preserving logical
validity. OThink-R1 dynamically employs the non-thinking mode (fast-thinking)
for straightforward problems while engaging in deliberate thinking
(slow-thinking) for complex problems. Experiments across mathematical and
question-answering tasks demonstrate that OThink-R1 reduces reasoning
redundancy by almost 23\% on average without compromising accuracy, offering
practical guidelines for efficient reasoning models. The code is available at
https://github.com/AgenticIR-Lab/OThink-R1.
QARI-OCR: High-Fidelity Arabic Text Recognition through Multimodal Large
  Language Model Adaptation
Authors: name: Ahmed Wasfy, name: Omer Nacar, name: Abdelakreem Elkhateb, name: Mahmoud Reda, name: Omar Elshehy, name: Adel Ammar, name: Wadii Boulila
Published: 2025-06-02 22:21:06

The inherent complexities of Arabic script; its cursive nature, diacritical
marks (tashkeel), and varied typography, pose persistent challenges for Optical
Character Recognition (OCR). We present Qari-OCR, a series of vision-language
models derived from Qwen2-VL-2B-Instruct, progressively optimized for Arabic
through iterative fine-tuning on specialized synthetic datasets. Our leading
model, QARI v0.2, establishes a new open-source state-of-the-art with a Word
Error Rate (WER) of 0.160, Character Error Rate (CER) of 0.061, and BLEU score
of 0.737 on diacritically-rich texts. Qari-OCR demonstrates superior handling
of tashkeel, diverse fonts, and document layouts, alongside impressive
performance on low-resolution images. Further explorations (QARI v0.3) showcase
strong potential for structural document understanding and handwritten text.
This work delivers a marked improvement in Arabic OCR accuracy and efficiency,
with all models and datasets released to foster further research.
MERIT: Multilingual Semantic Retrieval with Interleaved Multi-Condition
  Query
Authors: name: Wei Chow, name: Yuan Gao, name: Linfeng Li, name: Xian Wang, name: Qi Xu, name: Hang Song, name: Lingdong Kong, name: Ran Zhou, name: Yi Zeng, name: Yidong Cai, name: Botian Jiang, name: Shilin Xu, name: Jiajun Zhang, name: Minghui Qiu, name: Xiangtai Li, name: Tianshu Yang, name: Siliang Tang, name: Juncheng Li
Published: 2025-06-03 17:59:14

Semantic retrieval is crucial for modern applications yet remains
underexplored in current research. Existing datasets are limited to single
languages, single images, or singular retrieval conditions, often failing to
fully exploit the expressive capacity of visual information as evidenced by
maintained performance when images are replaced with captions. However,
practical retrieval scenarios frequently involve interleaved multi-condition
queries with multiple images. Hence, this paper introduces MERIT, the first
multilingual dataset for interleaved multi-condition semantic retrieval,
comprising 320,000 queries with 135,000 products in 5 languages, covering 7
distinct product categories. Extensive experiments on MERIT identify existing
models's limitation: focusing solely on global semantic information while
neglecting specific conditional elements in queries. Consequently, we propose
Coral, a novel fine-tuning framework that adapts pre-trained MLLMs by
integrating embedding reconstruction to preserve fine-grained conditional
elements and contrastive learning to extract comprehensive global semantics.
Experiments demonstrate that Coral achieves a 45.9% performance improvement
over conventional approaches on MERIT, with strong generalization capabilities
validated across 8 established retrieval benchmarks. Collectively, our
contributions - a novel dataset, identification of critical limitations in
existing approaches, and an innovative fine-tuning framework - establish a
foundation for future research in interleaved multi-condition semantic
retrieval.
FuseLIP: Multimodal Embeddings via Early Fusion of Discrete Tokens
Authors: name: Christian Schlarmann, name: Francesco Croce, name: Nicolas Flammarion, name: Matthias Hein
Published: 2025-06-03 17:27:12

Contrastive language-image pre-training aligns the features of text-image
pairs in a common latent space via distinct encoders for each modality. While
this approach achieves impressive performance in several zero-shot tasks, it
cannot natively handle multimodal inputs, i.e., encoding image and text into a
single feature vector. As a remedy, it is common practice to use additional
modules to merge the features extracted by the unimodal encoders. In this work,
we present FuseLIP, an alternative architecture for multimodal embedding.
Leveraging recent progress in discrete image tokenizers, we propose to use a
single transformer model which operates on an extended vocabulary of text and
image tokens. This early fusion approach allows the different modalities to
interact at each depth of encoding and obtain richer representations compared
to common late fusion. We collect new datasets for multimodal pre-training and
evaluation, designing challenging tasks for multimodal encoder models. We show
that FuseLIP outperforms other approaches in multimodal embedding tasks such as
VQA and text-guided image transformation retrieval, while being comparable to
baselines on unimodal tasks.
How Much Backtracking is Enough? Exploring the Interplay of SFT and RL
  in Enhancing LLM Reasoning
Authors: name: Hongyi James Cai, name: Junlin Wang, name: Xiaoyin Chen, name: Bhuwan Dhingra
Published: 2025-05-30 06:49:00

Recent breakthroughs in large language models (LLMs) have effectively
improved their reasoning abilities, particularly on mathematical and logical
problems that have verifiable answers, through techniques such as supervised
finetuning (SFT) and reinforcement learning (RL). Prior research indicates that
RL effectively internalizes search strategies, enabling long chain-of-thought
(CoT) reasoning, with backtracking emerging naturally as a learned capability.
However, the precise benefits of backtracking, specifically, how significantly
it contributes to reasoning improvements and the optimal extent of its use,
remain poorly understood. In this work, we systematically investigate the
dynamics between SFT and RL on eight reasoning tasks: Countdown, Sudoku, Arc
1D, Geometry, Color Cube Rotation, List Functions, Zebra Puzzles, and Self
Reference. Our findings highlight that short CoT sequences used in SFT as a
warm-up do have moderate contribution to RL training, compared with cold-start
RL; however such contribution diminishes when tasks become increasingly
difficult. Motivated by this observation, we construct synthetic datasets
varying systematically in the number of backtracking steps and conduct
controlled experiments to isolate the influence of either the correctness
(content) or the structure (i.e., backtrack frequency). We find that (1) longer
CoT with backtracks generally induce better and more stable RL training, (2)
more challenging problems with larger search space tend to need higher numbers
of backtracks during the SFT stage. Additionally, we demonstrate through
experiments on distilled data that RL training is largely unaffected by the
correctness of long CoT sequences, suggesting that RL prioritizes structural
patterns over content correctness. Collectively, our results offer practical
insights into designing optimal training strategies to effectively scale
reasoning in LLMs.
Knowing Before Saying: LLM Representations Encode Information About
  Chain-of-Thought Success Before Completion
Authors: name: Anum Afzal, name: Florian Matthes, name: Gal Chechik, name: Yftah Ziser
Published: 2025-05-30 08:54:28

We investigate whether the success of a zero-shot Chain-of-Thought (CoT)
process can be predicted before completion. We discover that a probing
classifier, based on LLM representations, performs well even before a
single token is generated, suggesting that crucial information about the
reasoning process is already present in the initial steps representations. In
contrast, a strong BERT-based baseline, which relies solely on the generated
tokens, performs worse, likely because it depends on shallow linguistic cues
rather than deeper reasoning dynamics. Surprisingly, using later reasoning
steps does not always improve classification. When additional context is
unhelpful, earlier representations resemble later ones more, suggesting LLMs
encode key information early. This implies reasoning can often stop early
without loss. To test this, we conduct early stopping experiments, showing that
truncating CoT reasoning still improves performance over not using CoT at all,
though a gap remains compared to full reasoning. However, approaches like
supervised learning or reinforcement learning designed to shorten CoT chains
could leverage our classifier's guidance to identify when early stopping is
effective. Our findings provide insights that may support such methods, helping
to optimize CoT's efficiency while preserving its benefits.
Hanfu-Bench: A Multimodal Benchmark on Cross-Temporal Cultural
  Understanding and Transcreation
Authors: name: Li Zhou, name: Lutong Yu, name: Dongchu Xie, name: Shaohuan Cheng, name: Wenyan Li, name: Haizhou Li
Published: 2025-06-02 11:43:46

Culture is a rich and dynamic domain that evolves across both geography and
time. However, existing studies on cultural understanding with vision-language
models (VLMs) primarily emphasize geographic diversity, often overlooking the
critical temporal dimensions. To bridge this gap, we introduce Hanfu-Bench, a
novel, expert-curated multimodal dataset. Hanfu, a traditional garment spanning
ancient Chinese dynasties, serves as a representative cultural heritage that
reflects the profound temporal aspects of Chinese culture while remaining
highly popular in Chinese contemporary society. Hanfu-Bench comprises two core
tasks: cultural visual understanding and cultural image transcreation.The
former task examines temporal-cultural feature recognition based on single- or
multi-image inputs through multiple-choice visual question answering, while the
latter focuses on transforming traditional attire into modern designs through
cultural element inheritance and modern context adaptation. Our evaluation
shows that closed VLMs perform comparably to non-experts on visual cutural
understanding but fall short by 10\% to human experts, while open VLMs lags
further behind non-experts. For the transcreation task, multi-faceted human
evaluation indicates that the best-performing model achieves a success rate of
only 42\%. Our benchmark provides an essential testbed, revealing significant
challenges in this new direction of temporal cultural understanding and
creative adaptation.
Deep Video Discovery: Agentic Search with Tool Use for Long-form Video
  Understanding
Authors: name: Xiaoyi Zhang, name: Zhaoyang Jia, name: Zongyu Guo, name: Jiahao Li, name: Bin Li, name: Houqiang Li, name: Yan Lu
Published: 2025-05-23 16:37:36

Long-form video understanding presents significant challenges due to
extensive temporal-spatial complexity and the difficulty of question answering
under such extended contexts. While Large Language Models (LLMs) have
demonstrated considerable advancements in video analysis capabilities and long
context handling, they continue to exhibit limitations when processing
information-dense hour-long videos. To overcome such limitations, we propose
the Deep Video Discovery agent to leverage an agentic search strategy over
segmented video clips. Different from previous video agents manually designing
a rigid workflow, our approach emphasizes the autonomous nature of agents. By
providing a set of search-centric tools on multi-granular video database, our
DVD agent leverages the advanced reasoning capability of LLM to plan on its
current observation state, strategically selects tools, formulates appropriate
parameters for actions, and iteratively refines its internal reasoning in light
of the gathered information. We perform comprehensive evaluation on multiple
long video understanding benchmarks that demonstrates the advantage of the
entire system design. Our DVD agent achieves SOTA performance, significantly
surpassing prior works by a large margin on the challenging LVBench dataset.
Comprehensive ablation studies and in-depth tool analyses are also provided,
yielding insights to further advance intelligent agents tailored for long-form
video understanding tasks. The code will be released later.
Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning
Authors: name: Shelly Bensal, name: Umar Jamil, name: Christopher Bryant, name: Melisa Russak, name: Kiran Kamble, name: Dmytro Mozolevskyi, name: Muayad Ali, name: Waseem AlShikh
Published: 2025-05-30 15:49:42

We explore a method for improving the performance of large language models
through self-reflection and reinforcement learning. By incentivizing the model
to generate better self-reflections when it answers incorrectly, we demonstrate
that a model's ability to solve complex, verifiable tasks can be enhanced even
when generating synthetic data is infeasible and only binary feedback is
available. Our framework operates in two stages: first, upon failing a given
task, the model generates a self-reflective commentary analyzing its previous
attempt; second, the model is given another attempt at the task with the
self-reflection in context. If the subsequent attempt succeeds, the tokens
generated during the self-reflection phase are rewarded. Our experimental
results show substantial performance gains across a variety of model
architectures, as high as 34.7% improvement at math equation writing and 18.1%
improvement at function calling. Notably, smaller fine-tuned models (1.5
billion to 7 billion parameters) outperform models in the same family that are
10 times larger. Our novel paradigm is thus an exciting pathway to more useful
and reliable language models that can self-improve on challenging tasks with
limited external feedback.
CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning
  Capabilities of VLMs
Authors: name: Ai Jian, name: Weijie Qiu, name: Xiaokun Wang, name: Peiyu Wang, name: Yunzhuo Hao, name: Jiangbo Pei, name: Yichen Wei, name: Yi Peng, name: Xuchen Song
Published: 2025-05-30 01:34:25

Vision-Language Models (VLMs) have demonstrated remarkable progress in
multimodal understanding, yet their capabilities for scientific reasoning
remains inadequately assessed. Current multimodal benchmarks predominantly
evaluate generic image comprehension or text-driven reasoning, lacking
authentic scientific contexts that require domain-specific knowledge
integration with visual evidence analysis. To fill this gap, we present CSVQA,
a diagnostic multimodal benchmark specifically designed for evaluating
scientific reasoning through domain-grounded visual question answering.Our
benchmark features 1,378 carefully constructed question-answer pairs spanning
diverse STEM disciplines, each demanding domain knowledge, integration of
visual evidence, and higher-order reasoning. Compared to prior multimodal
benchmarks, CSVQA places greater emphasis on real-world scientific content and
complex reasoning.We additionally propose a rigorous evaluation protocol to
systematically assess whether model predictions are substantiated by valid
intermediate reasoning steps based on curated explanations. Our comprehensive
evaluation of 15 VLMs on this benchmark reveals notable performance
disparities, as even the top-ranked proprietary model attains only 49.6\%
accuracy.This empirical evidence underscores the pressing need for advancing
scientific reasoning capabilities in VLMs. Our CSVQA is released at
https://huggingface.co/datasets/Skywork/CSVQA.