Orionnnn/hub-stats huggingface dataset:
Deep Think with Confidence
Authors: name: Yichao Fu, name: Xuewei Wang, name: Yuandong Tian, name: Jiawei Zhao
Published: 2025-08-21 05:48:38

Large Language Models (LLMs) have shown great potential in reasoning tasks
through test-time scaling methods like self-consistency with majority voting.
However, this approach often leads to diminishing returns in accuracy and high
computational overhead. To address these challenges, we introduce Deep Think
with Confidence (DeepConf), a simple yet powerful method that enhances both
reasoning efficiency and performance at test time. DeepConf leverages
model-internal confidence signals to dynamically filter out low-quality
reasoning traces during or after generation. It requires no additional model
training or hyperparameter tuning and can be seamlessly integrated into
existing serving frameworks. We evaluate DeepConf across a variety of reasoning
tasks and the latest open-source models, including Qwen 3 and GPT-OSS series.
Notably, on challenging benchmarks such as AIME 2025, DeepConf@512 achieves up
to 99.9% accuracy and reduces generated tokens by up to 84.7% compared to full
parallel thinking.
Dissecting Tool-Integrated Reasoning: An Empirical Study and Analysis
Authors: name: Yufeng Zhao, name: Junnan Liu, name: Hongwei Liu, name: Dongsheng Zhu, name: Yuan Shen, name: Songyang Zhang, name: Kai Chen
Published: 2025-08-21 17:50:24

Large Language Models (LLMs) have made significant strides in reasoning tasks
through methods like chain-of-thought (CoT) reasoning. However, they often fall
short in tasks requiring precise computations. Tool-Integrated Reasoning (TIR)
has emerged as a solution by incorporating external tools into the reasoning
process. Nevertheless, the generalization of TIR in improving the reasoning
ability of LLM is still unclear. Additionally, whether TIR has improved the
model's reasoning behavior and helped the model think remains to be studied. We
introduce ReasonZoo, a comprehensive benchmark encompassing nine diverse
reasoning categories, to evaluate the effectiveness of TIR across various
domains. Additionally, we propose two novel metrics, Performance-Aware Cost
(PAC) and Area Under the Performance-Cost Curve (AUC-PCC), to assess reasoning
efficiency. Our empirical evaluation demonstrates that TIR-enabled models
consistently outperform their non-TIR counterparts in both mathematical and
non-mathematical tasks. Furthermore, TIR enhances reasoning efficiency, as
evidenced by improved PAC and AUC-PCC, indicating reduced overthinking and more
streamlined reasoning. These findings underscore the domain-general benefits of
TIR and its potential to advance LLM capabilities in complex reasoning tasks.
Virtuous Machines: Towards Artificial General Science
Authors: name: Gabrielle Wehr, name: Reuben Rideaux, name: Amaya J. Fox, name: David R. Lightfoot, name: Jason Tangen, name: Jason B. Mattingley, name: Shane E. Ehrhardt
Published: 2025-08-19 00:35:56

Artificial intelligence systems are transforming scientific discovery by
accelerating specific research tasks, from protein structure prediction to
materials design, yet remain confined to narrow domains requiring substantial
human oversight. The exponential growth of scientific literature and increasing
domain specialisation constrain researchers' capacity to synthesise knowledge
across disciplines and develop unifying theories, motivating exploration of
more general-purpose AI systems for science. Here we show that a
domain-agnostic, agentic AI system can independently navigate the scientific
workflow - from hypothesis generation through data collection to manuscript
preparation. The system autonomously designed and executed three psychological
studies on visual working memory, mental rotation, and imagery vividness,
executed one new online data collection with 288 participants, developed
analysis pipelines through 8-hour+ continuous coding sessions, and produced
completed manuscripts. The results demonstrate the capability of AI scientific
discovery pipelines to conduct non-trivial research with theoretical reasoning
and methodological rigour comparable to experienced researchers, though with
limitations in conceptual nuance and theoretical interpretation. This is a step
toward embodied AI that can test hypotheses through real-world experiments,
accelerating discovery by autonomously exploring regions of scientific space
that human cognitive and resource constraints might otherwise leave unexplored.
It raises important questions about the nature of scientific understanding and
the attribution of scientific credit.
FLARE: Fast Low-rank Attention Routing Engine
Authors: name: Vedant Puri, name: Aditya Joglekar, name: Kevin Ferguson, name: Yu-hsuan Chen, name: Yongjie Jessica Zhang, name: Levent Burak Kara
Published: 2025-08-18 03:00:55

The quadratic complexity of self-attention limits its applicability and
scalability on large unstructured meshes. We introduce Fast Low-rank Attention
Routing Engine (FLARE), a linear complexity self-attention mechanism that
routes attention through fixed-length latent sequences. Each attention head
performs global communication among N tokens by projecting the input sequence
onto a fixed length latent sequence of M ll N tokens using learnable query
tokens. By routing attention through a bottleneck sequence, FLARE learns a
low-rank form of attention that can be applied at O(NM) cost. FLARE not only
scales to unprecedented problem sizes, but also delivers superior accuracy
compared to state-of-the-art neural PDE surrogates across diverse benchmarks.
We also release a new additive manufacturing dataset to spur further research.
Our code is available at https://github.com/vpuri3/FLARE.py.
Leuvenshtein: Efficient FHE-based Edit Distance Computation with Single
  Bootstrap per Cell
Authors: name: Wouter Legiest, name: Jan-Pieter D'Anvers, name: Bojan Spasic, name: Nam-Luc Tran, name: Ingrid Verbauwhede
Published: 2025-08-20 09:40:06

This paper presents a novel approach to calculating the Levenshtein (edit)
distance within the framework of Fully Homomorphic Encryption (FHE),
specifically targeting third-generation schemes like TFHE. Edit distance
computations are essential in applications across finance and genomics, such as
DNA sequence alignment. We introduce an optimised algorithm that significantly
reduces the cost of edit distance calculations called Leuvenshtein. This
algorithm specifically reduces the number of programmable bootstraps (PBS)
needed per cell of the calculation, lowering it from approximately 94
operations -- required by the conventional Wagner-Fisher algorithm -- to just
1. Additionally, we propose an efficient method for performing equality checks
on characters, reducing ASCII character comparisons to only 2 PBS operations.
Finally, we explore the potential for further performance improvements by
utilising preprocessing when one of the input strings is unencrypted. Our
Leuvenshtein achieves up to 278times faster performance compared to the best
available TFHE implementation and up to 39times faster than an optimised
implementation of the Wagner-Fisher algorithm. Moreover, when offline
preprocessing is possible due to the presence of one unencrypted input on the
server side, an additional 3times speedup can be achieved.
MCP-Universe: Benchmarking Large Language Models with Real-World Model
  Context Protocol Servers
Authors: name: Ziyang Luo, name: Zhiqi Shen, name: Wenzhuo Yang, name: Zirui Zhao, name: Prathyusha Jwalapuram, name: Amrita Saha, name: Doyen Sahoo, name: Silvio Savarese, name: Caiming Xiong, name: Junnan Li
Published: 2025-08-20 13:28:58

The Model Context Protocol has emerged as a transformative standard for
connecting large language models to external data sources and tools, rapidly
gaining adoption across major AI providers and development platforms. However,
existing benchmarks are overly simplistic and fail to capture real application
challenges such as long-horizon reasoning and large, unfamiliar tool spaces. To
address this critical gap, we introduce MCP-Universe, the first comprehensive
benchmark specifically designed to evaluate LLMs in realistic and hard tasks
through interaction with real-world MCP servers. Our benchmark encompasses 6
core domains spanning 11 different MCP servers: Location Navigation, Repository
Management, Financial Analysis, 3D Design, Browser Automation, and Web
Searching. To ensure rigorous evaluation, we implement execution-based
evaluators, including format evaluators for agent format compliance, static
evaluators for time-invariant content matching, and dynamic evaluators that
automatically retrieve real-time ground truth for temporally sensitive tasks.
Through extensive evaluation of leading LLMs, we find that even SOTA models
such as GPT-5 (43.72%), Grok-4 (33.33%) and Claude-4.0-Sonnet (29.44%) exhibit
significant performance limitations. In addition, our benchmark poses a
significant long-context challenge for LLM agents, as the number of input
tokens increases rapidly with the number of interaction steps. Moreover, it
introduces an unknown-tools challenge, as LLM agents often lack familiarity
with the precise usage of the MCP servers. Notably, enterprise-level agents
like Cursor cannot achieve better performance than standard ReAct frameworks.
Beyond evaluation, we open-source our extensible evaluation framework with UI
support, enabling researchers and practitioners to seamlessly integrate new
agents and MCP servers while fostering innovation in the rapidly evolving MCP
ecosystem.
ViExam: Are Vision Language Models Better than Humans on Vietnamese
  Multimodal Exam Questions?
Authors: name: Vy Tuong Dang, name: An Vo, name: Quang Tau, name: Duc Dm, name: Daeyoung Kim
Published: 2025-08-19 09:31:18

Vision language models (VLMs) demonstrate remarkable capabilities on English
multimodal tasks, but their performance on low-resource languages with
genuinely multimodal educational content remains largely unexplored. In this
work, we test how VLMs perform on Vietnamese educational assessments,
investigating whether VLMs trained predominantly on English data can handle
real-world cross-lingual multimodal reasoning. Our work presents the first
comprehensive evaluation of VLM capabilities on multimodal Vietnamese exams
through proposing ViExam, a benchmark containing 2,548 multimodal questions. We
find that state-of-the-art VLMs achieve only 57.74% while open-source models
achieve 27.70% mean accuracy across 7 academic domains, including Mathematics,
Physics, Chemistry, Biology, Geography, Driving Test, and IQ Test. Most VLMs
underperform average human test-takers (66.54%), with only the thinking VLM o3
(74.07%) exceeding human average performance, yet still falling substantially
short of human best performance (99.60%). Cross-lingual prompting with English
instructions while maintaining Vietnamese content fails to improve performance,
decreasing accuracy by 1 percentage point for SOTA VLMs. Human-in-the-loop
collaboration can partially improve VLM performance by 5 percentage points.
Code and data are available at: https://vi-exam.github.io.
On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised
  Fine-Tuning and Reinforcement Learning via Dynamic Weighting
Authors: name: Wenhao Zhang, name: Yuexiang Xie, name: Yuchang Sun, name: Yanxi Chen, name: Guoyin Wang, name: Yaliang Li, name: Bolin Ding, name: Jingren Zhou
Published: 2025-08-15 11:20:03

Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) are two
prominent post-training paradigms for refining the capabilities and aligning
the behavior of Large Language Models (LLMs). Existing approaches that
integrate SFT and RL often face the risk of disrupting established model
patterns and inducing overfitting to expert data. To address this, we present a
novel investigation into the unified view of SFT and RL through an off-policy
versus on-policy lens. We propose CHORD, a framework for the Controllable
Harmonization of On- and Off-Policy Reinforcement Learning via Dynamic
Weighting, which reframes SFT not as a separate stage but as a dynamically
weighted auxiliary objective within the on-policy RL process. Based on an
analysis of off-policy expert data's influence at both holistic and granular
levels, we incorporate a dual-control mechanism in CHORD. Specifically, the
framework first employs a global coefficient to holistically guide the
transition from off-policy imitation to on-policy exploration, and then applies
a token-wise weighting function that enables granular learning from expert
tokens, which preserves on-policy exploration and mitigates disruption from
off-policy data. We conduct extensive experiments on widely used benchmarks,
providing empirical evidence that CHORD achieves a stable and efficient
learning process. By effectively harmonizing off-policy expert data with
on-policy exploration, CHORD demonstrates significant improvements over
baselines. We release the implementation at
https://github.com/modelscope/Trinity-RFT/tree/main/examples/mix_chord to
inspire further research.
From AI for Science to Agentic Science: A Survey on Autonomous
  Scientific Discovery
Authors: name: Jiaqi Wei, name: Yuejin Yang, name: Xiang Zhang, name: Yuhan Chen, name: Xiang Zhuang, name: Zhangyang Gao, name: Dongzhan Zhou, name: Guangshuai Wang, name: Zhiqiang Gao, name: Juntai Cao, name: Zijie Qiu, name: Xuming He, name: Qiang Zhang, name: Chenyu You, name: Shuangjia Zheng, name: Ning Ding, name: Wanli Ouyang, name: Nanqing Dong, name: Yu Cheng, name: Siqi Sun, name: Lei Bai, name: Bowen Zhou
Published: 2025-08-18 05:25:54

Artificial intelligence (AI) is reshaping scientific discovery, evolving from
specialized computational tools into autonomous research partners. We position
Agentic Science as a pivotal stage within the broader AI for Science paradigm,
where AI systems progress from partial assistance to full scientific agency.
Enabled by large language models (LLMs), multimodal systems, and integrated
research platforms, agentic AI shows capabilities in hypothesis generation,
experimental design, execution, analysis, and iterative refinement -- behaviors
once regarded as uniquely human. This survey provides a domain-oriented review
of autonomous scientific discovery across life sciences, chemistry, materials
science, and physics. We unify three previously fragmented perspectives --
process-oriented, autonomy-oriented, and mechanism-oriented -- through a
comprehensive framework that connects foundational capabilities, core
processes, and domain-specific realizations. Building on this framework, we (i)
trace the evolution of AI for Science, (ii) identify five core capabilities
underpinning scientific agency, (iii) model discovery as a dynamic four-stage
workflow, (iv) review applications across the above domains, and (v) synthesize
key challenges and future opportunities. This work establishes a
domain-oriented synthesis of autonomous scientific discovery and positions
Agentic Science as a structured paradigm for advancing AI-driven research.
Refining Contrastive Learning and Homography Relations for Multi-Modal
  Recommendation
Authors: name: Shouxing Ma, name: Yawen Zeng, name: Shiqing Wu, name: Guandong Xu
Published: 2025-08-19 11:35:48

Multi-modal recommender system focuses on utilizing rich modal information (
i.e., images and textual descriptions) of items to improve recommendation
performance. The current methods have achieved remarkable success with the
powerful structure modeling capability of graph neural networks. However, these
methods are often hindered by sparse data in real-world scenarios. Although
contrastive learning and homography ( i.e., homogeneous graphs) are employed to
address the data sparsity challenge, existing methods still suffer two main
limitations: 1) Simple multi-modal feature contrasts fail to produce effective
representations, causing noisy modal-shared features and loss of valuable
information in modal-unique features; 2) The lack of exploration of the
homograph relations between user interests and item co-occurrence results in
incomplete mining of user-item interplay.
  To address the above limitations, we propose a novel framework for
REfining multi-modAl contRastive learning
and hoMography relations (REARM). Specifically, we complement
multi-modal contrastive learning by employing meta-network and orthogonal
constraint strategies, which filter out noise in modal-shared features and
retain recommendation-relevant information in modal-unique features. To mine
homogeneous relationships effectively, we integrate a newly constructed user
interest graph and an item co-occurrence graph with the existing user
co-occurrence and item semantic graphs for graph learning. The extensive
experiments on three real-world datasets demonstrate the superiority of REARM
to various state-of-the-art baselines. Our visualization further shows an
improvement made by REARM in distinguishing between modal-shared and
modal-unique features. Code is available
https://github.com/MrShouxingMa/REARM{here}.
mSCoRe: a Multilingual and Scalable Benchmark for Skill-based
  Commonsense Reasoning
Authors: name: Nghia Trung Ngo, name: Franck Dernoncourt, name: Thien Huu Nguyen
Published: 2025-08-13 18:59:02

Recent advancements in reasoning-reinforced Large Language Models (LLMs) have
shown remarkable capabilities in complex reasoning tasks. However, the
mechanism underlying their utilization of different human reasoning skills
remains poorly investigated, especially for multilingual commonsense reasoning
that involves everyday knowledge across different languages and cultures. To
address this gap, we propose a Multilingual and Scalable Benchmark for
Skill-based Commonsense Reasoning (mSCoRe).
Our benchmark incorporates three key components that are designed to
systematically evaluate LLM's reasoning capabilities, including: (1) a novel
taxonomy of reasoning skills that enables fine-grained analysis of models'
reasoning processes, (2) a robust data synthesis pipeline tailored specifically
for commonsense reasoning evaluation, and (3) a complexity scaling framework
allowing task difficulty to scale dynamically alongside future improvements in
LLM abilities. Extensive experiments on eights state-of-the-art LLMs of varying
sizes and training approaches demonstrate that mSCoRe remains
significantly challenging for current models, particularly at higher complexity
levels. Our results reveal the limitations of such reasoning-reinforced models
when confronted with nuanced multilingual general and cultural commonsense. We
further provide detailed analysis on the models' reasoning processes,
suggesting future directions for improving multilingual commonsense reasoning
capabilities.
Local Scale Equivariance with Latent Deep Equilibrium Canonicalizer
Authors: name: Md Ashiqur Rahman, name: Chiao-An Yang, name: Michael N. Cheng, name: Lim Jun Hao, name: Jeremiah Jiang, name: Teck-Yian Lim, name: Raymond A. Yeh
Published: 2025-08-19 18:21:59

Scale variation is a fundamental challenge in computer vision. Objects of the
same class can have different sizes, and their perceived size is further
affected by the distance from the camera. These variations are local to the
objects, i.e., different object sizes may change differently within the same
image. To effectively handle scale variations, we present a deep equilibrium
canonicalizer (DEC) to improve the local scale equivariance of a model. DEC can
be easily incorporated into existing network architectures and can be adapted
to a pre-trained model. Notably, we show that on the competitive ImageNet
benchmark, DEC improves both model performance and local scale consistency
across four popular pre-trained deep-nets, e.g., ViT, DeiT, Swin, and BEiT. Our
code is available at https://github.com/ashiq24/local-scale-equivariance.
Quantization Meets dLLMs: A Systematic Study of Post-training
  Quantization for Diffusion LLMs
Authors: name: Haokun Lin, name: Haobo Xu, name: Yichen Wu, name: Ziyu Guo, name: Renrui Zhang, name: Zhichao Lu, name: Ying Wei, name: Qingfu Zhang, name: Zhenan Sun
Published: 2025-08-20 17:59:51

Recent advances in diffusion large language models (dLLMs) have introduced a
promising alternative to autoregressive (AR) LLMs for natural language
generation tasks, leveraging full attention and denoising-based decoding
strategies. However, the deployment of these models on edge devices remains
challenging due to their massive parameter scale and high resource demands.
While post-training quantization (PTQ) has emerged as a widely adopted
technique for compressing AR LLMs, its applicability to dLLMs remains largely
unexplored. In this work, we present the first systematic study on quantizing
diffusion-based language models. We begin by identifying the presence of
activation outliers, characterized by abnormally large activation values that
dominate the dynamic range. These outliers pose a key challenge to low-bit
quantization, as they make it difficult to preserve precision for the majority
of values. More importantly, we implement state-of-the-art PTQ methods and
conduct a comprehensive evaluation across multiple task types and model
variants. Our analysis is structured along four key dimensions: bit-width,
quantization method, task category, and model type. Through this
multi-perspective evaluation, we offer practical insights into the quantization
behavior of dLLMs under different configurations. We hope our findings provide
a foundation for future research in efficient dLLM deployment. All codes and
experimental setups will be released to support the community.
DuPO: Enabling Reliable LLM Self-Verification via Dual Preference
  Optimization
Authors: name: Shuaijie She, name: Yu Bao, name: Yu Lu, name: Lu Xu, name: Tao Li, name: Wenhao Zhu, name: Shujian Huang, name: Shanbo Cheng, name: Lu Lu, name: Yuxuan Wang
Published: 2025-08-20 06:31:18

We present DuPO, a dual learning-based preference optimization framework that
generates annotation-free feedback via a generalized duality. DuPO addresses
two key limitations: Reinforcement Learning with Verifiable Rewards (RLVR)'s
reliance on costly labels and applicability restricted to verifiable tasks, and
traditional dual learning's restriction to strictly dual task pairs (e.g.,
translation and back-translation). Specifically, DuPO decomposes a primal
task's input into known and unknown components, then constructs its dual task
to reconstruct the unknown part using the primal output and known information
(e.g., reversing math solutions to recover hidden variables), broadening
applicability to non-invertible tasks. The quality of this reconstruction
serves as a self-supervised reward to optimize the primal task, synergizing
with LLMs' ability to instantiate both tasks via a single model. Empirically,
DuPO achieves substantial gains across diverse tasks: it enhances the average
translation quality by 2.13 COMET over 756 directions, boosts the mathematical
reasoning accuracy by an average of 6.4 points on three challenge benchmarks,
and enhances performance by 9.3 points as an inference-time reranker (trading
computation for accuracy). These results position DuPO as a scalable, general,
and annotation-free paradigm for LLM optimization.