Orionnnn/hub-stats huggingface dataset:
ToolRM: Outcome Reward Models for Tool-Calling Large Language Models
Authors: name: Mayank Agarwal, name: Ibrahim Abdelaziz, name: Kinjal Basu, name: Merve Unuvar, name: Luis A. Lastras, name: Yara Rizk, name: Pavan Kapanipathi
Published: 2025-09-15 14:17:17

As large language models (LLMs) increasingly interact with external tools,
reward modeling for tool use has become a critical yet underexplored area.
Existing reward models, trained primarily on natural language outputs, struggle
to evaluate tool-based reasoning and execution. To quantify this gap, we
introduce FC-RewardBench, the first benchmark designed to systematically assess
reward models' performance in tool-calling scenarios. Our analysis shows that
current reward models often miss key signals of effective tool use,
highlighting the need for domain-specific modeling. To address this, we propose
a training framework for outcome-based reward models using data synthesized
from permissively licensed, open-weight LLMs. We train models ranging from 1.7B
to 14B parameters and evaluate them across seven out-of-domain benchmarks.
These models consistently outperform general-purpose baselines, achieving up to
25\% average improvement in downstream task performance and enabling
data-efficient fine-tuning through reward-guided filtering.
ClaimIQ at CheckThat! 2025: Comparing Prompted and Fine-Tuned Language
  Models for Verifying Numerical Claims
Authors: name: Anirban Saha Anik, name: Md Fahimul Kabir Chowdhury, name: Andrew Wyckoff, name: Sagnik Ray Choudhury
Published: 2025-09-15 01:03:09

This paper presents our system for Task 3 of the CLEF 2025 CheckThat! Lab,
which focuses on verifying numerical and temporal claims using retrieved
evidence. We explore two complementary approaches: zero-shot prompting with
instruction-tuned large language models (LLMs) and supervised fine-tuning using
parameter-efficient LoRA. To enhance evidence quality, we investigate several
selection strategies, including full-document input and top-k sentence
filtering using BM25 and MiniLM. Our best-performing model LLaMA fine-tuned
with LoRA achieves strong performance on the English validation set. However, a
notable drop in the test set highlights a generalization challenge. These
findings underscore the importance of evidence granularity and model adaptation
for robust numerical fact verification.
LongEmotion: Measuring Emotional Intelligence of Large Language Models
  in Long-Context Interaction
Authors: name: Weichu Liu, name: Jing Xiong, name: Yuxuan Hu, name: Zixuan Li, name: Minghuan Tan, name: Ningning Mao, name: Chenyang Zhao, name: Zhongwei Wan, name: Chaofan Tao, name: Wendong Xu, name: Hui Shen, name: Chengming Li, name: Lingpeng Kong, name: Ngai Wong
Published: 2025-09-09 05:32:45

Large language models (LLMs) make significant progress in Emotional
Intelligence (EI) and long-context understanding. However, existing benchmarks
tend to overlook certain aspects of EI in long-context scenarios, especially
under realistic, practical settings where interactions are lengthy, diverse,
and often noisy. To move towards such realistic settings, we present
LongEmotion, a benchmark specifically designed for long-context EI tasks. It
covers a diverse set of tasks, including Emotion Classification, Emotion
Detection, Emotion QA, Emotion Conversation, Emotion Summary, and Emotion
Expression. On average, the input length for these tasks reaches 8,777 tokens,
with long-form generation required for Emotion Expression. To enhance
performance under realistic constraints, we incorporate Retrieval-Augmented
Generation (RAG) and Collaborative Emotional Modeling (CoEM), and compare them
with standard prompt-based methods. Unlike conventional approaches, our RAG
method leverages both the conversation context and the large language model
itself as retrieval sources, avoiding reliance on external knowledge bases. The
CoEM method further improves performance by decomposing the task into five
stages, integrating both retrieval augmentation and limited knowledge
injection. Experimental results show that both RAG and CoEM consistently
enhance EI-related performance across most long-context tasks, advancing LLMs
toward more practical and real-world EI applications. Furthermore, we conducted
a comparative case study experiment on the GPT series to demonstrate the
differences among various models in terms of EI. Code is available on GitHub at
https://github.com/LongEmotion/LongEmotion, and the project page can be found
at https://longemotion.github.io/.
SearchInstruct: Enhancing Domain Adaptation via Retrieval-Based
  Instruction Dataset Creation
Authors: name: Iman Barati, name: Mostafa Amiri, name: Heshaam Faili
Published: 2025-09-12 21:50:39

Supervised Fine-Tuning (SFT) is essential for training large language models
(LLMs), significantly enhancing critical capabilities such as instruction
following and in-context learning. Nevertheless, creating suitable training
datasets tailored for specific domains remains challenging due to unique domain
constraints and data scarcity. In this paper, we propose SearchInstruct, an
innovative method explicitly designed to construct high quality instruction
datasets for SFT. Our approach begins with a limited set of domain specific,
human generated questions, which are systematically expanded using a large
language model. Subsequently, domain relevant resources are dynamically
retrieved to generate accurate and contextually appropriate answers for each
augmented question. Experimental evaluation demonstrates that SearchInstruct
enhances both the diversity and quality of SFT datasets, leading to measurable
improvements in LLM performance within specialized domains. Additionally, we
show that beyond dataset generation, the proposed method can also effectively
facilitate tasks such as model editing, enabling efficient updates to existing
models. To facilitate reproducibility and community adoption, we provide full
implementation details, the complete set of generated instruction response
pairs, and the source code in a publicly accessible Git repository:
[https://github.com/mostafaamiri/SearchInstruct](https://github.com/mostafaamiri/SearchInstruct)
Dr.V: A Hierarchical Perception-Temporal-Cognition Framework to Diagnose
  Video Hallucination by Fine-grained Spatial-Temporal Grounding
Authors: name: Meng Luo, name: Shengqiong Wu, name: Liqiang Jing, name: Tianjie Ju, name: Li Zheng, name: Jinxiang Lai, name: Tianlong Wu, name: Xinya Du, name: Jian Li, name: Siyuan Yan, name: Jiebo Luo, name: William Yang Wang, name: Hao Fei, name: Mong-Li Lee, name: Wynne Hsu
Published: 2025-09-15 12:39:19

Recent advancements in large video models (LVMs) have significantly enhance
video understanding. However, these models continue to suffer from
hallucinations, producing content that conflicts with input videos. To address
this issue, we propose Dr.V, a hierarchical framework covering perceptive,
temporal, and cognitive levels to diagnose video hallucination by fine-grained
spatial-temporal grounding. Dr.V comprises of two key components: a benchmark
dataset Dr.V-Bench and a satellite video agent Dr.V-Agent. Dr.V-Bench includes
10k instances drawn from 4,974 videos spanning diverse tasks, each enriched
with detailed spatial-temporal annotation. Dr.V-Agent detects hallucinations in
LVMs by systematically applying fine-grained spatial-temporal grounding at the
perceptive and temporal levels, followed by cognitive level reasoning. This
step-by-step pipeline mirrors human-like video comprehension and effectively
identifies hallucinations. Extensive experiments demonstrate that Dr.V-Agent is
effective in diagnosing hallucination while enhancing interpretability and
reliability, offering a practical blueprint for robust video understanding in
real-world scenarios. All our data and code are available at
https://github.com/Eurekaleo/Dr.V.
Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language
  Models
Authors: name: Pu Jian, name: Junhong Wu, name: Wei Sun, name: Chen Wang, name: Shuo Ren, name: Jiajun Zhang
Published: 2025-09-15 16:57:25

Recent advances in text-only "slow-thinking" reasoning have prompted efforts
to transfer this capability to vision-language models (VLMs), for training
visual reasoning models (VRMs). owever, such transfer faces critical
challenges: Effective "slow thinking" in VRMs requires visual
reflection, the ability to check the reasoning process based on visual
information. Through quantitative analysis, we observe that current VRMs
exhibit limited visual reflection, as their attention to visual information
diminishes rapidly with longer generated responses. To address this challenge,
we propose a new VRM Reflection-V, which enhances visual reflection
based on reasoning data construction for cold-start and reward design for
reinforcement learning (RL). Firstly, we construct vision-centered reasoning
data by leveraging an agent that interacts between VLMs and reasoning LLMs,
enabling cold-start learning of visual reflection patterns. Secondly, a visual
attention based reward model is employed during RL to encourage reasoning based
on visual information. Therefore, Reflection-V demonstrates
significant improvements across multiple visual reasoning benchmarks.
Furthermore, Reflection-V maintains a stronger and more consistent
reliance on visual information during visual reasoning, indicating effective
enhancement in visual reflection capabilities.
Locality in Image Diffusion Models Emerges from Data Statistics
Authors: name: Artem Lukoianov, name: Chenyang Yuan, name: Justin Solomon, name: Vincent Sitzmann
Published: 2025-09-11 17:59:08

Among generative models, diffusion models are uniquely intriguing due to the
existence of a closed-form optimal minimizer of their training objective, often
referred to as the optimal denoiser. However, diffusion using this optimal
denoiser merely reproduces images in the training set and hence fails to
capture the behavior of deep diffusion models. Recent work has attempted to
characterize this gap between the optimal denoiser and deep diffusion models,
proposing analytical, training-free models that can generate images that
resemble those generated by a trained UNet. The best-performing method
hypothesizes that shift equivariance and locality inductive biases of
convolutional neural networks are the cause of the performance gap, hence
incorporating these assumptions into its analytical model. In this work, we
present evidence that the locality in deep diffusion models emerges as a
statistical property of the image dataset, not due to the inductive bias of
convolutional neural networks. Specifically, we demonstrate that an optimal
parametric linear denoiser exhibits similar locality properties to the deep
neural denoisers. We further show, both theoretically and experimentally, that
this locality arises directly from the pixel correlations present in natural
image datasets. Finally, we use these insights to craft an analytical denoiser
that better matches scores predicted by a deep diffusion model than the prior
expert-crafted alternative.
Lost in Embeddings: Information Loss in Vision-Language Models
Authors: name: Wenyan Li, name: Raphael Tang, name: Chengzu Li, name: Caiqi Zhang, name: Ivan Vulić, name: Anders Søgaard
Published: 2025-09-15 14:38:06

Vision--language models (VLMs) often process visual inputs through a
pretrained vision encoder, followed by a projection into the language model's
embedding space via a connector component. While crucial for modality fusion,
the potential information loss induced by this projection step and its direct
impact on model capabilities remain understudied. We introduce two
complementary approaches to examine and quantify this loss by analyzing the
latent representation space. First, we evaluate semantic information
preservation by analyzing changes in k-nearest neighbor relationships between
image representations, before and after projection. Second, we directly measure
information loss by reconstructing visual embeddings from the projected
representation, localizing loss at an image patch level. Experiments reveal
that connectors substantially distort the local geometry of visual
representations, with k-nearest neighbors diverging by 40--60\%
post-projection, correlating with degradation in retrieval performance. The
patch-level embedding reconstruction provides interpretable insights for model
behavior on visually grounded question-answering tasks, finding that areas of
high information loss reliably predict instances where models struggle.
Nav-R1: Reasoning and Navigation in Embodied Scenes
Authors: name: Qingxiang Liu, name: Ting Huang, name: Zeyu Zhang, name: Hao Tang
Published: 2025-09-13 16:31:03

Embodied navigation requires agents to integrate perception, reasoning, and
action for robust interaction in complex 3D environments. Existing approaches
often suffer from incoherent and unstable reasoning traces that hinder
generalization across diverse environments, and difficulty balancing
long-horizon semantic reasoning with low-latency control for real-time
navigation. To address these challenges, we propose Nav-R1, an embodied
foundation model that unifies reasoning in embodied environments. We first
construct Nav-CoT-110K, a large-scale dataset of step-by-step Chains-of-Thought
(CoT) for embodied tasks, which enables cold-start initialization with
structured reasoning. Building on this foundation, we design a GRPO-based
reinforcement learning framework with three complementary rewards: format,
understanding, and navigation, to improve structural adherence, semantic
grounding, and path fidelity. Furthermore, we introduce a Fast-in-Slow
reasoning paradigm, decoupling deliberate semantic reasoning from low-latency
reactive control for efficient yet coherent navigation. Extensive evaluations
on embodied AI benchmarks demonstrate that Nav-R1 consistently outperforms
strong baselines, with over 8% average improvement in reasoning and navigation
performance. Real-world deployment on a mobile robot further validates its
robustness under limited onboard resources. Code:
https://github.com/AIGeeksGroup/Nav-R1. Website:
https://aigeeksgroup.github.io/Nav-R1.
GAPrune: Gradient-Alignment Pruning for Domain-Aware Embeddings
Authors: name: Yixuan Tang, name: Yi Yang
Published: 2025-09-13 15:03:37

Domain-specific embedding models have shown promise for applications that
require specialized semantic understanding, such as coding agents and financial
retrieval systems, often achieving higher performance gains than general
models. However, state-of-the-art embedding models are typically based on LLMs,
which contain billions of parameters, making deployment challenging in
resource-constrained environments. Model compression through pruning offers a
promising solution, but existing pruning methods treat all parameters
uniformly, failing to distinguish between general semantic representations and
domain-specific patterns, leading to suboptimal pruning decisions. Thus, we
propose GAPrune, a pruning framework that addresses this challenge by
considering both domain importance and preserving general linguistic
foundation. Our method uses Fisher Information to measure importance and
general-domain gradient alignment to assess parameter behavior, then combines
these signals using our Domain Alignment Importance (DAI) scoring. Lower DAI
scores indicate that the parameter is either less important for the domain task
or creates conflicts between domain and general objectives. Experiments on two
domain benchmarks, FinMTEB and ChemTEB, show that GAPrune maintains performance
within 2.5% of dense models in one-shot pruning at 50% sparsity, while
outperforming all baselines. With retraining in 100 steps, GAPrune achieves
+4.51% improvement on FinMTEB and +1.73% on ChemTEB, demonstrating that our
pruning strategy not only preserves but enhances domain-specific capabilities.
Our findings demonstrate that principled pruning strategies can achieve model
compression and enhanced domain specialization, providing the research
community with a new approach for development.
PersonaX: Multimodal Datasets with LLM-Inferred Behavior Traits
Authors: name: Loka Li, name: Wong Yu Kang, name: Minghao Fu, name: Guangyi Chen, name: Zhenhao Chen, name: Gongxu Luo, name: Yuewen Sun, name: Salman Khan, name: Peter Spirtes, name: Kun Zhang
Published: 2025-09-14 17:30:03

Understanding human behavior traits is central to applications in
human-computer interaction, computational social science, and personalized AI
systems. Such understanding often requires integrating multiple modalities to
capture nuanced patterns and relationships. However, existing resources rarely
provide datasets that combine behavioral descriptors with complementary
modalities such as facial attributes and biographical information. To address
this gap, we present PersonaX, a curated collection of multimodal datasets
designed to enable comprehensive analysis of public traits across modalities.
PersonaX consists of (1) CelebPersona, featuring 9444 public figures from
diverse occupations, and (2) AthlePersona, covering 4181 professional athletes
across 7 major sports leagues. Each dataset includes behavioral trait
assessments inferred by three high-performing large language models, alongside
facial imagery and structured biographical features. We analyze PersonaX at two
complementary levels. First, we abstract high-level trait scores from text
descriptions and apply five statistical independence tests to examine their
relationships with other modalities. Second, we introduce a novel causal
representation learning (CRL) framework tailored to multimodal and
multi-measurement data, providing theoretical identifiability guarantees.
Experiments on both synthetic and real-world data demonstrate the effectiveness
of our approach. By unifying structured and unstructured analysis, PersonaX
establishes a foundation for studying LLM-inferred behavioral traits in
conjunction with visual and biographical attributes, advancing multimodal trait
analysis and causal reasoning.
InternScenes: A Large-scale Simulatable Indoor Scene Dataset with
  Realistic Layouts
Authors: name: Weipeng Zhong, name: Peizhou Cao, name: Yichen Jin, name: Li Luo, name: Wenzhe Cai, name: Jingli Lin, name: Hanqing Wang, name: Zhaoyang Lyu, name: Tai Wang, name: Bo Dai, name: Xudong Xu, name: Jiangmiao Pang
Published: 2025-09-13 14:25:17

The advancement of Embodied AI heavily relies on large-scale, simulatable 3D
scene datasets characterized by scene diversity and realistic layouts. However,
existing datasets typically suffer from limitations in data scale or diversity,
sanitized layouts lacking small items, and severe object collisions. To address
these shortcomings, we introduce InternScenes, a novel large-scale
simulatable indoor scene dataset comprising approximately 40,000 diverse scenes
by integrating three disparate scene sources, real-world scans, procedurally
generated scenes, and designer-created scenes, including 1.96M 3D objects and
covering 15 common scene types and 288 object classes. We particularly preserve
massive small items in the scenes, resulting in realistic and complex layouts
with an average of 41.5 objects per region. Our comprehensive data processing
pipeline ensures simulatability by creating real-to-sim replicas for real-world
scans, enhances interactivity by incorporating interactive objects into these
scenes, and resolves object collisions by physical simulations. We demonstrate
the value of InternScenes with two benchmark applications: scene layout
generation and point-goal navigation. Both show the new challenges posed by the
complex and realistic layouts. More importantly, InternScenes paves the way for
scaling up the model training for both tasks, making the generation and
navigation in such complex scenes possible. We commit to open-sourcing the
data, models, and benchmarks to benefit the whole community.
OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling
Authors: name: Yang Zhou, name: Yifan Wang, name: Jianjun Zhou, name: Wenzheng Chang, name: Haoyu Guo, name: Zizun Li, name: Kaijing Ma, name: Xinyue Li, name: Yating Wang, name: Haoyi Zhu, name: Mingyu Liu, name: Dingning Liu, name: Jiange Yang, name: Zhoujie Fu, name: Junyi Chen, name: Chunhua Shen, name: Jiangmiao Pang, name: Kaipeng Zhang, name: Tong He
Published: 2025-09-15 17:59:19

The field of 4D world modeling - aiming to jointly capture spatial geometry
and temporal dynamics - has witnessed remarkable progress in recent years,
driven by advances in large-scale generative models and multimodal learning.
However, the development of truly general 4D world models remains fundamentally
constrained by the availability of high-quality data. Existing datasets and
benchmarks often lack the dynamic complexity, multi-domain diversity, and
spatial-temporal annotations required to support key tasks such as 4D geometric
reconstruction, future prediction, and camera-control video generation. To
address this gap, we introduce OmniWorld, a large-scale, multi-domain,
multi-modal dataset specifically designed for 4D world modeling. OmniWorld
consists of a newly collected OmniWorld-Game dataset and several curated public
datasets spanning diverse domains. Compared with existing synthetic datasets,
OmniWorld-Game provides richer modality coverage, larger scale, and more
realistic dynamic interactions. Based on this dataset, we establish a
challenging benchmark that exposes the limitations of current state-of-the-art
(SOTA) approaches in modeling complex 4D environments. Moreover, fine-tuning
existing SOTA methods on OmniWorld leads to significant performance gains
across 4D reconstruction and video generation tasks, strongly validating
OmniWorld as a powerful resource for training and evaluation. We envision
OmniWorld as a catalyst for accelerating the development of general-purpose 4D
world models, ultimately advancing machines' holistic understanding of the
physical world.
LazyDrag: Enabling Stable Drag-Based Editing on Multi-Modal Diffusion
  Transformers via Explicit Correspondence
Authors: name: Zixin Yin, name: Xili Dai, name: Duomin Wang, name: Xianfang Zeng, name: Lionel M. Ni, name: Gang Yu, name: Heung-Yeung Shum
Published: 2025-09-15 17:59:47

The reliance on implicit point matching via attention has become a core
bottleneck in drag-based editing, resulting in a fundamental compromise on
weakened inversion strength and costly test-time optimization (TTO). This
compromise severely limits the generative capabilities of diffusion models,
suppressing high-fidelity inpainting and text-guided creation. In this paper,
we introduce LazyDrag, the first drag-based image editing method for
Multi-Modal Diffusion Transformers, which directly eliminates the reliance on
implicit point matching. In concrete terms, our method generates an explicit
correspondence map from user drag inputs as a reliable reference to boost the
attention control. This reliable reference opens the potential for a stable
full-strength inversion process, which is the first in the drag-based editing
task. It obviates the necessity for TTO and unlocks the generative capability
of models. Therefore, LazyDrag naturally unifies precise geometric control with
text guidance, enabling complex edits that were previously out of reach:
opening the mouth of a dog and inpainting its interior, generating new objects
like a ``tennis ball'', or for ambiguous drags, making context-aware changes
like moving a hand into a pocket. Additionally, LazyDrag supports multi-round
workflows with simultaneous move and scale operations. Evaluated on the
DragBench, our method outperforms baselines in drag accuracy and perceptual
quality, as validated by VIEScore and human evaluation. LazyDrag not only
establishes new state-of-the-art performance, but also paves a new way to
editing paradigms.