Orionnnn/hub-stats huggingface dataset:
UniPixel: Unified Object Referring and Segmentation for Pixel-Level
  Visual Reasoning
Authors: name: Ye Liu, name: Zongyang Ma, name: Junfu Pu, name: Zhongang Qi, name: Yang Wu, name: Ying Shan, name: Chang Wen Chen
Published: 2025-09-22 17:59:40

Recent advances in Large Multi-modal Models (LMMs) have demonstrated their
remarkable success as general-purpose multi-modal assistants, with particular
focuses on holistic image- and video-language understanding. Conversely, less
attention has been given to scaling fine-grained pixel-level understanding
capabilities, where the models are expected to realize pixel-level alignment
between visual signals and language semantics. Some previous studies have
applied LMMs to related tasks such as region-level captioning and referring
expression segmentation. However, these models are limited to performing either
referring or segmentation tasks independently and fail to integrate these
fine-grained perception capabilities into visual reasoning. To bridge this gap,
we propose UniPixel, a large multi-modal model capable of flexibly
comprehending visual prompt inputs and generating mask-grounded responses. Our
model distinguishes itself by seamlessly integrating pixel-level perception
with general visual understanding capabilities. Specifically, UniPixel
processes visual prompts and generates relevant masks on demand, and performs
subsequent reasoning conditioning on these intermediate pointers during
inference, thereby enabling fine-grained pixel-level reasoning. The
effectiveness of our approach has been verified on 10 benchmarks across a
diverse set of tasks, including pixel-level referring/segmentation and
object-centric understanding in images/videos. A novel PixelQA task that
jointly requires referring, segmentation, and question answering is also
designed to verify the flexibility of our method.
BeepBank-500: A Synthetic Earcon Mini-Corpus for UI Sound Research and
  Psychoacoustics Research
Authors: name: Mandip Goswami
Published: 2025-09-21 23:26:10

We introduce BeepBank-500, a compact, fully synthetic earcon/alert dataset
(300-500 clips) designed for rapid, rights-clean experimentation in
human-computer interaction and audio machine learning. Each clip is generated
from a parametric recipe controlling waveform family (sine, square, triangle,
FM), fundamental frequency, duration, amplitude envelope, amplitude modulation
(AM), and lightweight Schroeder-style reverberation. We use three reverberation
settings: dry, and two synthetic rooms denoted 'rir small' ('small') and 'rir
medium' ('medium') throughout the paper and in the metadata. We release mono 48
kHz WAV audio (16-bit), a rich metadata table (signal/spectral features), and
tiny reproducible baselines for (i) waveform-family classification and (ii) f0
regression on single tones. The corpus targets tasks such as earcon
classification, timbre analyses, and onset detection, with clearly stated
licensing and limitations. Audio is dedicated to the public domain via CC0-1.0;
code is under MIT. Data DOI: https://doi.org/10.5281/zenodo.17172015. Code:
https://github.com/mandip42/earcons-mini-500.
Cross-Attention is Half Explanation in Speech-to-Text Models
Authors: name: Sara Papi, name: Dennis Fucci, name: Marco Gaido, name: Matteo Negri, name: Luisa Bentivogli
Published: 2025-09-22 16:49:26

Cross-attention is a core mechanism in encoder-decoder architectures,
widespread in many fields, including speech-to-text (S2T) processing. Its
scores have been repurposed for various downstream applications--such as
timestamp estimation and audio-text alignment--under the assumption that they
reflect the dependencies between input speech representation and the generated
text. While the explanatory nature of attention mechanisms has been widely
debated in the broader NLP literature, this assumption remains largely
unexplored within the speech domain. To address this gap, we assess the
explanatory power of cross-attention in S2T models by comparing its scores to
input saliency maps derived from feature attribution. Our analysis spans
monolingual and multilingual, single-task and multi-task models at multiple
scales, and shows that attention scores moderately to strongly align with
saliency-based explanations, particularly when aggregated across heads and
layers. However, it also shows that cross-attention captures only about 50% of
the input relevance and, in the best case, only partially reflects how the
decoder attends to the encoder's representations--accounting for just 52-75% of
the saliency. These findings uncover fundamental limitations in interpreting
cross-attention as an explanatory proxy, suggesting that it offers an
informative yet incomplete view of the factors driving predictions in S2T
models.
DIWALI - Diversity and Inclusivity aWare cuLture specific Items for
  India: Dataset and Assessment of LLMs for Cultural Text Adaptation in Indian
  Context
Authors: name: Pramit Sahoo, name: Maharaj Brahma, name: Maunendra Sankar Desarkar
Published: 2025-09-22 06:58:02

Large language models (LLMs) are widely used in various tasks and
applications. However, despite their wide capabilities, they are shown to lack
cultural alignment ryan-etal-2024-unintended,
alkhamissi-etal-2024-investigating and produce biased generations
naous-etal-2024-beer due to a lack of cultural knowledge and competence.
Evaluation of LLMs for cultural awareness and alignment is particularly
challenging due to the lack of proper evaluation metrics and unavailability of
culturally grounded datasets representing the vast complexity of cultures at
the regional and sub-regional levels. Existing datasets for culture specific
items (CSIs) focus primarily on concepts at the regional level and may contain
false positives. To address this issue, we introduce a novel CSI dataset for
Indian culture, belonging to 17 cultural facets. The dataset comprises sim8k
cultural concepts from 36 sub-regions. To measure the cultural competence of
LLMs on a cultural text adaptation task, we evaluate the adaptations using the
CSIs created, LLM as Judge, and human evaluations from diverse
socio-demographic region. Furthermore, we perform quantitative analysis
demonstrating selective sub-regional coverage and surface-level adaptations
across all considered LLMs. Our dataset is available here:
https://huggingface.co/datasets/nlip/DIWALI{https://huggingface.co/datasets/nlip/DIWALI},
project
webpage\href{https://nlip-lab.github.io/nlip/publications/diwali/{https://nlip-lab.github.io/nlip/publications/diwali/}},
and our codebase with model outputs can be found here:
https://github.com/pramitsahoo/culture-evaluation{https://github.com/pramitsahoo/culture-evaluation}.
FlagEval Findings Report: A Preliminary Evaluation of Large Reasoning
  Models on Automatically Verifiable Textual and Visual Questions
Authors: name: Bowen Qin, name: Chen Yue, name: Fang Yin, name: Hui Wang, name: JG Yao, name: Jiakang Liu, name: Jing-Shu Zheng, name: Miguel Hu Chen, name: Richeng Xuan, name: Shibei Meng, name: Shiqi Zhou, name: Teng Dai, name: Tong-Shuai Ren, name: Wei Cui, name: Xi Yang, name: Xialin Du, name: Xiaojing Xu, name: Xue Sun, name: Xuejing Li, name: Yaming Liu, name: Yesheng Liu, name: Ying Liu, name: Yonghua Lin, name: Yu Zhao, name: Yunduo Zhang, name: Yuwen Luo, name: Zheqi He, name: Zhiyuan He, name: Zhongyuan Wang
Published: 2025-09-21 17:53:30

We conduct a moderate-scale contamination-free (to some extent) evaluation of
current large reasoning models (LRMs) with some preliminary findings. We also
release ROME, our evaluation benchmark for vision language models intended to
test reasoning from visual clues. We attach links to the benchmark, evaluation
data, and other updates on this website:
https://flageval-baai.github.io/LRM-Eval/
QWHA: Quantization-Aware Walsh-Hadamard Adaptation for
  Parameter-Efficient Fine-Tuning on Large Language Models
Authors: name: Hyesung Jeon, name: Seojune Lee, name: Beomseok Kang, name: Yulhwa Kim, name: Jae-Joon Kim
Published: 2025-09-22 07:21:41

The demand for efficient deployment of large language models (LLMs) has
driven interest in quantization, which reduces inference cost, and
parameter-efficient fine-tuning (PEFT), which lowers training overhead. This
motivated the development of quantization-aware PEFT to produce accurate yet
efficient quantized models. In this setting, reducing quantization error prior
to fine-tuning is crucial for achieving high model accuracy. However, existing
methods that rely on low-rank adaptation suffer from limited representational
capacity. Recent Fourier-related transform (FT)-based adapters offer greater
representational power than low-rank adapters, but their direct integration
into quantized models often results in ineffective error reduction and
increased computational overhead. To overcome these limitations, we propose
QWHA, a method that integrates FT-based adapters into quantized models by
employing the Walsh-Hadamard Transform (WHT) as the transform kernel, together
with a novel adapter initialization scheme incorporating adaptive parameter
selection and value refinement. We demonstrate that QWHA effectively mitigates
quantization errors while facilitating fine-tuning, and that its design
substantially reduces computational cost. Experimental results show that QWHA
consistently outperforms baselines in low-bit quantization accuracy and
achieves significant training speedups over existing FT-based adapters. The
code is available at https://github.com/vantaa89/qwha.
Reasoning Core: A Scalable RL Environment for LLM Symbolic Reasoning
Authors: name: Valentin Lacombe, name: Valentin Quesnel, name: Damien Sileo
Published: 2025-09-22 17:56:38

We introduce Reasoning Core, a new scalable environment for Reinforcement
Learning with Verifiable Rewards (RLVR), designed to advance foundational
symbolic reasoning in Large Language Models (LLMs). Unlike existing benchmarks
that focus on games or isolated puzzles, Reasoning Core procedurally generates
problems across core formal domains, including PDDL planning, first-order
logic, context-free grammar parsing, causal reasoning, and system equation
solving. The environment is built on key design principles of high-generality
problem distributions, verification via external tools, and continuous
difficulty control, which together provide a virtually infinite supply of novel
training instances. Initial zero-shot evaluations with frontier LLMs confirm
the difficulty of Reasoning Core's tasks, positioning it as a promising
resource to improve the reasoning capabilities of future models.
Turk-LettuceDetect: A Hallucination Detection Models for Turkish RAG
  Applications
Authors: name: Selva Taş, name: Mahmut El Huseyni, name: Özay Ezerceli, name: Reyhan Bayraktar, name: Fatma Betül Terzioğlu
Published: 2025-09-22 12:14:11

The widespread adoption of Large Language Models (LLMs) has been hindered by
their tendency to hallucinate, generating plausible but factually incorrect
information. While Retrieval-Augmented Generation (RAG) systems attempt to
address this issue by grounding responses in external knowledge, hallucination
remains a persistent challenge, particularly for morphologically complex,
low-resource languages like Turkish. This paper introduces Turk-LettuceDetect,
the first suite of hallucination detection models specifically designed for
Turkish RAG applications. Building on the LettuceDetect framework, we formulate
hallucination detection as a token-level classification task and fine-tune
three distinct encoder architectures: a Turkish-specific ModernBERT,
TurkEmbed4STS, and multilingual EuroBERT. These models were trained on a
machine-translated version of the RAGTruth benchmark dataset containing 17,790
instances across question answering, data-to-text generation, and summarization
tasks. Our experimental results show that the ModernBERT-based model achieves
an F1-score of 0.7266 on the complete test set, with particularly strong
performance on structured tasks. The models maintain computational efficiency
while supporting long contexts up to 8,192 tokens, making them suitable for
real-time deployment. Comparative analysis reveals that while state-of-the-art
LLMs demonstrate high recall, they suffer from low precision due to
over-generation of hallucinated content, underscoring the necessity of
specialized detection mechanisms. By releasing our models and translated
dataset, this work addresses a critical gap in multilingual NLP and establishes
a foundation for developing more reliable and trustworthy AI applications for
Turkish and other languages.
TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning
  for Video LLMs
Authors: name: Yunheng Li, name: Jing Cheng, name: Shaoyong Jia, name: Hangyi Kuang, name: Shaohui Jiao, name: Qibin Hou, name: Ming-Ming Cheng
Published: 2025-09-22 17:30:15

This paper introduces TempSamp-R1, a new reinforcement fine-tuning framework
designed to improve the effectiveness of adapting multimodal large language
models (MLLMs) to video temporal grounding tasks. We reveal that existing
reinforcement learning methods, such as Group Relative Policy Optimization
(GRPO), rely on on-policy sampling for policy updates. However, in tasks with
large temporal search spaces, this strategy becomes both inefficient and
limited in performance, as it often fails to identify temporally accurate
solutions. To address this limitation, TempSamp-R1 leverages ground-truth
annotations as off-policy supervision to provide temporally precise guidance,
effectively compensating for the sparsity and misalignment in on-policy
solutions. To further stabilize training and reduce variance in reward-based
updates, TempSamp-R1 provides a non-linear soft advantage computation method
that dynamically reshapes the reward feedback via an asymmetric transformation.
By employing a hybrid Chain-of-Thought (CoT) training paradigm, TempSamp-R1
optimizes a single unified model to support both CoT and non-CoT inference
modes, enabling efficient handling of queries with varying reasoning
complexity. Experimental results demonstrate that TempSamp-R1 outperforms
GRPO-based baselines, establishing new state-of-the-art performance on
benchmark datasets: Charades-STA (R1@0.7: 52.9%, +2.7%), ActivityNet Captions
(R1@0.5: 56.0%, +5.3%), and QVHighlights (mAP: 30.0%, +3.0%). Moreover,
TempSamp-R1 shows robust few-shot generalization capabilities under limited
data. Code: https://github.com/HVision-NKU/TempSamp-R1
CodeFuse-CR-Bench: A Comprehensiveness-aware Benchmark for End-to-End
  Code Review Evaluation in Python Projects
Authors: name: Hanyang Guo, name: Xunjin Zheng, name: Zihan Liao, name: Hang Yu, name: Peng DI, name: Ziyin Zhang, name: Hong-Ning Dai
Published: 2025-09-18 11:24:09

Automated code review (CR) is a key application for Large Language Models
(LLMs), but progress is hampered by a "reality gap": existing benchmarks
evaluate models on isolated sub-tasks using simplified, context-poor data. This
fails to reflect the holistic context-rich nature of real-world CR. To bridge
this gap, we introduce CodeFuse-CR-Bench, the first comprehensiveness-aware
benchmark for repository-level CR evaluation. CodeFuse-CR-Bench comprises 601
high-quality instances from 70 Python projects covering nine Pull-Request (PR)
problem domains, where each instance provides rich, multi-faceted context
including the associated issue, PR details, and repository state, enabling
end-to-end evaluation. Beyond superficial metrics, we also propose a novel
evaluation framework that combines rule-based checks for location and syntax
with model-based judgments of review quality. We present the first large-scale
assessment of state-of-the-art LLMs on this comprehensive CR task. Our results
establish crucial baselines and reveal that (1) no single LLM dominates all
aspects of CR; (2) Gemini 2.5 Pro achieves the highest comprehensive
performance; and (3) different LLMs exhibit varying robustness to redundant
context. These findings highlight the necessity of holistic, multi-dimensional
evaluation and provide actionable insights for advancing truly intelligent yet
practical CR assistants.
SCAN: Self-Denoising Monte Carlo Annotation for Robust Process Reward
  Learning
Authors: name: Yuyang Ding, name: Xinyu Shi, name: Juntao Li, name: Xiaobo Liang, name: Zhaopeng Tu, name: Min Zhang
Published: 2025-09-20 06:19:55

Process reward models (PRMs) offer fine-grained, step-level evaluations that
facilitate deeper reasoning processes in large language models (LLMs), proving
effective in complex tasks like mathematical reasoning. However, developing
PRMs is challenging due to the high cost and limited scalability of
human-annotated data. Synthetic data from Monte Carlo (MC) estimation is a
promising alternative but suffers from a high noise ratio, which can cause
overfitting and hinder large-scale training. In this work, we conduct a
preliminary study on the noise distribution in synthetic data from MC
estimation, identifying that annotation models tend to both underestimate and
overestimate step correctness due to limitations in their annotation
capabilities. Building on these insights, we propose Self-Denoising Monte Carlo
Annotation (SCAN), an efficient data synthesis and noise-tolerant learning
framework. Our key findings indicate that: (1) Even lightweight models (e.g.,
1.5B parameters) can produce high-quality annotations through a self-denoising
strategy, enabling PRMs to achieve superior performance with only 6% the
inference cost required by vanilla MC estimation. (2) With our robust learning
strategy, PRMs can effectively learn from this weak supervision, achieving a
39.2 F1 score improvement (from 19.9 to 59.1) in ProcessBench. Despite using
only a compact synthetic dataset, our models surpass strong baselines,
including those trained on large-scale human-annotated datasets such as
PRM800K. Furthermore, performance continues to improve as we scale up the
synthetic data, highlighting the potential of SCAN for scalable,
cost-efficient, and robust PRM training.
GeoPQA: Bridging the Visual Perception Gap in MLLMs for Geometric
  Reasoning
Authors: name: Guizhen Chen, name: Weiwen Xu, name: Hao Zhang, name: Hou Pong Chan, name: Deli Zhao, name: Anh Tuan Luu, name: Yu Rong
Published: 2025-09-22 07:28:09

Recent advancements in reinforcement learning (RL) have enhanced the
reasoning abilities of large language models (LLMs), yet the impact on
multimodal LLMs (MLLMs) is limited. Particularly in vision-intensive tasks like
geometric reasoning, MLLMs hallucinate frequently, leading to inaccurate
reasoning. We attribute this to the perceptual bottleneck in MLLMs, which caps
the benefits of reasoning training. To quantify this, we design a
Geo-Perception Question-Answering (GeoPQA) benchmark, targeting basic geometric
concepts and spatial relationships. Experiments on GeoPQA reveal significant
shortcomings of MLLMs in visual perception, which constrain RL reward signals
for effective training. To address this bottleneck, we propose a two-stage RL
training framework by first enhancing the visual perception of geometric
structures, then fostering reasoning capabilities. Applied to
Qwen2.5-VL-3B-Instruct, our two-stage training improves geometric reasoning by
9.7% and geometric problem solving by 9.1%, compared to the direct reasoning
training approach. Our method also generalizes to other vision-intensive
domains like figure understanding, highlighting the importance of perceptual
grounding in effective MLLM reasoning.
VaseVQA: Multimodal Agent and Benchmark for Ancient Greek Pottery
Authors: name: Jinchao Ge, name: Tengfei Cheng, name: Biao Wu, name: Zeyu Zhang, name: Shiya Huang, name: Judith Bishop, name: Gillian Shepherd, name: Meng Fang, name: Ling Chen, name: Yang Zhao
Published: 2025-09-21 18:36:54

Analyzing cultural-heritage artifacts remains challenging for MLLMs: general
models lack domain expertise, and SFT often overfits superficial patterns,
yielding brittle reasoning for authentication and historical attribution. This
raises the question of how to equip MLLMs with robust, expert-level reasoning
for ancient Greek pottery. We present VaseVL, an SFT-then-RL system that turns
evaluation into supervision: we construct a taxonomy of question types, probe
the SFT model to localize type-specific performance gaps, and optimize with
type-conditioned, compositionality-oriented rewards targeting those gaps. We
also release VaseVQA, a comprehensive benchmark of 31,773 images designed to
probe deep understanding. Experiments show state-of-the-art results on style
classification and historical attribution with marked gains in compositional
robustness over SFT-only baselines, validating diagnosis-guided,
taxonomy-conditioned reward engineering and providing a reusable resource for
future research. Code and dataset will be available at
https://github.com/AIGeeksGroup/VaseVQA.
Understanding Embedding Scaling in Collaborative Filtering
Authors: name: Zhuangzhuang He, name: Zhou Kaiyu, name: Haoyue Bai, name: Fengbin Zhu, name: Yonghui Yang
Published: 2025-09-19 07:33:50

Scaling recommendation models into large recommendation models has become one
of the most widely discussed topics. Recent efforts focus on components beyond
the scaling embedding dimension, as it is believed that scaling embedding may
lead to performance degradation. Although there have been some initial
observations on embedding, the root cause of their non-scalability remains
unclear. Moreover, whether performance degradation occurs across different
types of models and datasets is still an unexplored area. Regarding the effect
of embedding dimensions on performance, we conduct large-scale experiments
across 10 datasets with varying sparsity levels and scales, using 4
representative classical architectures. We surprisingly observe two novel
phenomenon: double-peak and logarithmic. For the former, as the embedding
dimension increases, performance first improves, then declines, rises again,
and eventually drops. For the latter, it exhibits a perfect logarithmic curve.
Our contributions are threefold. First, we discover two novel phenomena when
scaling collaborative filtering models. Second, we gain an understanding of the
underlying causes of the double-peak phenomenon. Lastly, we theoretically
analyze the noise robustness of collaborative filtering models, with results
matching empirical observations.
From Uniform to Heterogeneous: Tailoring Policy Optimization to Every
  Token's Nature
Authors: name: Zheng Liu, name: Mengjie Liu, name: Siwei Wen, name: Mengzhang Cai, name: Bin Cui, name: Conghui He, name: Wentao Zhang
Published: 2025-09-20 09:30:25

Reinforcement Learning has emerged as the fundamental technique for enhancing
reasoning in LLMs. However, existing algorithms apply uniform optimization to
all tokens, ignoring their different roles in reasoning process. To address
this limitation, we introduce Heterogeneous Adaptive Policy Optimization
(HAPO), a comprehensive token-aware algorithm that dynamically adapts
optimization based on token entropy. For rollout sampling, we propose Adaptive
Temperature Sampling, which adjusts sampling temperature in real time,
promoting exploration at high-entropy tokens while preserving coherence at
low-entropy ones. For advantage calculation, we introduce Token Level Group
Average that normalizes advantages at token level, jointly accounting for
sequence-length as in token-mean loss while preserving non-biased treatment. We
then develop Differential Advantage Redistribution that leverages entropy and
importance ratios to modulate rewards-adjusting updates for tokens with clear
signals. For clipping loss, we design Asymmetric Adaptive Clipping, allowing
aggressive probability reduction for noisy low-entropy tokens while enabling
exploration for high-entropy tokens. Through systematic investigation between
entropy and training dynamics, we embedded token-level treatment into every
stages to achieve fine-grained control. Extensive experiments demonstrate that
HAPO consistently outperforms DAPO across multiple model scales. Our code can
be found in https://github.com/starriver030515/HAPO.