Orionnnn/hub-stats huggingface dataset:
Optimizing What Matters: AUC-Driven Learning for Robust Neural Retrieval
Authors: name: Nima Sheikholeslami, name: Erfan Hosseini, name: Patrice Bechard, name: Srivatsava Daruru, name: Sai Rajeswar
Published: 2025-09-30 18:14:01

Dual-encoder retrievers depend on the principle that relevant documents
should score higher than irrelevant ones for a given query. Yet the dominant
Noise Contrastive Estimation (NCE) objective, which underpins Contrastive Loss,
optimizes a softened ranking surrogate that we rigorously prove is
fundamentally oblivious to score separation quality and unrelated to AUC. This
mismatch leads to poor calibration and suboptimal performance in downstream
tasks like retrieval-augmented generation (RAG). To address this fundamental
limitation, we introduce the MW loss, a new training objective that maximizes
the Mann-Whitney U statistic, which is mathematically equivalent to the Area
under the ROC Curve (AUC). MW loss encourages each positive-negative pair to be
correctly ranked by minimizing binary cross entropy over score differences. We
provide theoretical guarantees that MW loss directly upper-bounds the AoC,
better aligning optimization with retrieval goals. We further promote ROC
curves and AUC as natural threshold free diagnostics for evaluating retriever
calibration and ranking quality. Empirically, retrievers trained with MW loss
consistently outperform contrastive counterparts in AUC and standard retrieval
metrics. Our experiments show that MW loss is an empirically superior
alternative to Contrastive Loss, yielding better-calibrated and more
discriminative retrievers for high-stakes applications like RAG.
ModernVBERT: Towards Smaller Visual Document Retrievers
Authors: name: Paul Teiletche, name: Quentin Macé, name: Max Conti, name: Antonio Loison, name: Gautier Viaud, name: Pierre Colombo, name: Manuel Faysse
Published: 2025-10-01 17:41:17

Multimodal embedding models are gaining prevalence, notably for document
retrieval as efficient alternatives to text-only pipelines. These models are
typically built by finetuning large vision-language decoders (VLMs) with
contrastive losses on text-image pairs. In this work, we show that, while
cost-efficient, this repurposing approach often bottlenecks retrieval
performance. Through controlled experiments, we establish a principled recipe
for improving visual document retrieval models. We notably measure the impact
of attention masking, image resolution, modality alignment data regimes, and
late interaction centered contrastive objectives which emerge as central
performance factors. Building on these insights, we release ModernVBERT, a
compact 250M-parameter vision-language encoder that outperforms models up to 10
times larger when finetuned on document retrieval tasks. Models and code are
made available at https://huggingface.co/ModernVBERT.
The Rogue Scalpel: Activation Steering Compromises LLM Safety
Authors: name: Anton Korznikov, name: Andrey Galichin, name: Alexey Dontsov, name: Oleg Y. Rogov, name: Ivan Oseledets, name: Elena Tutubalina
Published: 2025-09-26 08:49:47

Activation steering is a promising technique for controlling LLM behavior by
adding semantically meaningful vectors directly into a model's hidden states
during inference. It is often framed as a precise, interpretable, and
potentially safer alternative to fine-tuning. We demonstrate the opposite:
steering systematically breaks model alignment safeguards, making it comply
with harmful requests. Through extensive experiments on different model
families, we show that even steering in a random direction can increase the
probability of harmful compliance from 0% to 2-27%. Alarmingly, steering benign
features from a sparse autoencoder (SAE), a common source of interpretable
directions, increases these rates by a further 2-4%. Finally, we show that
combining 20 randomly sampled vectors that jailbreak a single prompt creates a
universal attack, significantly increasing harmful compliance on unseen
requests. These results challenge the paradigm of safety through
interpretability, showing that precise control over model internals does not
guarantee precise control over model behavior.
IoT-MCP: Bridging LLMs and IoT Systems Through Model Context Protocol
Authors: name: Ningyuan Yang, name: Guanliang Lyu, name: Mingchen Ma, name: Yiyi Lu, name: Yiming Li, name: Zhihui Gao, name: Hancheng Ye, name: Jianyi Zhang, name: Tingjun Chen, name: Yiran Chen
Published: 2025-09-25 08:35:47

The integration of Large Language Models (LLMs) with Internet-of-Things (IoT)
systems faces significant challenges in hardware heterogeneity and control
complexity. The Model Context Protocol (MCP) emerges as a critical enabler,
providing standardized communication between LLMs and physical devices. We
propose IoT-MCP, a novel framework that implements MCP through edge-deployed
servers to bridge LLMs and IoT ecosystems. To support rigorous evaluation, we
introduce IoT-MCP Bench, the first benchmark containing 114 Basic Tasks (e.g.,
``What is the current temperature?'') and 1,140 Complex Tasks (e.g., ``I feel
so hot, do you have any ideas?'') for IoT-enabled LLMs. Experimental validation
across 22 sensor types and 6 microcontroller units demonstrates IoT-MCP's 100%
task success rate to generate tool calls that fully meet expectations and
obtain completely accurate results, 205ms average response time, and 74KB peak
memory footprint. This work delivers both an open-source integration framework
(https://github.com/Duke-CEI-Center/IoT-MCP-Servers) and a standardized
evaluation methodology for LLM-IoT systems.
RLAD: Training LLMs to Discover Abstractions for Solving Reasoning
  Problems
Authors: name: Yuxiao Qu, name: Anikait Singh, name: Yoonho Lee, name: Amrith Setlur, name: Ruslan Salakhutdinov, name: Chelsea Finn, name: Aviral Kumar
Published: 2025-10-02 17:44:23

Reasoning requires going beyond pattern matching or memorization of solutions
to identify and implement "algorithmic procedures" that can be used to deduce
answers to hard problems. Doing so requires realizing the most relevant
primitives, intermediate results, or shared procedures, and building upon them.
While RL post-training on long chains of thought ultimately aims to uncover
this kind of algorithmic behavior, most reasoning traces learned by large
models fail to consistently capture or reuse procedures, instead drifting into
verbose and degenerate exploration. To address more effective reasoning, we
introduce reasoning abstractions: concise natural language descriptions of
procedural and factual knowledge that guide the model toward learning
successful reasoning. We train models to be capable of proposing multiple
abstractions given a problem, followed by RL that incentivizes building a
solution while using the information provided by these abstractions. This
results in a two-player RL training paradigm, abbreviated as RLAD, that jointly
trains an abstraction generator and a solution generator. This setup
effectively enables structured exploration, decouples learning signals of
abstraction proposal and solution generation, and improves generalization to
harder problems. We also show that allocating more test-time compute to
generating abstractions is more beneficial for performance than generating more
solutions at large test budgets, illustrating the role of abstractions in
guiding meaningful exploration.
Generalized Parallel Scaling with Interdependent Generations
Authors: name: Harry Dong, name: David Brandfonbrener, name: Eryk Helenowski, name: Yun He, name: Mrinal Kumar, name: Han Fang, name: Yuejie Chi, name: Karthik Abinav Sankararaman
Published: 2025-10-01 17:33:35

Parallel LLM inference scaling involves sampling a set of N>1 responses for
a single input prompt. However, these N parallel responses tend to be
generated independently from each other, partitioning compute resources and
leaving potentially useful information in one generation untapped by others.
This is in contrast to response length scaling where past computation is used
in all future steps. For higher quality responses and response sets, we propose
Bridge to generate interdependent responses in parallel by rethinking batched
LLM hidden states as holistic tensors rather than independent slices. With only
a small amount (2.8%-5.1%) of new parameters, Bridge improves the relative mean
accuracy gains from reinforcement learning with verifiable rewards by up to 50%
and boosts consistency of correct responses. Trained once, Bridge scales to any
generation width, all with greater performance than independent generations,
unlocking a more general mode of parallel scaling that effectively leverages
information between sequences, compatible with any post-generation aggregation
technique.
SQUARE: Semantic Query-Augmented Fusion and Efficient Batch Reranking
  for Training-free Zero-Shot Composed Image Retrieval
Authors: name: Ren-Di Wu, name: Yu-Yen Lin, name: Huei-Fang Yang
Published: 2025-09-30 14:41:24

Composed Image Retrieval (CIR) aims to retrieve target images that preserve
the visual content of a reference image while incorporating user-specified
textual modifications. Training-free zero-shot CIR (ZS-CIR) approaches, which
require no task-specific training or labeled data, are highly desirable, yet
accurately capturing user intent remains challenging. In this paper, we present
SQUARE, a novel two-stage training-free framework that leverages Multimodal
Large Language Models (MLLMs) to enhance ZS-CIR. In the Semantic
Query-Augmented Fusion (SQAF) stage, we enrich the query embedding derived from
a vision-language model (VLM) such as CLIP with MLLM-generated captions of the
target image. These captions provide high-level semantic guidance, enabling the
query to better capture the user's intent and improve global retrieval quality.
In the Efficient Batch Reranking (EBR) stage, top-ranked candidates are
presented as an image grid with visual marks to the MLLM, which performs joint
visual-semantic reasoning across all candidates. Our reranking strategy
operates in a single pass and yields more accurate rankings. Experiments show
that SQUARE, with its simplicity and effectiveness, delivers strong performance
on four standard CIR benchmarks. Notably, it maintains high performance even
with lightweight pre-trained, demonstrating its potential applicability.
VideoNSA: Native Sparse Attention Scales Video Understanding
Authors: name: Enxin Song, name: Wenhao Chai, name: Shusheng Yang, name: Ethan Armand, name: Xiaojun Shan, name: Haiyang Xu, name: Jianwen Xie, name: Zhuowen Tu
Published: 2025-10-02 17:58:54

Video understanding in multimodal language models remains limited by context
length: models often miss key transition frames and struggle to maintain
coherence across long time scales. To address this, we adapt Native Sparse
Attention (NSA) to video-language models. Our method, VideoNSA, adapts
Qwen2.5-VL through end-to-end training on a 216K video instruction dataset. We
employ a hardware-aware hybrid approach to attention, preserving dense
attention for text, while employing NSA for video. Compared to
token-compression and training-free sparse baselines, VideoNSA achieves
improved performance on long-video understanding, temporal reasoning, and
spatial benchmarks. Further ablation analysis reveals four key findings: (1)
reliable scaling to 128K tokens; (2) an optimal global-local attention
allocation at a fixed budget; (3) task-dependent branch usage patterns; and (4)
the learnable combined sparse attention help induce dynamic attention sinks.
Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming
  Attacks
Authors: name: Ruohao Guo, name: Afshin Oroojlooy, name: Roshan Sridhar, name: Miguel Ballesteros, name: Alan Ritter, name: Dan Roth
Published: 2025-10-02 17:57:05

Despite recent rapid progress in AI safety, current large language models
remain vulnerable to adversarial attacks in multi-turn interaction settings,
where attackers strategically adapt their prompts across conversation turns and
pose a more critical yet realistic challenge. Existing approaches that discover
safety vulnerabilities either rely on manual red-teaming with human experts or
employ automated methods using pre-defined templates and human-curated attack
data, with most focusing on single-turn attacks. However, these methods did not
explore the vast space of possible multi-turn attacks, failing to consider
novel attack trajectories that emerge from complex dialogue dynamics and
strategic conversation planning. This gap is particularly critical given recent
findings that LLMs exhibit significantly higher vulnerability to multi-turn
attacks compared to single-turn attacks. We propose DialTree-RPO, an on-policy
reinforcement learning framework integrated with tree search that autonomously
discovers diverse multi-turn attack strategies by treating the dialogue as a
sequential decision-making problem, enabling systematic exploration without
manually curated data. Through extensive experiments, our approach not only
achieves more than 25.9% higher ASR across 10 target models compared to
previous state-of-the-art approaches, but also effectively uncovers new attack
strategies by learning optimal dialogue policies that maximize attack success
across multiple turns.
Visual Multi-Agent System: Mitigating Hallucination Snowballing via
  Visual Flow
Authors: name: Xinlei Yu, name: Chengming Xu, name: Guibin Zhang, name: Yongbo He, name: Zhangquan Chen, name: Zhucun Xue, name: Jiangning Zhang, name: Yue Liao, name: Xiaobin Hu, name: Yu-Gang Jiang, name: Shuicheng Yan
Published: 2025-09-26 02:43:24

Multi-Agent System (MAS) powered by Visual Language Models (VLMs) enables
challenging tasks but suffers from a novel failure term, multi-agent visual
hallucination snowballing, where hallucinations are seeded in a single agent
and amplified by following ones due to the over-reliance on textual flow to
relay visual information. Through turn-, layer-, and token-wise attention
analyses, we provide detailed insights into the essence of hallucination
snowballing regarding the reduction of visual attention allocation. It leads us
to identify a subset of vision tokens with a unimodal attention peak in middle
layers that best preserve visual evidence but gradually diminish in deeper
agent turns, resulting in the visual hallucination snowballing in MAS. Thus, we
propose ViF, a lightweight, plug-and-play mitigation paradigm that relays
inter-agent messages with Visual Flow powered by the selected visual relay
tokens and applies attention reallocation to amplify this pattern. The
experiment results demonstrate that our method markedly reduces hallucination
snowballing, consistently improving the performance across eight benchmarks
based on four common MAS structures and ten base models. The source code will
be available at: https://github.com/YU-deep/ViF.git.
Learning to Reason for Hallucination Span Detection
Authors: name: Hsuan Su, name: Ting-Yao Hu, name: Hema Swetha Koppula, name: Kundan Krishna, name: Hadi Pouransari, name: Cheng-Yu Hsieh, name: Cem Koc, name: Joseph Yitan Cheng, name: Oncel Tuzel, name: Raviteja Vemulapalli
Published: 2025-10-02 16:24:28

Large language models (LLMs) often generate hallucinations -- unsupported
content that undermines reliability. While most prior works frame hallucination
detection as a binary task, many real-world applications require identifying
hallucinated spans, which is a multi-step decision making process. This
naturally raises the question of whether explicit reasoning can help the
complex task of detecting hallucination spans. To answer this question, we
first evaluate pretrained models with and without Chain-of-Thought (CoT)
reasoning, and show that CoT reasoning has the potential to generate at least
one correct answer when sampled multiple times. Motivated by this, we propose
RL4HS, a reinforcement learning framework that incentivizes reasoning with a
span-level reward function. RL4HS builds on Group Relative Policy Optimization
and introduces Class-Aware Policy Optimization to mitigate reward imbalance
issue. Experiments on the RAGTruth benchmark (summarization, question
answering, data-to-text) show that RL4HS surpasses pretrained reasoning models
and supervised fine-tuning, demonstrating the necessity of reinforcement
learning with span-level rewards for detecting hallucination spans.
Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and
  Reasoning in Vision-Language Models
Authors: name: Yu Zeng, name: Wenxuan Huang, name: Shiting Huang, name: Xikun Bao, name: Yukun Qi, name: Yiming Zhao, name: Qiuchen Wang, name: Lin Chen, name: Zehui Chen, name: Huaian Chen, name: Wanli Ouyang, name: Feng Zhao
Published: 2025-10-01 17:58:05

Although current large Vision-Language Models (VLMs) have advanced in
multimodal understanding and reasoning, their fundamental perceptual and
reasoning abilities remain limited. Specifically, even on simple jigsaw tasks,
existing VLMs perform near randomly, revealing deficiencies in core perception
and reasoning capabilities. While high-quality vision-language data can enhance
these capabilities, its scarcity and limited scalability impose significant
constraints. To address this, we propose AGILE, an Agentic jiGsaw Interaction
Learning for Enhancing visual perception and reasoning in VLMs. AGILE
formulates jigsaw solving as an interactive process, enabling the model to
progressively engage with the environment. At each step, the model generates
executable code to perform an action based on the current state, while the
environment provides fine-grained visual feedback to guide task completion.
Through this iterative cycle of observation and interaction, the model
incrementally improves its perceptual and reasoning capabilities via
exploration and feedback. Experimental results show that AGILE not only
substantially boosts performance on jigsaw tasks of varying complexity (e.g.,
increasing accuracy from 9.5% to 82.8% under the 2 times 2 setting) but also
demonstrates strong generalization across 9 general vision tasks, achieving an
average improvement of 3.1%. These results indicate notable enhancements in
both perceptual and reasoning abilities. This work opens a new avenue for
advancing reasoning and generalization in multimodal models and provides an
efficient, scalable solution to the scarcity of multimodal reinforcement
learning data. The code and datasets is available at
https://github.com/yuzeng0-0/AGILE .
StealthAttack: Robust 3D Gaussian Splatting Poisoning via Density-Guided
  Illusions
Authors: name: Bo-Hsu Ke, name: You-Zhe Xie, name: Yu-Lun Liu, name: Wei-Chen Chiu
Published: 2025-10-02 17:59:57

3D scene representation methods like Neural Radiance Fields (NeRF) and 3D
Gaussian Splatting (3DGS) have significantly advanced novel view synthesis. As
these methods become prevalent, addressing their vulnerabilities becomes
critical. We analyze 3DGS robustness against image-level poisoning attacks and
propose a novel density-guided poisoning method. Our method strategically
injects Gaussian points into low-density regions identified via Kernel Density
Estimation (KDE), embedding viewpoint-dependent illusory objects clearly
visible from poisoned views while minimally affecting innocent views.
Additionally, we introduce an adaptive noise strategy to disrupt multi-view
consistency, further enhancing attack effectiveness. We propose a KDE-based
evaluation protocol to assess attack difficulty systematically, enabling
objective benchmarking for future research. Extensive experiments demonstrate
our method's superior performance compared to state-of-the-art techniques.
Project page: https://hentci.github.io/stealthattack/
RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via
  Multi-Stage Reinforcement Learning
Authors: name: Sicheng Feng, name: Kaiwen Tuo, name: Song Wang, name: Lingdong Kong, name: Jianke Zhu, name: Huan Wang
Published: 2025-10-02 17:29:46

Fine-grained visual reasoning remains a core challenge for multimodal large
language models (MLLMs). The recently introduced ReasonMap highlights this gap
by showing that even advanced MLLMs struggle with spatial reasoning in
structured and information-rich settings such as transit maps, a task of clear
practical and scientific importance. However, standard reinforcement learning
(RL) on such tasks is impeded by sparse rewards and unstable optimization. To
address this, we first construct ReasonMap-Plus, an extended dataset that
introduces dense reward signals through Visual Question Answering (VQA) tasks,
enabling effective cold-start training of fine-grained visual understanding
skills. Next, we propose RewardMap, a multi-stage RL framework designed to
improve both visual understanding and reasoning capabilities of MLLMs.
RewardMap incorporates two key designs. First, we introduce a difficulty-aware
reward design that incorporates detail rewards, directly tackling the sparse
rewards while providing richer supervision. Second, we propose a multi-stage RL
scheme that bootstraps training from simple perception to complex reasoning
tasks, offering a more effective cold-start strategy than conventional
Supervised Fine-Tuning (SFT). Experiments on ReasonMap and ReasonMap-Plus
demonstrate that each component of RewardMap contributes to consistent
performance gains, while their combination yields the best results. Moreover,
models trained with RewardMap achieve an average improvement of 3.47% across 6
benchmarks spanning spatial reasoning, fine-grained visual reasoning, and
general tasks beyond transit maps, underscoring enhanced visual understanding
and reasoning capabilities.
Optimal Control Meets Flow Matching: A Principled Route to Multi-Subject
  Fidelity
Authors: name: Eric Tillmann Bill, name: Enis Simsar, name: Thomas Hofmann
Published: 2025-10-02 17:59:58

Text-to-image (T2I) models excel on single-entity prompts but struggle with
multi-subject descriptions, often showing attribute leakage, identity
entanglement, and subject omissions. We introduce the first theoretical
framework with a principled, optimizable objective for steering sampling
dynamics toward multi-subject fidelity. Viewing flow matching (FM) through
stochastic optimal control (SOC), we formulate subject disentanglement as
control over a trained FM sampler. This yields two architecture-agnostic
algorithms: (i) a training-free test-time controller that perturbs the base
velocity with a single-pass update, and (ii) Adjoint Matching, a lightweight
fine-tuning rule that regresses a control network to a backward adjoint signal
while preserving base-model capabilities. The same formulation unifies prior
attention heuristics, extends to diffusion models via a flow-diffusion
correspondence, and provides the first fine-tuning route explicitly designed
for multi-subject fidelity. Empirically, on Stable Diffusion 3.5, FLUX, and
Stable Diffusion XL, both algorithms consistently improve multi-subject
alignment while maintaining base-model style. Test-time control runs
efficiently on commodity GPUs, and fine-tuned controllers trained on limited
prompts generalize to unseen ones. We further highlight FOCUS (Flow Optimal
Control for Unentangled Subjects), which achieves state-of-the-art
multi-subject fidelity across models.