Orionnnn/hub-stats huggingface dataset:
Multi-Agent Game Generation and Evaluation via Audio-Visual Recordings
Authors: name: Alexia Jolicoeur-Martineau
Published: 2025-08-01 13:45:13

While AI excels at generating text, audio, images, and videos, creating
interactive audio-visual content such as video games remains challenging.
Current LLMs can generate JavaScript games and animations, but lack automated
evaluation metrics and struggle with complex content that normally requires
teams of humans working for many months (multi-shot, multi-agents) using assets
made by artists. To tackle these issues, we built a new metric and a
multi-agent system.
  We propose AVR-Eval, a relative metric for multimedia content quality using
Audio-Visual Recordings (AVRs). An omni-modal model (processing text, video,
and audio) compares the AVRs of two contents, with a text model reviewing
evaluations to determine superiority. We show that AVR-Eval properly identifies
good from broken or mismatched content.
  We built AVR-Agent, a multi-agent system generating JavaScript code from a
bank of multimedia assets (audio, images, 3D models). The coding agent selects
relevant assets, generates multiple initial codes, uses AVR-Eval to identify
the best version, and iteratively improves it through omni-modal agent feedback
from the AVR.
  We run experiments on games and animations with AVR-Eval (win rate of content
A against B). We find that content generated by AVR-Agent has a significantly
higher win rate against content made through one-shot generation. However,
models struggle to leverage custom assets and AVR feedback effectively, showing
no higher win rate. This reveals a critical gap: while humans benefit from
high-quality assets and audio-visual feedback, current coding models do not
seem to utilize these resources as effectively, highlighting fundamental
differences between human and machine content creation approaches.
SpA2V: Harnessing Spatial Auditory Cues for Audio-driven Spatially-aware
  Video Generation
Authors: name: Kien T. Pham, name: Yingqing He, name: Yazhou Xing, name: Qifeng Chen, name: Long Chen
Published: 2025-08-01 17:05:04

Audio-driven video generation aims to synthesize realistic videos that align
with input audio recordings, akin to the human ability to visualize scenes from
auditory input. However, existing approaches predominantly focus on exploring
semantic information, such as the classes of sounding sources present in the
audio, limiting their ability to generate videos with accurate content and
spatial composition. In contrast, we humans can not only naturally identify the
semantic categories of sounding sources but also determine their deeply encoded
spatial attributes, including locations and movement directions. This useful
information can be elucidated by considering specific spatial indicators
derived from the inherent physical properties of sound, such as loudness or
frequency. As prior methods largely ignore this factor, we present SpA2V, the
first framework explicitly exploits these spatial auditory cues from audios to
generate videos with high semantic and spatial correspondence. SpA2V decomposes
the generation process into two stages: 1) Audio-guided Video Planning: We
meticulously adapt a state-of-the-art MLLM for a novel task of harnessing
spatial and semantic cues from input audio to construct Video Scene Layouts
(VSLs). This serves as an intermediate representation to bridge the gap between
the audio and video modalities. 2) Layout-grounded Video Generation: We develop
an efficient and effective approach to seamlessly integrate VSLs as conditional
guidance into pre-trained diffusion models, enabling VSL-grounded video
generation in a training-free manner. Extensive experiments demonstrate that
SpA2V excels in generating realistic videos with semantic and spatial alignment
to the input audios.
Investigating Hallucination in Conversations for Low Resource Languages
Authors: name: Amit Das, name: Md. Najib Hasan, name: Souvika Sarkar, name: Zheng Zhang, name: Fatemeh Jamshidi, name: Tathagata Bhattacharya, name: Nilanjana Raychawdhury, name: Dongji Feng, name: Vinija Jain, name: Aman Chadha
Published: 2025-07-30 14:39:51

Large Language Models (LLMs) have demonstrated remarkable proficiency in
generating text that closely resemble human writing. However, they often
generate factually incorrect statements, a problem typically referred to as
'hallucination'. Addressing hallucination is crucial for enhancing the
reliability and effectiveness of LLMs. While much research has focused on
hallucinations in English, our study extends this investigation to
conversational data in three languages: Hindi, Farsi, and Mandarin. We offer a
comprehensive analysis of a dataset to examine both factual and linguistic
errors in these languages for GPT-3.5, GPT-4o, Llama-3.1, Gemma-2.0,
DeepSeek-R1 and Qwen-3. We found that LLMs produce very few hallucinated
responses in Mandarin but generate a significantly higher number of
hallucinations in Hindi and Farsi.
3D-R1: Enhancing Reasoning in 3D VLMs for Unified Scene Understanding
Authors: name: Ting Huang, name: Zeyu Zhang, name: Hao Tang
Published: 2025-07-31 11:59:06

Large vision-language models (VLMs) have made significant strides in 2D
visual understanding tasks, sparking interest in extending these capabilities
to 3D scene understanding. However, current 3D VLMs often struggle with robust
reasoning and generalization due to limitations in high-quality spatial data
and the static nature of viewpoint assumptions. To address these challenges, we
propose 3D-R1, a foundation model that enhances the reasoning capabilities of
3D VLMs. Specifically, we first construct a high-quality synthetic dataset with
CoT, named Scene-30K, leveraging existing 3D-VL datasets and a data engine
based on Gemini 2.5 Pro. It serves as cold-start initialization data for 3D-R1.
Moreover, we leverage RLHF policy such as GRPO in the reinforcement learning
training process to enhance reasoning capabilities and introduce three reward
functions: a perception reward, a semantic similarity reward and a format
reward to maintain detection accuracy and answer semantic precision.
Furthermore, we introduce a dynamic view selection strategy that adaptively
chooses the most informative perspectives for 3D scene understanding. Extensive
experiments demonstrate that 3D-R1 delivers an average improvement of 10%
across various 3D scene benchmarks, highlighting its effectiveness in enhancing
reasoning and generalization in 3D scene understanding. Code:
https://github.com/AIGeeksGroup/3D-R1. Website:
https://aigeeksgroup.github.io/3D-R1.
IGL-Nav: Incremental 3D Gaussian Localization for Image-goal Navigation
Authors: name: Wenxuan Guo, name: Xiuwei Xu, name: Hang Yin, name: Ziwei Wang, name: Jianjiang Feng, name: Jie Zhou, name: Jiwen Lu
Published: 2025-08-01 17:59:56

Visual navigation with an image as goal is a fundamental and challenging
problem. Conventional methods either rely on end-to-end RL learning or
modular-based policy with topological graph or BEV map as memory, which cannot
fully model the geometric relationship between the explored 3D environment and
the goal image. In order to efficiently and accurately localize the goal image
in 3D space, we build our navigation system upon the renderable 3D gaussian
(3DGS) representation. However, due to the computational intensity of 3DGS
optimization and the large search space of 6-DoF camera pose, directly
leveraging 3DGS for image localization during agent exploration process is
prohibitively inefficient. To this end, we propose IGL-Nav, an Incremental 3D
Gaussian Localization framework for efficient and 3D-aware image-goal
navigation. Specifically, we incrementally update the scene representation as
new images arrive with feed-forward monocular prediction. Then we coarsely
localize the goal by leveraging the geometric information for discrete space
matching, which can be equivalent to efficient 3D convolution. When the agent
is close to the goal, we finally solve the fine target pose with optimization
via differentiable rendering. The proposed IGL-Nav outperforms existing
state-of-the-art methods by a large margin across diverse experimental
configurations. It can also handle the more challenging free-view image-goal
setting and be deployed on real-world robotic platform using a cellphone to
capture goal image at arbitrary pose. Project page:
https://gwxuan.github.io/IGL-Nav/.
Learning an Efficient Multi-Turn Dialogue Evaluator from Multiple Judges
Authors: name: Yuqi Tang, name: Kehua Feng, name: Yunfeng Wang, name: Zhiwen Chen, name: Chengfei Lv, name: Gang Yu, name: Qiang Zhang, name: Keyan Ding
Published: 2025-08-01 09:26:01

Evaluating the conversational abilities of large language models (LLMs)
remains a challenging task. Current mainstream approaches primarily rely on the
``LLM-as-a-judge" paradigm, where an LLM is prompted to serve as an evaluator
to assess dialogue quality. However, such methods often suffer from various
biases, which undermine the reliability and consistency of the evaluation
results. To mitigate these biases, recent methods employ multiple LLMs as
judges and aggregate their judgments to select the optimal assessment. Although
effective, this multi-judge approach incurs significant computational overhead
during inference. In this paper, we propose an efficient multi-turn dialogue
evaluator that captures the collective wisdom of multiple LLM judges by
aggregating their preference knowledge into a single model. Our approach
preserves the advantages of diverse multi-judge feedback while drastically
reducing the evaluation cost, enabling fast and flexible dialogue quality
assessment. Extensive experiments on seven single rating and pairwise
comparison dialogue evaluation benchmarks demonstrate that our method
outperforms existing baselines across diverse scenarios, showcasing its
efficiency and robustness.
Beyond Fixed: Variable-Length Denoising for Diffusion Large Language
  Models
Authors: name: Jinsong Li, name: Xiaoyi Dong, name: Yuhang Zang, name: Yuhang Cao, name: Jiaqi Wang, name: Dahua Lin
Published: 2025-08-01 17:56:07

Diffusion Large Language Models (DLLMs) are emerging as a powerful
alternative to the dominant Autoregressive Large Language Models, offering
efficient parallel generation and capable global context modeling. However, the
practical application of DLLMs is hindered by a critical architectural
constraint: the need for a statically predefined generation length. This static
length allocation leads to a problematic trade-off: insufficient lengths
cripple performance on complex tasks, while excessive lengths incur significant
computational overhead and sometimes result in performance degradation. While
the inference framework is rigid, we observe that the model itself possesses
internal signals that correlate with the optimal response length for a given
task. To bridge this gap, we leverage these latent signals and introduce
DAEDAL, a novel training-free denoising strategy that enables Dynamic Adaptive
Length Expansion for Diffusion Large Language Models. DAEDAL operates in two
phases: 1) Before the denoising process, DAEDAL starts from a short initial
length and iteratively expands it to a coarse task-appropriate length, guided
by a sequence completion metric. 2) During the denoising process, DAEDAL
dynamically intervenes by pinpointing and expanding insufficient generation
regions through mask token insertion, ensuring the final output is fully
developed. Extensive experiments on DLLMs demonstrate that DAEDAL achieves
performance comparable, and in some cases superior, to meticulously tuned
fixed-length baselines, while simultaneously enhancing computational efficiency
by achieving a higher effective token ratio. By resolving the static length
constraint, DAEDAL unlocks new potential for DLLMs, bridging a critical gap
with their Autoregressive counterparts and paving the way for more efficient
and capable generation.
SWE-Exp: Experience-Driven Software Issue Resolution
Authors: name: Silin Chen, name: Shaoxin Lin, name: Xiaodong Gu, name: Yuling Shi, name: Heng Lian, name: Longfei Yun, name: Dong Chen, name: Weiguo Sun, name: Lin Cao, name: Qianxiang Wang
Published: 2025-07-31 09:13:42

Recent advances in large language model (LLM) agents have shown remarkable
progress in software issue resolution, leveraging advanced techniques such as
multi-agent collaboration and Monte Carlo Tree Search (MCTS). However, current
agents act as memoryless explorers - treating each problem separately without
retaining or reusing knowledge from previous repair experiences. This leads to
redundant exploration of failed trajectories and missed chances to adapt
successful issue resolution methods to similar problems. To address this
problem, we introduce SWE-Exp, an experience - enhanced approach that distills
concise and actionable experience from prior agent trajectories, enabling
continuous learning across issues. Our method introduces a multi-faceted
experience bank that captures both successful and failed repair attempts.
Specifically, it extracts reusable issue resolution knowledge at different
levels - from high-level problem comprehension to specific code changes.
Experiments show that SWE-Exp achieves state-of-the-art resolution rate (41.6%
Pass@1) on SWE-bench-Verified under open-source agent frameworks. Our approach
establishes a new paradigm in which automated software engineering agents
systematically accumulate and leverage repair expertise, fundamentally shifting
from trial-and-error exploration to strategic, experience-driven issue
resolution.
SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolution
Authors: name: Han Li, name: Yuling Shi, name: Shaoxin Lin, name: Xiaodong Gu, name: Heng Lian, name: Xin Wang, name: Yantao Jia, name: Tao Huang, name: Qianxiang Wang
Published: 2025-07-31 08:54:46

Issue resolution has made remarkable progress thanks to the advanced
reasoning capabilities of large language models (LLMs). Recently, agent-based
frameworks such as SWE-agent have further advanced this progress by enabling
autonomous, tool-using agents to tackle complex software engineering tasks.
While existing agent-based issue resolution approaches are primarily based on
agents' independent explorations, they often get stuck in local solutions and
fail to identify issue patterns that span across different parts of the
codebase. To address this limitation, we propose SWE-Debate, a competitive
multi-agent debate framework that encourages diverse reasoning paths and
achieves more consolidated issue localization. SWE-Debate first creates
multiple fault propagation traces as localization proposals by traversing a
code dependency graph. Then, it organizes a three-round debate among
specialized agents, each embodying distinct reasoning perspectives along the
fault propagation trace. This structured competition enables agents to
collaboratively converge on a consolidated fix plan. Finally, this consolidated
fix plan is integrated into an MCTS-based code modification agent for patch
generation. Experiments on the SWE-bench benchmark show that SWE-Debate
achieves new state-of-the-art results in open-source agent frameworks and
outperforms baselines by a large margin.
PixNerd: Pixel Neural Field Diffusion
Authors: name: Shuai Wang, name: Ziteng Gao, name: Chenhui Zhu, name: Weilin Huang, name: Limin Wang
Published: 2025-07-31 06:07:20

The current success of diffusion transformers heavily depends on the
compressed latent space shaped by the pre-trained variational autoencoder(VAE).
However, this two-stage training paradigm inevitably introduces accumulated
errors and decoding artifacts. To address the aforementioned problems,
researchers return to pixel space at the cost of complicated cascade pipelines
and increased token complexity. In contrast to their efforts, we propose to
model the patch-wise decoding with neural field and present a single-scale,
single-stage, efficient, end-to-end solution, coined as pixel neural field
diffusion~(PixelNerd). Thanks to the efficient neural field representation in
PixNerd, we directly achieved 2.15 FID on ImageNet 256times256 and 2.84 FID
on ImageNet 512times512 without any complex cascade pipeline or VAE. We also
extend our PixNerd framework to text-to-image applications. Our PixNerd-XXL/16
achieved a competitive 0.73 overall score on the GenEval benchmark and 80.9
overall score on the DPG benchmark.
Multimodal Referring Segmentation: A Survey
Authors: name: Henghui Ding, name: Song Tang, name: Shuting He, name: Chang Liu, name: Zuxuan Wu, name: Yu-Gang Jiang
Published: 2025-08-01 02:14:00

Multimodal referring segmentation aims to segment target objects in visual
scenes, such as images, videos, and 3D scenes, based on referring expressions
in text or audio format. This task plays a crucial role in practical
applications requiring accurate object perception based on user instructions.
Over the past decade, it has gained significant attention in the multimodal
community, driven by advances in convolutional neural networks, transformers,
and large language models, all of which have substantially improved multimodal
perception capabilities. This paper provides a comprehensive survey of
multimodal referring segmentation. We begin by introducing this field's
background, including problem definitions and commonly used datasets. Next, we
summarize a unified meta architecture for referring segmentation and review
representative methods across three primary visual scenes, including images,
videos, and 3D scenes. We further discuss Generalized Referring Expression
(GREx) methods to address the challenges of real-world complexity, along with
related tasks and practical applications. Extensive performance comparisons on
standard benchmarks are also provided. We continually track related works at
https://github.com/henghuiding/Awesome-Multimodal-Referring-Segmentation.
iLRM: An Iterative Large 3D Reconstruction Model
Authors: name: Gyeongjin Kang, name: Seungtae Nam, name: Xiangyu Sun, name: Sameh Khamis, name: Abdelrahman Mohamed, name: Eunbyung Park
Published: 2025-07-31 06:33:07

Feed-forward 3D modeling has emerged as a promising approach for rapid and
high-quality 3D reconstruction. In particular, directly generating explicit 3D
representations, such as 3D Gaussian splatting, has attracted significant
attention due to its fast and high-quality rendering, as well as numerous
applications. However, many state-of-the-art methods, primarily based on
transformer architectures, suffer from severe scalability issues because they
rely on full attention across image tokens from multiple input views, resulting
in prohibitive computational costs as the number of views or image resolution
increases. Toward a scalable and efficient feed-forward 3D reconstruction, we
introduce an iterative Large 3D Reconstruction Model (iLRM) that generates 3D
Gaussian representations through an iterative refinement mechanism, guided by
three core principles: (1) decoupling the scene representation from input-view
images to enable compact 3D representations; (2) decomposing fully-attentional
multi-view interactions into a two-stage attention scheme to reduce
computational costs; and (3) injecting high-resolution information at every
layer to achieve high-fidelity reconstruction. Experimental results on widely
used datasets, such as RE10K and DL3DV, demonstrate that iLRM outperforms
existing methods in both reconstruction quality and speed. Notably, iLRM
exhibits superior scalability, delivering significantly higher reconstruction
quality under comparable computational cost by efficiently leveraging a larger
number of input views.
Efficient Machine Unlearning via Influence Approximation
Authors: name: Jiawei Liu, name: Chenwang Wu, name: Defu Lian, name: Enhong Chen
Published: 2025-07-31 05:34:27

Due to growing privacy concerns, machine unlearning, which aims at enabling
machine learning models to ``forget" specific training data, has received
increasing attention. Among existing methods, influence-based unlearning has
emerged as a prominent approach due to its ability to estimate the impact of
individual training samples on model parameters without retraining. However,
this approach suffers from prohibitive computational overhead arising from the
necessity to compute the Hessian matrix and its inverse across all training
samples and parameters, rendering it impractical for large-scale models and
scenarios involving frequent data deletion requests. This highlights the
difficulty of forgetting. Inspired by cognitive science, which suggests that
memorizing is easier than forgetting, this paper establishes a theoretical link
between memorizing (incremental learning) and forgetting (unlearning). This
connection allows machine unlearning to be addressed from the perspective of
incremental learning. Unlike the time-consuming Hessian computations in
unlearning (forgetting), incremental learning (memorizing) typically relies on
more efficient gradient optimization, which supports the aforementioned
cognitive theory. Based on this connection, we introduce the Influence
Approximation Unlearning (IAU) algorithm for efficient machine unlearning from
the incremental perspective. Extensive empirical evaluations demonstrate that
IAU achieves a superior balance among removal guarantee, unlearning efficiency,
and comparable model utility, while outperforming state-of-the-art methods
across diverse datasets and model architectures. Our code is available at
https://github.com/Lolo1222/IAU.
TARS: MinMax Token-Adaptive Preference Strategy for Hallucination
  Reduction in MLLMs
Authors: name: Kejia Zhang, name: Keda Tao, name: Zhiming Luo, name: Chang Liu, name: Jiasheng Tang, name: Huan Wang
Published: 2025-07-29 08:39:19

Multimodal large language models (MLLMs) enable vision-language reasoning,
yet often generate plausible outputs that are factually incorrect or visually
ungrounded, thereby compromising their reliability. Direct preference
optimization (DPO) is a common strategy for correcting hallucinations by
aligning model outputs with human preferences. Existing DPO strategies
typically treat hallucination-related preferences as fixed targets, relying on
static supervision signals during training. This approach tends to overfit to
superficial linguistic cues in preference data, leading to distributional
rigidity and spurious correlations that impair grounding in causally relevant
visual information. To overcome this limitation, we propose TARS, a
token-adaptive preference strategy that reformulates DPO as a min-max
optimization problem. TARS maximizes token-level distributional shifts under
semantic constraints to simulate alignment uncertainty, and simultaneously
minimizes the expected preference loss under these controlled perturbations.
This joint objective preserves causal grounding while mitigating overfitting to
preference patterns, thereby reducing hallucinations in multimodal reasoning.
We evaluate TARS on multiple hallucination benchmarks and find consistently
strong performance. Using only 4.8k preference samples and no expert feedback,
TARS reduces hallucination rates from 26.4% to 13.2% and decreases cognition
value from 2.5 to 0.4. It outperforms standard DPO and matches GPT-4o on
several key metrics.