Orionnnn/hub-stats huggingface dataset:
CannyEdit: Selective Canny Control and Dual-Prompt Guidance for
  Training-Free Image Editing
Authors: name: Weiyan Xie, name: Han Gao, name: Didan Deng, name: Kaican Li, name: April Hua Liu, name: Yongxiang Huang, name: Nevin L. Zhang
Published: 2025-08-09 11:06:58

Recent advances in text-to-image (T2I) models have enabled training-free
regional image editing by leveraging the generative priors of foundation
models. However, existing methods struggle to balance text adherence in edited
regions, context fidelity in unedited areas, and seamless integration of edits.
We introduce CannyEdit, a novel training-free framework that addresses these
challenges through two key innovations: (1) Selective Canny Control, which
masks the structural guidance of Canny ControlNet in user-specified editable
regions while strictly preserving details of the source images in unedited
areas via inversion-phase ControlNet information retention. This enables
precise, text-driven edits without compromising contextual integrity. (2)
Dual-Prompt Guidance, which combines local prompts for object-specific edits
with a global target prompt to maintain coherent scene interactions. On
real-world image editing tasks (addition, replacement, removal), CannyEdit
outperforms prior methods like KV-Edit, achieving a 2.93 to 10.49 percent
improvement in the balance of text adherence and context fidelity. In terms of
editing seamlessness, user studies reveal only 49.2 percent of general users
and 42.0 percent of AIGC experts identified CannyEdit's results as AI-edited
when paired with real images without edits, versus 76.08 to 89.09 percent for
competitor methods.
Decentralized Aerial Manipulation of a Cable-Suspended Load using
  Multi-Agent Reinforcement Learning
Authors: name: Jack Zeng, name: Andreu Matoses Gimenez, name: Eugene Vinitsky, name: Javier Alonso-Mora, name: Sihao Sun
Published: 2025-08-02 23:52:33

This paper presents the first decentralized method to enable real-world 6-DoF
manipulation of a cable-suspended load using a team of Micro-Aerial Vehicles
(MAVs). Our method leverages multi-agent reinforcement learning (MARL) to train
an outer-loop control policy for each MAV. Unlike state-of-the-art controllers
that utilize a centralized scheme, our policy does not require global states,
inter-MAV communications, nor neighboring MAV information. Instead, agents
communicate implicitly through load pose observations alone, which enables high
scalability and flexibility. It also significantly reduces computing costs
during inference time, enabling onboard deployment of the policy. In addition,
we introduce a new action space design for the MAVs using linear acceleration
and body rates. This choice, combined with a robust low-level controller,
enables reliable sim-to-real transfer despite significant uncertainties caused
by cable tension during dynamic 3D motion. We validate our method in various
real-world experiments, including full-pose control under load model
uncertainties, showing setpoint tracking performance comparable to the
state-of-the-art centralized method. We also demonstrate cooperation amongst
agents with heterogeneous control policies, and robustness to the complete
in-flight loss of one MAV. Videos of experiments:
https://autonomousrobots.nl/paper_websites/aerial-manipulation-marl
GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video
  Diffusion Priors
Authors: name: Xingyilang Yin, name: Qi Zhang, name: Jiahao Chang, name: Ying Feng, name: Qingnan Fan, name: Xi Yang, name: Chi-Man Pun, name: Huaqi Zhang, name: Xiaodong Cun
Published: 2025-08-13 09:56:28

Reconstructing 3D scenes using 3D Gaussian Splatting (3DGS) from sparse views
is an ill-posed problem due to insufficient information, often resulting in
noticeable artifacts. While recent approaches have sought to leverage
generative priors to complete information for under-constrained regions, they
struggle to generate content that remains consistent with input observations.
To address this challenge, we propose GSFixer, a novel framework designed to
improve the quality of 3DGS representations reconstructed from sparse inputs.
The core of our approach is the reference-guided video restoration model, built
upon a DiT-based video diffusion model trained on paired artifact 3DGS renders
and clean frames with additional reference-based conditions. Considering the
input sparse views as references, our model integrates both 2D semantic
features and 3D geometric features of reference views extracted from the visual
geometry foundation model, enhancing the semantic coherence and 3D consistency
when fixing artifact novel views. Furthermore, considering the lack of suitable
benchmarks for 3DGS artifact restoration evaluation, we present DL3DV-Res which
contains artifact frames rendered using low-quality 3DGS. Extensive experiments
demonstrate our GSFixer outperforms current state-of-the-art methods in 3DGS
artifact restoration and sparse-view 3D reconstruction. Project page:
https://github.com/GVCLab/GSFixer.
AMFT: Aligning LLM Reasoners by Meta-Learning the Optimal
  Imitation-Exploration Balance
Authors: name: Lixuan He, name: Jie Feng, name: Yong Li
Published: 2025-08-09 11:40:54

Large Language Models (LLMs) are typically fine-tuned for reasoning tasks
through a two-stage pipeline of Supervised Fine-Tuning (SFT) followed by
Reinforcement Learning (RL), a process fraught with catastrophic forgetting and
suboptimal trade-offs between imitation and exploration. Recent single-stage
methods attempt to unify SFT and RL using heuristics, but lack a principled
mechanism for dynamically balancing the two paradigms. In this paper, we
reframe this challenge through the theoretical lens of implicit
rewards, viewing SFT and RL not as distinct methods but as complementary
reward signals. We introduce Adaptive Meta Fine-Tuning (AMFT), a novel
single-stage algorithm that learns the optimal balance between SFT's implicit,
path-level reward and RL's explicit, outcome-based reward. The core of AMFT is
a meta-gradient adaptive weight controller that treats the SFT-RL
balance as a learnable parameter, dynamically optimizing it to maximize
long-term task performance. This forward-looking approach, regularized by
policy entropy for stability, autonomously discovers an effective training
curriculum. We conduct a comprehensive evaluation on challenging benchmarks
spanning mathematical reasoning, abstract visual reasoning (General Points),
and vision-language navigation (V-IRL). AMFT consistently establishes a new
state-of-the-art and demonstrats superior generalization on out-of-distribution
(OOD) tasks. Ablation studies and training dynamic analysis confirm that the
meta-learning controller is crucial for AMFT's stability, sample efficiency,
and performance, offering a more principled and effective paradigm for LLM
alignment.Our codes are open-sourced via https://github.com/hlxtsyj/AMFT.
VisCodex: Unified Multimodal Code Generation via Merging Vision and
  Coding Models
Authors: name: Lingjie Jiang, name: Shaohan Huang, name: Xun Wu, name: Yixia Li, name: Dongdong Zhang, name: Furu Wei
Published: 2025-08-13 17:00:44

Multimodal large language models (MLLMs) have significantly advanced the
integration of visual and textual understanding. However, their ability to
generate code from multimodal inputs remains limited. In this work, we
introduce VisCodex, a unified framework that seamlessly merges vision and
coding language models to empower MLLMs with strong multimodal code generation
abilities. Leveraging a task vector-based model merging technique, we integrate
a state-of-the-art coding LLM into a strong vision-language backbone, while
preserving both visual comprehension and advanced coding skills. To support
training and evaluation, we introduce the Multimodal Coding Dataset (MCD), a
large-scale and diverse collection of 598k samples, including high-quality HTML
code, chart image-code pairs, image-augmented StackOverflow QA, and algorithmic
problems. Furthermore, we propose InfiBench-V, a novel and challenging
benchmark specifically designed to assess models on visually-rich, real-world
programming questions that demand a nuanced understanding of both textual and
visual contexts. Extensive experiments show that VisCodex achieves
state-of-the-art performance among open-source MLLMs and approaches proprietary
models like GPT-4o, highlighting the effectiveness of our model merging
strategy and new datasets.
Learning to Align, Aligning to Learn: A Unified Approach for
  Self-Optimized Alignment
Authors: name: Haowen Wang, name: Yun Yue, name: Zhiling Ye, name: Shuowen Zhang, name: Lei Fan, name: Jiaxin Liang, name: Jiadi Jiang, name: Cheng Wei, name: Jingyuan Deng, name: Xudong Han, name: Ji Li, name: Chunxiao Guo, name: Peng Wei, name: Jian Wang, name: Jinjie Gu
Published: 2025-08-11 08:28:47

Alignment methodologies have emerged as a critical pathway for enhancing
language model alignment capabilities. While SFT (supervised fine-tuning)
accelerates convergence through direct token-level loss intervention, its
efficacy is constrained by offline policy trajectory. In contrast,
RL(reinforcement learning) facilitates exploratory policy optimization, but
suffers from low sample efficiency and stringent dependency on high-quality
base models. To address these dual challenges, we propose GRAO (Group Relative
Alignment Optimization), a unified framework that synergizes the respective
strengths of SFT and RL through three key innovations: 1) A multi-sample
generation strategy enabling comparative quality assessment via reward
feedback; 2) A novel Group Direct Alignment Loss formulation leveraging
intra-group relative advantage weighting; 3) Reference-aware parameter updates
guided by pairwise preference dynamics. Our theoretical analysis establishes
GRAO's convergence guarantees and sample efficiency advantages over
conventional approaches. Comprehensive evaluations across complex human
alignment tasks demonstrate GRAO's superior performance, achieving
57.70\%,17.65\% 7.95\% and 5.18\% relative improvements over SFT, DPO, PPO and
GRPO baselines respectively. This work provides both a theoretically grounded
alignment framework and empirical evidence for efficient capability evolution
in language models.
Can LLM-Generated Textual Explanations Enhance Model Classification
  Performance? An Empirical Study
Authors: name: Mahdi Dhaini, name: Juraj Vladika, name: Ege Erdogan, name: Zineb Attaoui, name: Gjergji Kasneci
Published: 2025-08-13 12:59:08

In the rapidly evolving field of Explainable Natural Language Processing
(NLP), textual explanations, i.e., human-like rationales, are pivotal for
explaining model predictions and enriching datasets with interpretable labels.
Traditional approaches rely on human annotation, which is costly,
labor-intensive, and impedes scalability. In this work, we present an automated
framework that leverages multiple state-of-the-art large language models (LLMs)
to generate high-quality textual explanations. We rigorously assess the quality
of these LLM-generated explanations using a comprehensive suite of Natural
Language Generation (NLG) metrics. Furthermore, we investigate the downstream
impact of these explanations on the performance of pre-trained language models
(PLMs) and LLMs across natural language inference tasks on two diverse
benchmark datasets. Our experiments demonstrate that automated explanations
exhibit highly competitive effectiveness compared to human-annotated
explanations in improving model performance. Our findings underscore a
promising avenue for scalable, automated LLM-based textual explanation
generation for extending NLP datasets and enhancing model performance.
Stand-In: A Lightweight and Plug-and-Play Identity Control for Video
  Generation
Authors: name: Bowen Xue, name: Qixin Yan, name: Wenjing Wang, name: Hao Liu, name: Chen Li
Published: 2025-08-11 12:17:38

Generating high-fidelity human videos that match user-specified identities is
important yet challenging in the field of generative AI. Existing methods often
rely on an excessive number of training parameters and lack compatibility with
other AIGC tools. In this paper, we propose Stand-In, a lightweight and
plug-and-play framework for identity preservation in video generation.
Specifically, we introduce a conditional image branch into the pre-trained
video generation model. Identity control is achieved through restricted
self-attentions with conditional position mapping, and can be learned quickly
with only 2000 pairs. Despite incorporating and training just sim1\%
additional parameters, our framework achieves excellent results in video
quality and identity preservation, outperforming other full-parameter training
methods. Moreover, our framework can be seamlessly integrated for other tasks,
such as subject-driven video generation, pose-referenced video generation,
stylization, and face swapping.
Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with
  Long-Term Memory
Authors: name: Lin Long, name: Yichen He, name: Wentao Ye, name: Yiyuan Pan, name: Yuan Lin, name: Hang Li, name: Junbo Zhao, name: Wei Li
Published: 2025-08-13 12:03:03

We introduce M3-Agent, a novel multimodal agent framework equipped with
long-term memory. Like humans, M3-Agent can process real-time visual and
auditory inputs to build and update its long-term memory. Beyond episodic
memory, it also develops semantic memory, enabling it to accumulate world
knowledge over time. Its memory is organized in an entity-centric, multimodal
format, allowing deeper and more consistent understanding of the environment.
Given an instruction, M3-Agent autonomously performs multi-turn, iterative
reasoning and retrieves relevant information from memory to accomplish the
task. To evaluate memory effectiveness and memory-based reasoning in multimodal
agents, we develop M3-Bench, a new long-video question answering benchmark.
M3-Bench comprises 100 newly recorded real-world videos captured from a robot's
perspective (M3-Bench-robot) and 929 web-sourced videos across diverse
scenarios (M3-Bench-web). We annotate question-answer pairs designed to test
key capabilities essential for agent applications, such as human understanding,
general knowledge extraction, and cross-modal reasoning. Experimental results
show that M3-Agent, trained via reinforcement learning, outperforms the
strongest baseline, a prompting agent using Gemini-1.5-pro and GPT-4o,
achieving 6.7%, 7.7%, and 5.3% higher accuracy on M3-Bench-robot, M3-Bench-web
and VideoMME-long, respectively. Our work advances the multimodal agents toward
more human-like long-term memory and provides insights into their practical
design. Model, code and data are available at
https://github.com/bytedance-seed/m3-agent
Story2Board: A Training-Free Approach for Expressive Storyboard
  Generation
Authors: name: David Dinkevich, name: Matan Levy, name: Omri Avrahami, name: Dvir Samuel, name: Dani Lischinski
Published: 2025-08-13 17:56:26

We present Story2Board, a training-free framework for expressive storyboard
generation from natural language. Existing methods narrowly focus on subject
identity, overlooking key aspects of visual storytelling such as spatial
composition, background evolution, and narrative pacing. To address this, we
introduce a lightweight consistency framework composed of two components:
Latent Panel Anchoring, which preserves a shared character reference across
panels, and Reciprocal Attention Value Mixing, which softly blends visual
features between token pairs with strong reciprocal attention. Together, these
mechanisms enhance coherence without architectural changes or fine-tuning,
enabling state-of-the-art diffusion models to generate visually diverse yet
consistent storyboards. To structure generation, we use an off-the-shelf
language model to convert free-form stories into grounded panel-level prompts.
To evaluate, we propose the Rich Storyboard Benchmark, a suite of open-domain
narratives designed to assess layout diversity and background-grounded
storytelling, in addition to consistency. We also introduce a new Scene
Diversity metric that quantifies spatial and pose variation across storyboards.
Our qualitative and quantitative results, as well as a user study, show that
Story2Board produces more dynamic, coherent, and narratively engaging
storyboards than existing baselines.
Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved
  Image Generation
Authors: name: Junyan Ye, name: Dongzhi Jiang, name: Zihao Wang, name: Leqi Zhu, name: Zhenghao Hu, name: Zilong Huang, name: Jun He, name: Zhiyuan Yan, name: Jinghua Yu, name: Hongsheng Li, name: Conghui He, name: Weijia Li
Published: 2025-08-13 17:59:28

Recently, GPT-4o has garnered significant attention for its strong
performance in image generation, yet open-source models still lag behind.
Several studies have explored distilling image data from GPT-4o to enhance
open-source models, achieving notable progress. However, a key question
remains: given that real-world image datasets already constitute a natural
source of high-quality data, why should we use GPT-4o-generated synthetic data?
In this work, we identify two key advantages of synthetic images. First, they
can complement rare scenarios in real-world datasets, such as surreal fantasy
or multi-reference image generation, which frequently occur in user queries.
Second, they provide clean and controllable supervision. Real-world data often
contains complex background noise and inherent misalignment between text
descriptions and image content, whereas synthetic images offer pure backgrounds
and long-tailed supervision signals, facilitating more accurate text-to-image
alignment. Building on these insights, we introduce Echo-4o-Image, a 180K-scale
synthetic dataset generated by GPT-4o, harnessing the power of synthetic image
data to address blind spots in real-world coverage. Using this dataset, we
fine-tune the unified multimodal generation baseline Bagel to obtain Echo-4o.
In addition, we propose two new evaluation benchmarks for a more accurate and
challenging assessment of image generation capabilities: GenEval++, which
increases instruction complexity to mitigate score saturation, and
Imagine-Bench, which focuses on evaluating both the understanding and
generation of imaginative content. Echo-4o demonstrates strong performance
across standard benchmarks. Moreover, applying Echo-4o-Image to other
foundation models (e.g., OmniGen2, BLIP3-o) yields consistent performance gains
across multiple metrics, highlighting the datasets strong transferability.
Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion
  Forcing
Authors: name: Xu Wang, name: Chenkai Xu, name: Yijie Jin, name: Jiachun Jin, name: Hao Zhang, name: Zhijie Deng
Published: 2025-08-08 04:51:37

Diffusion Large Language Models (dLLMs) have emerged as a promising
alternative to autoregressive (AR) LLMs for text generation, with the potential
to decode multiple tokens in a single iteration. However, none of the existing
open-source dLLMs have achieved superior inference speed over AR LLMs of
similar size. This paper breaks this barrier based on a simple and effective
strategy named discrete diffusion forcing (D2F). D2F equips dLLMs with two key
capabilities: (1) block-wise autoregressive generation to enable KV cache
utilization; (2) prediction of following tokens without requiring completion of
prior blocks for inter-block parallel decoding. In this way, the vanilla dLLMs
are refurbished into an AR-diffusion hybrid paradigm for efficient inference.
D2F can be implemented with an asymmetric distillation process based on
pre-trained dLLMs. We further propose a pipelined parallel decoding algorithm,
which enables a trade-off between efficiency and efficacy. Empirically, D2F
dLLMs achieve more than 2.5times inference speed than LLaMA3 and
Qwen2.5 on GSM8K. Compared to vanilla dLLMs like LLaDA and Dream, the
acceleration can be more than 50times while maintaining comparable
output quality. The code is available at
https://github.com/zhijie-group/Discrete-Diffusion-Forcing.
Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models
Authors: name: Luca Eyring, name: Shyamgopal Karthik, name: Alexey Dosovitskiy, name: Nataniel Ruiz, name: Zeynep Akata
Published: 2025-08-13 17:33:37

The new paradigm of test-time scaling has yielded remarkable breakthroughs in
Large Language Models (LLMs) (e.g. reasoning models) and in generative vision
models, allowing models to allocate additional computation during inference to
effectively tackle increasingly complex problems. Despite the improvements of
this approach, an important limitation emerges: the substantial increase in
computation time makes the process slow and impractical for many applications.
Given the success of this paradigm and its growing usage, we seek to preserve
its benefits while eschewing the inference overhead. In this work we propose
one solution to the critical problem of integrating test-time scaling knowledge
into a model during post-training. Specifically, we replace reward guided
test-time noise optimization in diffusion models with a Noise Hypernetwork that
modulates initial input noise. We propose a theoretically grounded framework
for learning this reward-tilted distribution for distilled generators, through
a tractable noise-space objective that maintains fidelity to the base model
while optimizing for desired characteristics. We show that our approach
recovers a substantial portion of the quality gains from explicit test-time
optimization at a fraction of the computational cost. Code is available at
https://github.com/ExplainableML/HyperNoise
IAG: Input-aware Backdoor Attack on VLMs for Visual Grounding
Authors: name: Junxian Li, name: Beining Xu, name: Di Zhang
Published: 2025-08-13 03:22:19

Vision-language models (VLMs) have shown significant advancements in tasks
such as visual grounding, where they localize specific objects in images based
on natural language queries and images. However, security issues in visual
grounding tasks for VLMs remain underexplored, especially in the context of
backdoor attacks. In this paper, we introduce a novel input-aware backdoor
attack method, IAG, designed to manipulate the grounding behavior of VLMs. This
attack forces the model to ground a specific target object in the input image,
regardless of the user's query. We propose an adaptive trigger generator that
embeds the semantic information of the attack target's description into the
original image using a text-conditional U-Net, thereby overcoming the
open-vocabulary attack challenge. To ensure the attack's stealthiness, we
utilize a reconstruction loss to minimize visual discrepancies between poisoned
and clean images. Additionally, we introduce a unified method for generating
attack data. IAG is evaluated theoretically and empirically, demonstrating its
feasibility and effectiveness. Notably, our ASR@0.5 on InternVL-2.5-8B reaches
over 65\% on various testing sets. IAG also shows promising potential on
manipulating Ferret-7B and LlaVA-1.5-7B with very little accuracy decrease on
clean samples. Extensive specific experiments, such as ablation study and
potential defense, also indicate the robustness and transferability of our
attack.