Orionnnn/hub-stats huggingface dataset:
In-Context Edit: Enabling Instructional Image Editing with In-Context
  Generation in Large Scale Diffusion Transformer
Authors: name: Zechuan Zhang, name: Ji Xie, name: Yu Lu, name: Zongxin Yang, name: Yi Yang
Published: 2025-04-29 12:14:47

Instruction-based image editing enables robust image modification via natural
language prompts, yet current methods face a precision-efficiency tradeoff.
Fine-tuning methods demand significant computational resources and large
datasets, while training-free techniques struggle with instruction
comprehension and edit quality. We resolve this dilemma by leveraging
large-scale Diffusion Transformer (DiT)' enhanced generation capacity and
native contextual awareness. Our solution introduces three contributions: (1)
an in-context editing framework for zero-shot instruction compliance using
in-context prompting, avoiding structural changes; (2) a LoRA-MoE hybrid tuning
strategy that enhances flexibility with efficient adaptation and dynamic expert
routing, without extensive retraining; and (3) an early filter inference-time
scaling method using vision-language models (VLMs) to select better initial
noise early, improving edit quality. Extensive evaluations demonstrate our
method's superiority: it outperforms state-of-the-art approaches while
requiring only 0.5% training data and 1% trainable parameters compared to
conventional baselines. This work establishes a new paradigm that enables
high-precision yet efficient instruction-guided editing. Codes and demos can be
found in https://river-zhang.github.io/ICEdit-gh-pages/.
TreeHop: Generate and Filter Next Query Embeddings Efficiently for
  Multi-hop Question Answering
Authors: name: Zhonghao Li, name: Kunpeng Zhang, name: Jinghuai Ou, name: Shuliang Liu, name: Xuming Hu
Published: 2025-04-28 01:56:31

Retrieval-augmented generation (RAG) systems face significant challenges in
multi-hop question answering (MHQA), where complex queries require synthesizing
information across multiple document chunks. Existing approaches typically rely
on iterative LLM-based query rewriting and routing, resulting in high
computational costs due to repeated LLM invocations and multi-stage processes.
To address these limitations, we propose TreeHop, an embedding-level framework
without the need for LLMs in query refinement. TreeHop dynamically updates
query embeddings by fusing semantic information from prior queries and
retrieved documents, enabling iterative retrieval through embedding-space
operations alone. This method replaces the traditional
"Retrieve-Rewrite-Vectorize-Retrieve" cycle with a streamlined
"Retrieve-Embed-Retrieve" loop, significantly reducing computational overhead.
Moreover, a rule-based stop criterion is introduced to further prune redundant
retrievals, balancing efficiency and recall rate. Experimental results show
that TreeHop rivals advanced RAG methods across three open-domain MHQA
datasets, achieving comparable performance with only 5\%-0.4\% of the model
parameter size and reducing the query latency by approximately 99\% compared to
concurrent approaches. This makes TreeHop a faster and more cost-effective
solution for deployment in a range of knowledge-intensive applications. For
reproducibility purposes, codes and data are available here:
https://github.com/allen-li1231/TreeHop.
ISDrama: Immersive Spatial Drama Generation through Multimodal Prompting
Authors: name: Yu Zhang, name: Wenxiang Guo, name: Changhao Pan, name: Zhiyuan Zhu, name: Tao Jin, name: Zhou Zhao
Published: 2025-04-29 10:56:44

Multimodal immersive spatial drama generation focuses on creating continuous
multi-speaker binaural speech with dramatic prosody based on multimodal
prompts, with potential applications in AR, VR, and others. This task requires
simultaneous modeling of spatial information and dramatic prosody based on
multimodal inputs, with high data collection costs. To the best of our
knowledge, our work is the first attempt to address these challenges. We
construct MRSDrama, the first multimodal recorded spatial drama dataset,
containing binaural drama audios, scripts, videos, geometric poses, and textual
prompts. Then, we propose ISDrama, the first immersive spatial drama generation
model through multimodal prompting. ISDrama comprises these primary components:
1) Multimodal Pose Encoder, based on contrastive learning, considering the
Doppler effect caused by moving speakers to extract unified pose information
from multimodal prompts. 2) Immersive Drama Transformer, a flow-based
mamba-transformer model that generates high-quality drama, incorporating
Drama-MOE to select proper experts for enhanced prosody and pose control. We
also design a context-consistent classifier-free guidance strategy to
coherently generate complete drama. Experimental results show that ISDrama
outperforms baseline models on objective and subjective metrics. The demos and
dataset are available at https://aaronz345.github.io/ISDramaDemo.
Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional
  Talking Portrait Generation
Authors: name: Weipeng Tan, name: Chuming Lin, name: Chengming Xu, name: FeiFan Xu, name: Xiaobin Hu, name: Xiaozhong Ji, name: Junwei Zhu, name: Chengjie Wang, name: Yanwei Fu
Published: 2025-04-25 05:28:21

Recent advances in Talking Head Generation (THG) have achieved impressive lip
synchronization and visual quality through diffusion models; yet existing
methods struggle to generate emotionally expressive portraits while preserving
speaker identity. We identify three critical limitations in current emotional
talking head generation: insufficient utilization of audio's inherent emotional
cues, identity leakage in emotion representations, and isolated learning of
emotion correlations. To address these challenges, we propose a novel framework
dubbed as DICE-Talk, following the idea of disentangling identity with emotion,
and then cooperating emotions with similar characteristics. First, we develop a
disentangled emotion embedder that jointly models audio-visual emotional cues
through cross-modal attention, representing emotions as identity-agnostic
Gaussian distributions. Second, we introduce a correlation-enhanced emotion
conditioning module with learnable Emotion Banks that explicitly capture
inter-emotion relationships through vector quantization and attention-based
feature aggregation. Third, we design an emotion discrimination objective that
enforces affective consistency during the diffusion process through
latent-space classification. Extensive experiments on MEAD and HDTF datasets
demonstrate our method's superiority, outperforming state-of-the-art approaches
in emotion accuracy while maintaining competitive lip-sync performance.
Qualitative results and user studies further confirm our method's ability to
generate identity-preserving portraits with rich, correlated emotional
expressions that naturally adapt to unseen identities.
X-Fusion: Introducing New Modality to Frozen Large Language Models
Authors: name: Sicheng Mo, name: Thao Nguyen, name: Xun Huang, name: Siddharth Srinivasan Iyer, name: Yijun Li, name: Yuchen Liu, name: Abhishek Tandon, name: Eli Shechtman, name: Krishna Kumar Singh, name: Yong Jae Lee, name: Bolei Zhou, name: Yuheng Li
Published: 2025-04-29 17:59:45

We propose X-Fusion, a framework that extends pretrained Large Language
Models (LLMs) for multimodal tasks while preserving their language
capabilities. X-Fusion employs a dual-tower design with modality-specific
weights, keeping the LLM's parameters frozen while integrating vision-specific
information for both understanding and generation. Our experiments demonstrate
that X-Fusion consistently outperforms alternative architectures on both
image-to-text and text-to-image tasks. We find that incorporating
understanding-focused data improves generation quality, reducing image data
noise enhances overall performance, and feature alignment accelerates
convergence for smaller models but has minimal impact on larger ones. Our
findings provide valuable insights into building efficient unified multimodal
models.
The Leaderboard Illusion
Authors: name: Shivalika Singh, name: Yiyang Nan, name: Alex Wang, name: Daniel D'Souza, name: Sayash Kapoor, name: Ahmet Üstün, name: Sanmi Koyejo, name: Yuntian Deng, name: Shayne Longpre, name: Noah Smith, name: Beyza Ermis, name: Marzieh Fadaee, name: Sara Hooker
Published: 2025-04-29 15:48:49

Measuring progress is fundamental to the advancement of any scientific field.
As benchmarks play an increasingly central role, they also grow more
susceptible to distortion. Chatbot Arena has emerged as the go-to leaderboard
for ranking the most capable AI systems. Yet, in this work we identify
systematic issues that have resulted in a distorted playing field. We find that
undisclosed private testing practices benefit a handful of providers who are
able to test multiple variants before public release and retract scores if
desired. We establish that the ability of these providers to choose the best
score leads to biased Arena scores due to selective disclosure of performance
results. At an extreme, we identify 27 private LLM variants tested by Meta in
the lead-up to the Llama-4 release. We also establish that proprietary closed
models are sampled at higher rates (number of battles) and have fewer models
removed from the arena than open-weight and open-source alternatives. Both
these policies lead to large data access asymmetries over time. Providers like
Google and OpenAI have received an estimated 19.2% and 20.4% of all data on the
arena, respectively. In contrast, a combined 83 open-weight models have only
received an estimated 29.7% of the total data. We show that access to Chatbot
Arena data yields substantial benefits; even limited additional data can result
in relative performance gains of up to 112% on the arena distribution, based on
our conservative estimates. Together, these dynamics result in overfitting to
Arena-specific dynamics rather than general model quality. The Arena builds on
the substantial efforts of both the organizers and an open community that
maintains this valuable evaluation platform. We offer actionable
recommendations to reform the Chatbot Arena's evaluation framework and promote
fairer, more transparent benchmarking for the field
Certified Mitigation of Worst-Case LLM Copyright Infringement
Authors: name: Jingyu Zhang, name: Jiacan Yu, name: Marc Marone, name: Benjamin Van Durme, name: Daniel Khashabi
Published: 2025-04-22 17:16:53

The exposure of large language models (LLMs) to copyrighted material during
pre-training raises concerns about unintentional copyright infringement post
deployment. This has driven the development of "copyright takedown" methods,
post-training approaches aimed at preventing models from generating content
substantially similar to copyrighted ones. While current mitigation approaches
are somewhat effective for average-case risks, we demonstrate that they
overlook worst-case copyright risks exhibits by the existence of long, verbatim
quotes from copyrighted sources. We propose BloomScrub, a remarkably simple yet
highly effective inference-time approach that provides certified copyright
takedown. Our method repeatedly interleaves quote detection with rewriting
techniques to transform potentially infringing segments. By leveraging
efficient data sketches (Bloom filters), our approach enables scalable
copyright screening even for large-scale real-world corpora. When quotes beyond
a length threshold cannot be removed, the system can abstain from responding,
offering certified risk reduction. Experimental results show that BloomScrub
reduces infringement risk, preserves utility, and accommodates different levels
of enforcement stringency with adaptive abstention. Our results suggest that
lightweight, inference-time methods can be surprisingly effective for copyright
prevention.
Toward Evaluative Thinking: Meta Policy Optimization with Evolving
  Reward Models
Authors: name: Zae Myung Kim, name: Chanwoo Park, name: Vipul Raheja, name: Dongyeop Kang
Published: 2025-04-28 18:02:35

Reward-based alignment methods for large language models (LLMs) face two key
limitations: vulnerability to reward hacking, where models exploit flaws in the
reward signal; and reliance on brittle, labor-intensive prompt engineering when
LLMs are used as reward models. We introduce Meta Policy Optimization (MPO), a
framework that addresses these challenges by integrating a meta-reward model
that dynamically refines the reward model's prompt throughout training. In MPO,
the meta-reward model monitors the evolving training context and continuously
adjusts the reward model's prompt to maintain high alignment, providing an
adaptive reward signal that resists exploitation by the policy. This
meta-learning approach promotes a more stable policy optimization, and greatly
reduces the need for manual reward prompt design. It yields performance on par
with or better than models guided by extensively hand-crafted reward prompts.
Furthermore, we show that MPO maintains its effectiveness across diverse tasks,
such as question answering and mathematical reasoning, without requiring
specialized reward designs. Beyond standard RLAIF, MPO's meta-learning
formulation is readily extensible to higher-level alignment frameworks.
Overall, this method addresses theoretical and practical challenges in
reward-based RL alignment for LLMs, paving the way for more robust and
adaptable alignment strategies. The code and models will be publicly shared.
UniversalRAG: Retrieval-Augmented Generation over Multiple Corpora with
  Diverse Modalities and Granularities
Authors: name: Woongyeong Yeo, name: Kangsan Kim, name: Soyeong Jeong, name: Jinheon Baek, name: Sung Ju Hwang
Published: 2025-04-29 13:18:58

Retrieval-Augmented Generation (RAG) has shown substantial promise in
improving factual accuracy by grounding model responses with external knowledge
relevant to queries. However, most existing RAG approaches are limited to a
text-only corpus, and while recent efforts have extended RAG to other
modalities such as images and videos, they typically operate over a single
modality-specific corpus. In contrast, real-world queries vary widely in the
type of knowledge they require, which a single type of knowledge source cannot
address. To address this, we introduce UniversalRAG, a novel RAG framework
designed to retrieve and integrate knowledge from heterogeneous sources with
diverse modalities and granularities. Specifically, motivated by the
observation that forcing all modalities into a unified representation space
derived from a single combined corpus causes a modality gap, where the
retrieval tends to favor items from the same modality as the query, we propose
a modality-aware routing mechanism that dynamically identifies the most
appropriate modality-specific corpus and performs targeted retrieval within it.
Also, beyond modality, we organize each modality into multiple granularity
levels, enabling fine-tuned retrieval tailored to the complexity and scope of
the query. We validate UniversalRAG on 8 benchmarks spanning multiple
modalities, showing its superiority over modality-specific and unified
baselines.
TesserAct: Learning 4D Embodied World Models
Authors: name: Haoyu Zhen, name: Qiao Sun, name: Hongxin Zhang, name: Junyan Li, name: Siyuan Zhou, name: Yilun Du, name: Chuang Gan
Published: 2025-04-29 17:59:30

This paper presents an effective approach for learning novel 4D embodied
world models, which predict the dynamic evolution of 3D scenes over time in
response to an embodied agent's actions, providing both spatial and temporal
consistency. We propose to learn a 4D world model by training on RGB-DN (RGB,
Depth, and Normal) videos. This not only surpasses traditional 2D models by
incorporating detailed shape, configuration, and temporal changes into their
predictions, but also allows us to effectively learn accurate inverse dynamic
models for an embodied agent. Specifically, we first extend existing robotic
manipulation video datasets with depth and normal information leveraging
off-the-shelf models. Next, we fine-tune a video generation model on this
annotated dataset, which jointly predicts RGB-DN (RGB, Depth, and Normal) for
each frame. We then present an algorithm to directly convert generated RGB,
Depth, and Normal videos into a high-quality 4D scene of the world. Our method
ensures temporal and spatial coherence in 4D scene predictions from embodied
scenarios, enables novel view synthesis for embodied environments, and
facilitates policy learning that significantly outperforms those derived from
prior video-based world models.
ReasonIR: Training Retrievers for Reasoning Tasks
Authors: name: Rulin Shao, name: Rui Qiao, name: Varsha Kishore, name: Niklas Muennighoff, name: Xi Victoria Lin, name: Daniela Rus, name: Bryan Kian Hsiang Low, name: Sewon Min, name: Wen-tau Yih, name: Pang Wei Koh, name: Luke Zettlemoyer
Published: 2025-04-29 09:49:28

We present ReasonIR-8B, the first retriever specifically trained for general
reasoning tasks. Existing retrievers have shown limited gains on reasoning
tasks, in part because existing training datasets focus on short factual
queries tied to documents that straightforwardly answer them. We develop a
synthetic data generation pipeline that, for each document, our pipeline
creates a challenging and relevant query, along with a plausibly related but
ultimately unhelpful hard negative. By training on a mixture of our synthetic
data and existing public data, ReasonIR-8B achieves a new state-of-the-art of
29.9 nDCG@10 without reranker and 36.9 nDCG@10 with reranker on BRIGHT, a
widely-used reasoning-intensive information retrieval (IR) benchmark. When
applied to RAG tasks, ReasonIR-8B improves MMLU and GPQA performance by 6.4%
and 22.6% respectively, relative to the closed-book baseline, outperforming
other retrievers and search engines. In addition, ReasonIR-8B uses test-time
compute more effectively: on BRIGHT, its performance consistently increases
with longer and more information-rich rewritten queries; it continues to
outperform other retrievers when combined with an LLM reranker. Our training
recipe is general and can be easily extended to future LLMs; to this end, we
open-source our code, data, and model.
YoChameleon: Personalized Vision and Language Generation
Authors: name: Thao Nguyen, name: Krishna Kumar Singh, name: Jing Shi, name: Trung Bui, name: Yong Jae Lee, name: Yuheng Li
Published: 2025-04-29 17:59:57

Large Multimodal Models (e.g., GPT-4, Gemini, Chameleon) have evolved into
powerful tools with millions of users. However, they remain generic models and
lack personalized knowledge of specific user concepts. Previous work has
explored personalization for text generation, yet it remains unclear how these
methods can be adapted to new modalities, such as image generation. In this
paper, we introduce Yo'Chameleon, the first attempt to study personalization
for large multimodal models. Given 3-5 images of a particular concept,
Yo'Chameleon leverages soft-prompt tuning to embed subject-specific information
to (i) answer questions about the subject and (ii) recreate pixel-level details
to produce images of the subject in new contexts. Yo'Chameleon is trained with
(i) a self-prompting optimization mechanism to balance performance across
multiple modalities, and (ii) a ``soft-positive" image generation approach to
enhance image quality in a few-shot setting.
Reinforcement Learning for Reasoning in Large Language Models with One
  Training Example
Authors: name: Yiping Wang, name: Qing Yang, name: Zhiyuan Zeng, name: Liliang Ren, name: Lucas Liu, name: Baolin Peng, name: Hao Cheng, name: Xuehai He, name: Kuan Wang, name: Jianfeng Gao, name: Weizhu Chen, name: Shuohang Wang, name: Simon Shaolei Du, name: Yelong Shen
Published: 2025-04-29 09:24:30

We show that reinforcement learning with verifiable reward using one training
example (1-shot RLVR) is effective in incentivizing the math reasoning
capabilities of large language models (LLMs). Applying RLVR to the base model
Qwen2.5-Math-1.5B, we identify a single example that elevates model performance
on MATH500 from 36.0% to 73.6%, and improves the average performance across six
common mathematical reasoning benchmarks from 17.6% to 35.7%. This result
matches the performance obtained using the 1.2k DeepScaleR subset (MATH500:
73.6%, average: 35.9%), which includes the aforementioned example. Similar
substantial improvements are observed across various models (Qwen2.5-Math-7B,
Llama3.2-3B-Instruct, DeepSeek-R1-Distill-Qwen-1.5B), RL algorithms (GRPO and
PPO), and different math examples (many of which yield approximately 30% or
greater improvement on MATH500 when employed as a single training example). In
addition, we identify some interesting phenomena during 1-shot RLVR, including
cross-domain generalization, increased frequency of self-reflection, and
sustained test performance improvement even after the training accuracy has
saturated, a phenomenon we term post-saturation generalization. Moreover, we
verify that the effectiveness of 1-shot RLVR primarily arises from the policy
gradient loss, distinguishing it from the "grokking" phenomenon. We also show
the critical role of promoting exploration (e.g., by adding entropy loss with
an appropriate coefficient) in 1-shot RLVR training. As a bonus, we observe
that applying entropy loss alone, without any outcome reward, significantly
enhances Qwen2.5-Math-1.5B's performance on MATH500 by 27.4%. These findings
can inspire future work on RLVR data efficiency and encourage a re-examination
of both recent progress and the underlying mechanisms in RLVR. Our code, model,
and data are open source at https://github.com/ypwang61/One-Shot-RLVR
Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
Authors: name: Prateek Chhikara, name: Dev Khant, name: Saket Aryan, name: Taranjeet Singh, name: Deshraj Yadav
Published: 2025-04-28 01:46:35

Large Language Models (LLMs) have demonstrated remarkable prowess in
generating contextually coherent responses, yet their fixed context windows
pose fundamental challenges for maintaining consistency over prolonged
multi-session dialogues. We introduce Mem0, a scalable memory-centric
architecture that addresses this issue by dynamically extracting,
consolidating, and retrieving salient information from ongoing conversations.
Building on this foundation, we further propose an enhanced variant that
leverages graph-based memory representations to capture complex relational
structures among conversational elements. Through comprehensive evaluations on
LOCOMO benchmark, we systematically compare our approaches against six baseline
categories: (i) established memory-augmented systems, (ii) retrieval-augmented
generation (RAG) with varying chunk sizes and k-values, (iii) a full-context
approach that processes the entire conversation history, (iv) an open-source
memory solution, (v) a proprietary model system, and (vi) a dedicated memory
management platform. Empirical results show that our methods consistently
outperform all existing memory systems across four question categories:
single-hop, temporal, multi-hop, and open-domain. Notably, Mem0 achieves 26%
relative improvements in the LLM-as-a-Judge metric over OpenAI, while Mem0 with
graph memory achieves around 2% higher overall score than the base
configuration. Beyond accuracy gains, we also markedly reduce computational
overhead compared to full-context method. In particular, Mem0 attains a 91%
lower p95 latency and saves more than 90% token cost, offering a compelling
balance between advanced reasoning capabilities and practical deployment
constraints. Our findings highlight critical role of structured, persistent
memory mechanisms for long-term conversational coherence, paving the way for
more reliable and efficient LLM-driven AI agents.