Orionnnn/hub-stats huggingface dataset:
Beyond Decoder-only: Large Language Models Can be Good Encoders for
  Machine Translation
Authors: name: Yingfeng Luo, name: Tong Zheng, name: Yongyu Mu, name: Bei Li, name: Qinghong Zhang, name: Yongqi Gao, name: Ziqiang Xu, name: Peinan Feng, name: Xiaoqian Liu, name: Tong Xiao, name: Jingbo Zhu
Published: 2025-03-09 12:54:05

The field of neural machine translation (NMT) has changed with the advent of
large language models (LLMs). Much of the recent emphasis in natural language
processing (NLP) has been on modeling machine translation and many other
problems using a single pre-trained Transformer decoder, while encoder-decoder
architectures, which were the standard in earlier NMT models, have received
relatively less attention. In this paper, we explore translation models that
are universal, efficient, and easy to optimize, by marrying the world of LLMs
with the world of NMT. We apply LLMs to NMT encoding and leave the NMT decoder
unchanged. We also develop methods for adapting LLMs to work better with the
NMT decoder. Furthermore, we construct a new dataset involving multiple tasks
to assess how well the machine translation system generalizes across various
tasks. Evaluations on the WMT and our datasets show that results using our
method match or surpass a range of baselines in terms of translation quality,
but achieve 2.4 sim 6.5 times inference speedups and a 75% reduction in
the memory footprint of the KV cache. It also demonstrates strong
generalization across a variety of translation-related tasks.
NullFace: Training-Free Localized Face Anonymization
Authors: name: Han-Wei Kung, name: Tuomas Varanka, name: Terence Sim, name: Nicu Sebe
Published: 2025-03-11 14:29:37

Privacy concerns around ever increasing number of cameras are increasing in
today's digital age. Although existing anonymization methods are able to
obscure identity information, they often struggle to preserve the utility of
the images. In this work, we introduce a training-free method for face
anonymization that preserves key non-identity-related attributes. Our approach
utilizes a pre-trained text-to-image diffusion model without requiring
optimization or training. It begins by inverting the input image to recover its
initial noise. The noise is then denoised through an identity-conditioned
diffusion process, where modified identity embeddings ensure the anonymized
face is distinct from the original identity. Our approach also supports
localized anonymization, giving users control over which facial regions are
anonymized or kept intact. Comprehensive evaluations against state-of-the-art
methods show our approach excels in anonymization, attribute preservation, and
image quality. Its flexibility, robustness, and practicality make it
well-suited for real-world applications. Code and data can be found at
https://github.com/hanweikung/nullface .
VisualSimpleQA: A Benchmark for Decoupled Evaluation of Large
  Vision-Language Models in Fact-Seeking Question Answering
Authors: name: Yanling Wang, name: Yihan Zhao, name: Xiaodong Chen, name: Shasha Guo, name: Lixin Liu, name: Haoyang Li, name: Yong Xiao, name: Jing Zhang, name: Qi Li, name: Ke Xu
Published: 2025-03-09 07:25:32

Large vision-language models (LVLMs) have demonstrated remarkable
achievements, yet the generation of non-factual responses remains prevalent in
fact-seeking question answering (QA). Current multimodal fact-seeking
benchmarks primarily focus on comparing model outputs to ground truth answers,
providing limited insights into the performance of modality-specific modules.
To bridge this gap, we introduce VisualSimpleQA, a multimodal fact-seeking
benchmark with two key features. First, it enables streamlined and decoupled
evaluation of LVLMs in visual and linguistic modalities. Second, it
incorporates well-defined difficulty criteria to guide human annotation and
facilitates the extraction of a challenging subset, VisualSimpleQA-hard.
Experiments on 15 LVLMs show that even state-of-the-art models such as GPT-4o
achieve merely 60%+ correctness in multimodal fact-seeking QA on VisualSimpleQA
and 30%+ on VisualSimpleQA-hard. Furthermore, the decoupled evaluation across
these models highlights substantial opportunities for improvement in both
visual and linguistic modules. The dataset is available at
https://huggingface.co/datasets/WYLing/VisualSimpleQA.
AI-native Memory 2.0: Second Me
Authors: name: Jiale Wei, name: Xiang Ying, name: Tao Gao, name: Felix Tao, name: Jingbo Shang
Published: 2025-03-11 07:05:52

Human interaction with the external world fundamentally involves the exchange
of personal memory, whether with other individuals, websites, applications, or,
in the future, AI agents. A significant portion of this interaction is
redundant, requiring users to repeatedly provide the same information across
different contexts. Existing solutions, such as browser-stored credentials,
autofill mechanisms, and unified authentication systems, have aimed to mitigate
this redundancy by serving as intermediaries that store and retrieve commonly
used user data. The advent of large language models (LLMs) presents an
opportunity to redefine memory management through an AI-native paradigm: SECOND
ME. SECOND ME acts as an intelligent, persistent memory offload system that
retains, organizes, and dynamically utilizes user-specific knowledge. By
serving as an intermediary in user interactions, it can autonomously generate
context-aware responses, prefill required information, and facilitate seamless
communication with external systems, significantly reducing cognitive load and
interaction friction. Unlike traditional memory storage solutions, SECOND ME
extends beyond static data retention by leveraging LLM-based memory
parameterization. This enables structured organization, contextual reasoning,
and adaptive knowledge retrieval, facilitating a more systematic and
intelligent approach to memory management. As AI-driven personal agents like
SECOND ME become increasingly integrated into digital ecosystems, SECOND ME
further represents a critical step toward augmenting human-world interaction
with persistent, contextually aware, and self-optimizing memory systems. We
have open-sourced the fully localizable deployment system at GitHub:
https://github.com/Mindverse/Second-Me.
MagicInfinite: Generating Infinite Talking Videos with Your Words and
  Voice
Authors: name: Hongwei Yi, name: Tian Ye, name: Shitong Shao, name: Xuancheng Yang, name: Jiantong Zhao, name: Hanzhong Guo, name: Terrance Wang, name: Qingyu Yin, name: Zeke Xie, name: Lei Zhu, name: Wei Li, name: Michael Lingelbach, name: Daquan Zhou
Published: 2025-03-07 23:21:11

We present MagicInfinite, a novel diffusion Transformer (DiT) framework that
overcomes traditional portrait animation limitations, delivering high-fidelity
results across diverse character types-realistic humans, full-body figures, and
stylized anime characters. It supports varied facial poses, including
back-facing views, and animates single or multiple characters with input masks
for precise speaker designation in multi-character scenes. Our approach tackles
key challenges with three innovations: (1) 3D full-attention mechanisms with a
sliding window denoising strategy, enabling infinite video generation with
temporal coherence and visual quality across diverse character styles; (2) a
two-stage curriculum learning scheme, integrating audio for lip sync, text for
expressive dynamics, and reference images for identity preservation, enabling
flexible multi-modal control over long sequences; and (3) region-specific masks
with adaptive loss functions to balance global textual control and local audio
guidance, supporting speaker-specific animations. Efficiency is enhanced via
our innovative unified step and cfg distillation techniques, achieving a 20x
inference speed boost over the basemodel: generating a 10 second 540x540p video
in 10 seconds or 720x720p in 30 seconds on 8 H100 GPUs, without quality loss.
Evaluations on our new benchmark demonstrate MagicInfinite's superiority in
audio-lip synchronization, identity preservation, and motion naturalness across
diverse scenarios. It is publicly available at https://www.hedra.com/, with
examples at https://magicinfinite.github.io/.
BiasEdit: Debiasing Stereotyped Language Models via Model Editing
Authors: name: Xin Xu, name: Wei Xu, name: Ningyu Zhang, name: Julian McAuley
Published: 2025-03-11 16:25:36

Previous studies have established that language models manifest stereotyped
biases. Existing debiasing strategies, such as retraining a model with
counterfactual data, representation projection, and prompting often fail to
efficiently eliminate bias or directly alter the models' biased internal
representations. To address these issues, we propose BiasEdit, an efficient
model editing method to remove stereotypical bias from language models through
lightweight networks that act as editors to generate parameter updates.
BiasEdit employs a debiasing loss guiding editor networks to conduct local
edits on partial parameters of a language model for debiasing while preserving
the language modeling abilities during editing through a retention loss.
Experiments on StereoSet and Crows-Pairs demonstrate the effectiveness,
efficiency, and robustness of BiasEdit in eliminating bias compared to
tangental debiasing baselines and little to no impact on the language models'
general capabilities. In addition, we conduct bias tracing to probe bias in
various modules and explore bias editing impacts on different components of
language models.
RayFlow: Instance-Aware Diffusion Acceleration via Adaptive Flow
  Trajectories
Authors: name: Huiyang Shao, name: Xin Xia, name: Yuhong Yang, name: Yuxi Ren, name: Xing Wang, name: Xuefeng Xiao
Published: 2025-03-10 17:20:52

Diffusion models have achieved remarkable success across various domains.
However, their slow generation speed remains a critical challenge. Existing
acceleration methods, while aiming to reduce steps, often compromise sample
quality, controllability, or introduce training complexities. Therefore, we
propose RayFlow, a novel diffusion framework that addresses these limitations.
Unlike previous methods, RayFlow guides each sample along a unique path towards
an instance-specific target distribution. This method minimizes sampling steps
while preserving generation diversity and stability. Furthermore, we introduce
Time Sampler, an importance sampling technique to enhance training efficiency
by focusing on crucial timesteps. Extensive experiments demonstrate RayFlow's
superiority in generating high-quality images with improved speed, control, and
training efficiency compared to existing acceleration techniques.
Evaluating Intelligence via Trial and Error
Authors: name: Jingtao Zhan, name: Jiahao Zhao, name: Jiayu Li, name: Yiqun Liu, name: Bo Zhang, name: Qingyao Ai, name: Jiaxin Mao, name: Hongning Wang, name: Min Zhang, name: Shaoping Ma
Published: 2025-02-26 05:59:45

Intelligence is a crucial trait for species to find solutions within a
limited number of trial-and-error attempts. Building on this idea, we introduce
Survival Game as a framework to evaluate intelligence based on the number of
failed attempts in a trial-and-error process. Fewer failures indicate higher
intelligence. When the expectation and variance of failure counts are both
finite, it signals the ability to consistently find solutions to new
challenges, which we define as the Autonomous Level of intelligence. Using
Survival Game, we comprehensively evaluate existing AI systems. Our results
show that while AI systems achieve the Autonomous Level in simple tasks, they
are still far from it in more complex tasks, such as vision, search,
recommendation, and language. While scaling current AI technologies might help,
this would come at an astronomical cost. Projections suggest that achieving the
Autonomous Level for general tasks would require 10^{26} parameters. To put
this into perspective, loading such a massive model requires so many H100 GPUs
that their total value is 10^{7} times that of Apple Inc.'s market value.
Even with Moore's Law, supporting such a parameter scale would take 70 years.
This staggering cost highlights the complexity of human tasks and the
inadequacies of current AI technologies. To further investigate this
phenomenon, we conduct a theoretical analysis of Survival Game and its
experimental results. Our findings suggest that human tasks possess a
criticality property. As a result, Autonomous Level requires a deep
understanding of the task's underlying mechanisms. Current AI systems, however,
do not fully grasp these mechanisms and instead rely on superficial mimicry,
making it difficult for them to reach an autonomous level. We believe Survival
Game can not only guide the future development of AI but also offer profound
insights into human intelligence.
Gemini Embedding: Generalizable Embeddings from Gemini
Authors: name: Jinhyuk Lee, name: Feiyang Chen, name: Sahil Dua, name: Daniel Cer, name: Madhuri Shanbhogue, name: Iftekhar Naim, name: Gustavo Hernández Ábrego, name: Zhe Li, name: Kaifeng Chen, name: Henrique Schechter Vera, name: Xiaoqi Ren, name: Shanfeng Zhang, name: Daniel Salz, name: Michael Boratko, name: Jay Han, name: Blair Chen, name: Shuo Huang, name: Vikram Rao, name: Paul Suganthan, name: Feng Han, name: Andreas Doumanoglou, name: Nithi Gupta, name: Fedor Moiseev, name: Cathy Yip, name: Aashi Jain, name: Simon Baumgartner, name: Shahrokh Shahi, name: Frank Palma Gomez, name: Sandeep Mariserla, name: Min Choi, name: Parashar Shah, name: Sonam Goenka, name: Ke Chen, name: Ye Xia, name: Koert Chen, name: Sai Meher Karthik Duddu, name: Yichang Chen, name: Trevor Walker, name: Wenlei Zhou, name: Rakesh Ghiya, name: Zach Gleicher, name: Karan Gill, name: Zhe Dong, name: Mojtaba Seyedhosseini, name: Yunhsuan Sung, name: Raphael Hoffmann, name: Tom Duerig
Published: 2025-03-10 22:16:45

In this report, we introduce Gemini Embedding, a state-of-the-art embedding
model leveraging the power of Gemini, Google's most capable large language
model. Capitalizing on Gemini's inherent multilingual and code understanding
capabilities, Gemini Embedding produces highly generalizable embeddings for
text spanning numerous languages and textual modalities. The representations
generated by Gemini Embedding can be precomputed and applied to a variety of
downstream tasks including classification, similarity, clustering, ranking, and
retrieval. Evaluated on the Massive Multilingual Text Embedding Benchmark
(MMTEB), which includes over one hundred tasks across 250+ languages, Gemini
Embedding substantially outperforms prior state-of-the-art models,
demonstrating considerable improvements in embedding quality. Achieving
state-of-the-art performance across MMTEB's multilingual, English, and code
benchmarks, our unified model demonstrates strong capabilities across a broad
selection of tasks and surpasses specialized domain-specific models.
YuE: Scaling Open Foundation Models for Long-Form Music Generation
Authors: name: Ruibin Yuan, name: Hanfeng Lin, name: Shuyue Guo, name: Ge Zhang, name: Jiahao Pan, name: Yongyi Zang, name: Haohe Liu, name: Yiming Liang, name: Wenye Ma, name: Xingjian Du, name: Xinrun Du, name: Zhen Ye, name: Tianyu Zheng, name: Yinghao Ma, name: Minghao Liu, name: Zeyue Tian, name: Ziya Zhou, name: Liumeng Xue, name: Xingwei Qu, name: Yizhi Li, name: Shangda Wu, name: Tianhao Shen, name: Ziyang Ma, name: Jun Zhan, name: Chunhui Wang, name: Yatian Wang, name: Xiaowei Chi, name: Xinyue Zhang, name: Zhenzhu Yang, name: Xiangzhou Wang, name: Shansong Liu, name: Lingrui Mei, name: Peng Li, name: Junjie Wang, name: Jianwei Yu, name: Guojian Pang, name: Xu Li, name: Zihao Wang, name: Xiaohuan Zhou, name: Lijun Yu, name: Emmanouil Benetos, name: Yong Chen, name: Chenghua Lin, name: Xie Chen, name: Gus Xia, name: Zhaoxiang Zhang, name: Chao Zhang, name: Wenhu Chen, name: Xinyu Zhou, name: Xipeng Qiu, name: Roger Dannenberg, name: Jiaheng Liu, name: Jian Yang, name: Wenhao Huang, name: Wei Xue, name: Xu Tan, name: Yike Guo
Published: 2025-03-11 17:26:50

We tackle the task of long-form music generation--particularly the
challenging lyrics-to-song problem--by introducing YuE, a family of
open foundation models based on the LLaMA2 architecture. Specifically, YuE
scales to trillions of tokens and generates up to five minutes of music while
maintaining lyrical alignment, coherent musical structure, and engaging vocal
melodies with appropriate accompaniment. It achieves this through (1)
track-decoupled next-token prediction to overcome dense mixture signals, (2)
structural progressive conditioning for long-context lyrical alignment, and (3)
a multitask, multiphase pre-training recipe to converge and generalize. In
addition, we redesign the in-context learning technique for music generation,
enabling versatile style transfer (e.g., converting Japanese city pop into an
English rap while preserving the original accompaniment) and bidirectional
generation. Through extensive evaluation, we demonstrate that YuE matches or
even surpasses some of the proprietary systems in musicality and vocal agility.
In addition, fine-tuning YuE enables additional controls and enhanced support
for tail languages. Furthermore, beyond generation, we show that YuE's learned
representations can perform well on music understanding tasks, where the
results of YuE match or exceed state-of-the-art methods on the MARBLE
benchmark. Keywords: lyrics2song, song generation, long-form, foundation model,
music generation
LightGen: Efficient Image Generation through Knowledge Distillation and
  Direct Preference Optimization
Authors: name: Xianfeng Wu, name: Yajing Bai, name: Haoze Zheng, name: Harold Haodong Chen, name: Yexin Liu, name: Zihao Wang, name: Xuran Ma, name: Wen-Jie Shu, name: Xianzu Wu, name: Harry Yang, name: Ser-Nam Lim
Published: 2025-03-11 16:58:02

Recent advances in text-to-image generation have primarily relied on
extensive datasets and parameter-heavy architectures. These requirements
severely limit accessibility for researchers and practitioners who lack
substantial computational resources. In this paper, we introduce \model, an
efficient training paradigm for image generation models that uses knowledge
distillation (KD) and Direct Preference Optimization (DPO). Drawing inspiration
from the success of data KD techniques widely adopted in Multi-Modal Large
Language Models (MLLMs), LightGen distills knowledge from state-of-the-art
(SOTA) text-to-image models into a compact Masked Autoregressive (MAR)
architecture with only 0.7B parameters. Using a compact synthetic dataset of
just 2M high-quality images generated from varied captions, we demonstrate
that data diversity significantly outweighs data volume in determining model
performance. This strategy dramatically reduces computational demands and
reduces pre-training time from potentially thousands of GPU-days to merely 88
GPU-days. Furthermore, to address the inherent shortcomings of synthetic data,
particularly poor high-frequency details and spatial inaccuracies, we integrate
the DPO technique that refines image fidelity and positional accuracy.
Comprehensive experiments confirm that LightGen achieves image generation
quality comparable to SOTA models while significantly reducing computational
resources and expanding accessibility for resource-constrained environments.
Code is available at https://github.com/XianfengWu01/LightGen
SegAgent: Exploring Pixel Understanding Capabilities in MLLMs by
  Imitating Human Annotator Trajectories
Authors: name: Muzhi Zhu, name: Yuzhuo Tian, name: Hao Chen, name: Chunluan Zhou, name: Qingpei Guo, name: Yang Liu, name: Ming Yang, name: Chunhua Shen
Published: 2025-03-11 17:08:54

While MLLMs have demonstrated adequate image understanding capabilities, they
still struggle with pixel-level comprehension, limiting their practical
applications. Current evaluation tasks like VQA and visual grounding remain too
coarse to assess fine-grained pixel comprehension accurately. Though
segmentation is foundational for pixel-level understanding, existing methods
often require MLLMs to generate implicit tokens, decoded through external pixel
decoders. This approach disrupts the MLLM's text output space, potentially
compromising language capabilities and reducing flexibility and extensibility,
while failing to reflect the model's intrinsic pixel-level understanding.
  Thus, we introduce the Human-Like Mask Annotation Task (HLMAT), a new
paradigm where MLLMs mimic human annotators using interactive segmentation
tools. Modeling segmentation as a multi-step Markov Decision Process, HLMAT
enables MLLMs to iteratively generate text-based click points, achieving
high-quality masks without architectural changes or implicit tokens. Through
this setup, we develop SegAgent, a model fine-tuned on human-like annotation
trajectories, which achieves performance comparable to state-of-the-art (SOTA)
methods and supports additional tasks like mask refinement and annotation
filtering.
  HLMAT provides a protocol for assessing fine-grained pixel understanding in
MLLMs and introduces a vision-centric, multi-step decision-making task that
facilitates exploration of MLLMs' visual reasoning abilities. Our adaptations
of policy improvement method StaR and PRM-guided tree search further enhance
model robustness in complex segmentation tasks, laying a foundation for future
advancements in fine-grained visual perception and multi-step decision-making
for MLLMs.
QuoTA: Query-oriented Token Assignment via CoT Query Decouple for Long
  Video Comprehension
Authors: name: Yongdong Luo, name: Wang Chen, name: Xiawu Zheng, name: Weizhong Huang, name: Shukang Yin, name: Haojia Lin, name: Chaoyou Fu, name: Jinfa Huang, name: Jiayi Ji, name: Jiebo Luo, name: Rongrong Ji
Published: 2025-03-11 17:59:57

Recent advances in long video understanding typically mitigate visual
redundancy through visual token pruning based on attention distribution.
However, while existing methods employ post-hoc low-response token pruning in
decoder layers, they overlook the input-level semantic correlation between
visual tokens and instructions (query). In this paper, we propose QuoTA, an
ante-hoc training-free modular that extends existing large video-language
models (LVLMs) for visual token assignment based on query-oriented frame-level
importance assessment. The query-oriented token selection is crucial as it
aligns visual processing with task-specific requirements, optimizing token
budget utilization while preserving semantically relevant content.
Specifically, (i) QuoTA strategically allocates frame-level importance scores
based on query relevance, enabling one-time visual token assignment before
cross-modal interactions in decoder layers, (ii) we decouple the query through
Chain-of-Thoughts reasoning to facilitate more precise LVLM-based frame
importance scoring, and (iii) QuoTA offers a plug-and-play functionality that
extends to existing LVLMs. Extensive experimental results demonstrate that
implementing QuoTA with LLaVA-Video-7B yields an average performance
improvement of 3.2% across six benchmarks (including Video-MME and MLVU) while
operating within an identical visual token budget as the baseline. Codes are
open-sourced at https://github.com/MAC-AutoML/QuoTA.