Orionnnn/hub-stats huggingface dataset:
Effective and Efficient Masked Image Generation Models
Authors: name: Zebin You, name: Jingyang Ou, name: Xiaolu Zhang, name: Jun Hu, name: Jun Zhou, name: Chongxuan Li
Published: 2025-03-10 11:27:12

Although masked image generation models and masked diffusion models are
designed with different motivations and objectives, we observe that they can be
unified within a single framework. Building upon this insight, we carefully
explore the design space of training and sampling, identifying key factors that
contribute to both performance and efficiency. Based on the improvements
observed during this exploration, we develop our model, referred to as eMIGM.
Empirically, eMIGM demonstrates strong performance on ImageNet generation, as
measured by Fr\'echet Inception Distance (FID). In particular, on ImageNet
256x256, with similar number of function evaluations (NFEs) and model
parameters, eMIGM outperforms the seminal VAR. Moreover, as NFE and model
parameters increase, eMIGM achieves performance comparable to the
state-of-the-art continuous diffusion models while requiring less than 40% of
the NFE. Additionally, on ImageNet 512x512, with only about 60% of the NFE,
eMIGM outperforms the state-of-the-art continuous diffusion models.
FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA
  Subparameter Updates
Authors: name: Sangwoo Park, name: Seanie Lee, name: Byungjoo Kim, name: Sung Ju Hwang
Published: 2025-03-10 11:55:50

Federated Learning (FL) is a widely used framework for training models in a
decentralized manner, ensuring that the central server does not have direct
access to data from local clients. However, this approach may still fail to
fully preserve data privacy, as models from local clients are exposed to the
central server during the aggregation process. This issue becomes even more
critical when training vision-language models (VLMs) with FL, as VLMs can
easily memorize training data instances, making them vulnerable to membership
inference attacks (MIAs). To address this challenge, we propose the FedRand
framework, which avoids disclosing the full set of client parameters. In this
framework, each client randomly selects subparameters of Low-Rank Adaptation
(LoRA) from the server and keeps the remaining counterparts of the LoRA weights
as private parameters. After training both parameters on the client's private
dataset, only the non-private client parameters are sent back to the server for
aggregation. This approach mitigates the risk of exposing client-side VLM
parameters, thereby enhancing data privacy. We empirically validate that
FedRand improves robustness against MIAs compared to relevant baselines while
achieving accuracy comparable to methods that communicate full LoRA parameters
across several benchmark datasets.
Agent models: Internalizing Chain-of-Action Generation into Reasoning
  models
Authors: name: Yuxiang Zhang, name: Yuqi Yang, name: Jiangming Shu, name: Xinyan Wen, name: Jitao Sang
Published: 2025-03-09 12:19:47

Traditional agentic workflows rely on external prompts to manage interactions
with tools and the environment, which limits the autonomy of reasoning models.
We position Large Agent Models (LAMs) that internalize the generation of
Chain-of-Action (CoA), enabling the model to autonomously decide when
and how to use external tools. Our proposed AutoCoA framework combines
supervised fine-tuning (SFT) and reinforcement learning (RL), allowing the
model to seamlessly switch between reasoning and action while efficiently
managing environment interactions. Main components include step-level action
triggering, trajectory-level CoA optimization, and an internal world model to
reduce real-environment interaction costs. Evaluations on open-domain QA tasks
demonstrate that AutoCoA-trained agent models significantly outperform
ReAct-based workflows in task completion, especially in tasks that require
long-term reasoning and multi-step actions. Code and dataset are available at
https://github.com/ADaM-BJTU/AutoCoA
Automated Movie Generation via Multi-Agent CoT Planning
Authors: name: Weijia Wu, name: Zeyu Zhu, name: Mike Zheng Shou
Published: 2025-03-10 13:33:27

Existing long-form video generation frameworks lack automated planning,
requiring manual input for storylines, scenes, cinematography, and character
interactions, resulting in high costs and inefficiencies. To address these
challenges, we present MovieAgent, an automated movie generation via
multi-agent Chain of Thought (CoT) planning. MovieAgent offers two key
advantages: 1) We firstly explore and define the paradigm of automated
movie/long-video generation. Given a script and character bank, our MovieAgent
can generates multi-scene, multi-shot long-form videos with a coherent
narrative, while ensuring character consistency, synchronized subtitles, and
stable audio throughout the film. 2) MovieAgent introduces a hierarchical
CoT-based reasoning process to automatically structure scenes, camera settings,
and cinematography, significantly reducing human effort. By employing multiple
LLM agents to simulate the roles of a director, screenwriter, storyboard
artist, and location manager, MovieAgent streamlines the production pipeline.
Experiments demonstrate that MovieAgent achieves new state-of-the-art results
in script faithfulness, character consistency, and narrative coherence. Our
hierarchical framework takes a step forward and provides new insights into
fully automated movie generation. The code and project website are available
at: https://github.com/showlab/MovieAgent and
https://weijiawu.github.io/MovieAgent.
SEAP: Training-free Sparse Expert Activation Pruning Unlock the
  Brainpower of Large Language Models
Authors: name: Xun Liang, name: Hanyu Wang, name: Huayi Lai, name: Simin Niu, name: Shichao Song, name: Jiawei Yang, name: Jihao Zhao, name: Feiyu Xiong, name: Bo Tang, name: Zhiyu Li
Published: 2025-03-10 17:59:03

Large Language Models have achieved remarkable success across various natural
language processing tasks, yet their high computational cost during inference
remains a major bottleneck. This paper introduces Sparse Expert Activation
Pruning (SEAP), a training-free pruning method that selectively retains
task-relevant parameters to reduce inference overhead. Inspired by the
clustering patterns of hidden states and activations in LLMs, SEAP identifies
task-specific expert activation patterns and prunes the model while preserving
task performance and enhancing computational efficiency. Experimental results
demonstrate that SEAP significantly reduces computational overhead while
maintaining competitive accuracy. Notably, at 50% pruning, SEAP surpasses both
WandA and FLAP by over 20%, and at 20% pruning, it incurs only a 2.2%
performance drop compared to the dense model. These findings highlight SEAP's
scalability and effectiveness, making it a promising approach for optimizing
large-scale LLMs.
Detection Avoidance Techniques for Large Language Models
Authors: name: Sinclair Schneider, name: Florian Steuber, name: Joao A. G. Schneider, name: Gabi Dreo Rodosek
Published: 2025-03-10 17:56:25

The increasing popularity of large language models has not only led to
widespread use but has also brought various risks, including the potential for
systematically spreading fake news. Consequently, the development of
classification systems such as DetectGPT has become vital. These detectors are
vulnerable to evasion techniques, as demonstrated in an experimental series:
Systematic changes of the generative models' temperature proofed shallow
learning-detectors to be the least reliable. Fine-tuning the generative model
via reinforcement learning circumvented BERT-based-detectors. Finally,
rephrasing led to a >90\% evasion of zero-shot-detectors like DetectGPT,
although texts stayed highly similar to the original. A comparison with
existing work highlights the better performance of the presented methods.
Possible implications for society and further research are discussed.
BlackGoose Rimer: Harnessing RWKV-7 as a Simple yet Superior Replacement
  for Transformers in Large-Scale Time Series Modeling
Authors: name: Li weile, name: Liu Xiao
Published: 2025-03-08 08:31:18

Time series models face significant challenges in scaling to handle large and
complex datasets, akin to the scaling achieved by large language models (LLMs).
The unique characteristics of time series data and the computational demands of
model scaling necessitate innovative approaches. While researchers have
explored various architectures such as Transformers, LSTMs, and GRUs to address
these challenges, we propose a novel solution using RWKV-7, which incorporates
meta-learning into its state update mechanism. By integrating RWKV-7's time mix
and channel mix components into the transformer-based time series model Timer,
we achieve a substantial performance improvement of approximately 1.13 to 43.3x
and a 4.5x reduction in training time with 1/23 parameters, all while utilizing
fewer parameters. Our code and model weights are publicly available for further
research and development at https://github.com/Alic-Li/BlackGoose_Rimer.
PE3R: Perception-Efficient 3D Reconstruction
Authors: name: Jie Hu, name: Shizun Wang, name: Xinchao Wang
Published: 2025-03-10 16:29:10

Recent advancements in 2D-to-3D perception have significantly improved the
understanding of 3D scenes from 2D images. However, existing methods face
critical challenges, including limited generalization across scenes, suboptimal
perception accuracy, and slow reconstruction speeds. To address these
limitations, we propose Perception-Efficient 3D Reconstruction (PE3R), a novel
framework designed to enhance both accuracy and efficiency. PE3R employs a
feed-forward architecture to enable rapid 3D semantic field reconstruction. The
framework demonstrates robust zero-shot generalization across diverse scenes
and objects while significantly improving reconstruction speed. Extensive
experiments on 2D-to-3D open-vocabulary segmentation and 3D reconstruction
validate the effectiveness and versatility of PE3R. The framework achieves a
minimum 9-fold speedup in 3D semantic field reconstruction, along with
substantial gains in perception accuracy and reconstruction precision, setting
new benchmarks in the field. The code is publicly available at:
https://github.com/hujiecpp/PE3R.
LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted
  Contrastive Learning
Authors: name: Zhibin Lan, name: Liqiang Niu, name: Fandong Meng, name: Jie Zhou, name: Jinsong Su
Published: 2025-03-04 10:21:57

Universal multimodal embedding models play a critical role in tasks such as
interleaved image-text retrieval, multimodal RAG, and multimodal clustering.
However, our empirical results indicate that existing LMM-based embedding
models trained with the standard InfoNCE loss exhibit a high degree of overlap
in similarity distribution between positive and negative pairs, making it
challenging to distinguish hard negative pairs effectively. To deal with this
issue, we propose a simple yet effective framework that dynamically improves
the embedding model's representation learning for negative pairs based on their
discriminative difficulty. Within this framework, we train a series of models,
named LLaVE, and evaluate them on the MMEB benchmark, which covers 4 meta-tasks
and 36 datasets. Experimental results show that LLaVE establishes stronger
baselines that achieve state-of-the-art (SOTA) performance while demonstrating
strong scalability and efficiency. Specifically, LLaVE-2B surpasses the
previous SOTA 7B models, while LLaVE-7B achieves a further performance
improvement of 6.2 points. Although LLaVE is trained on image-text data, it can
generalize to text-video retrieval tasks in a zero-shot manner and achieve
strong performance, demonstrating its remarkable potential for transfer to
other embedding tasks.
MM-Eureka: Exploring Visual Aha Moment with Rule-based Large-scale
  Reinforcement Learning
Authors: name: Fanqing Meng, name: Lingxiao Du, name: Zongkai Liu, name: Zhixiang Zhou, name: Quanfeng Lu, name: Daocheng Fu, name: Botian Shi, name: Wenhai Wang, name: Junjun He, name: Kaipeng Zhang, name: Ping Luo, name: Yu Qiao, name: Qiaosheng Zhang, name: Wenqi Shao
Published: 2025-03-10 14:23:12

We present MM-Eureka, a multimodal reasoning model that successfully extends
large-scale rule-based reinforcement learning (RL) to multimodal reasoning.
While rule-based RL has shown remarkable success in improving LLMs' reasoning
abilities in text domains, its application to multimodal settings has remained
challenging. Our work reproduces key characteristics of text-based RL systems
like DeepSeek-R1 in the multimodal space, including steady increases in
accuracy reward and response length, and the emergence of reflection behaviors.
We demonstrate that both instruction-tuned and pre-trained models can develop
strong multimodal reasoning capabilities through rule-based RL without
supervised fine-tuning, showing superior data efficiency compared to
alternative approaches. We open-source our complete pipeline to foster further
research in this area. We release all our codes, models, data, etc. at
https://github.com/ModalMinds/MM-EUREKA
Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive
  Reinforcement
Authors: name: Yuqi Liu, name: Bohao Peng, name: Zhisheng Zhong, name: Zihao Yue, name: Fanbin Lu, name: Bei Yu, name: Jiaya Jia
Published: 2025-03-09 08:48:51

Traditional methods for reasoning segmentation rely on supervised fine-tuning
with categorical labels and simple descriptions, limiting its out-of-domain
generalization and lacking explicit reasoning processes. To address these
limitations, we propose Seg-Zero, a novel framework that demonstrates
remarkable generalizability and derives explicit chain-of-thought reasoning
through cognitive reinforcement. Seg-Zero introduces a decoupled architecture
consisting of a reasoning model and a segmentation model. The reasoning model
interprets user intentions, generates explicit reasoning chains, and produces
positional prompts, which are subsequently used by the segmentation model to
generate precious pixel-level masks. We design a sophisticated reward mechanism
that integrates both format and accuracy rewards to effectively guide
optimization directions. Trained exclusively via reinforcement learning with
GRPO and without explicit reasoning data, Seg-Zero achieves robust zero-shot
generalization and exhibits emergent test-time reasoning capabilities.
Experiments show that Seg-Zero-7B achieves a zero-shot performance of 57.5 on
the ReasonSeg benchmark, surpassing the prior LISA-7B by 18\%. This significant
improvement highlights Seg-Zero's ability to generalize across domains while
presenting an explicit reasoning process. Code is available at
https://github.com/dvlab-research/Seg-Zero.
EasyControl: Adding Efficient and Flexible Control for Diffusion
  Transformer
Authors: name: Yuxuan Zhang, name: Yirui Yuan, name: Yiren Song, name: Haofan Wang, name: Jiaming Liu
Published: 2025-03-10 08:07:17

Recent advancements in Unet-based diffusion models, such as ControlNet and
IP-Adapter, have introduced effective spatial and subject control mechanisms.
However, the DiT (Diffusion Transformer) architecture still struggles with
efficient and flexible control. To tackle this issue, we propose EasyControl, a
novel framework designed to unify condition-guided diffusion transformers with
high efficiency and flexibility. Our framework is built on three key
innovations. First, we introduce a lightweight Condition Injection LoRA Module.
This module processes conditional signals in isolation, acting as a
plug-and-play solution. It avoids modifying the base model weights, ensuring
compatibility with customized models and enabling the flexible injection of
diverse conditions. Notably, this module also supports harmonious and robust
zero-shot multi-condition generalization, even when trained only on
single-condition data. Second, we propose a Position-Aware Training Paradigm.
This approach standardizes input conditions to fixed resolutions, allowing the
generation of images with arbitrary aspect ratios and flexible resolutions. At
the same time, it optimizes computational efficiency, making the framework more
practical for real-world applications. Third, we develop a Causal Attention
Mechanism combined with the KV Cache technique, adapted for conditional
generation tasks. This innovation significantly reduces the latency of image
synthesis, improving the overall efficiency of the framework. Through extensive
experiments, we demonstrate that EasyControl achieves exceptional performance
across various application scenarios. These innovations collectively make our
framework highly efficient, flexible, and suitable for a wide range of tasks.
State-offset Tuning: State-based Parameter-Efficient Fine-Tuning for
  State Space Models
Authors: name: Wonjun Kang, name: Kevin Galim, name: Yuchen Zeng, name: Minjae Lee, name: Hyung Il Koo, name: Nam Ik Cho
Published: 2025-03-05 13:44:42

State Space Models (SSMs) have emerged as efficient alternatives to
Transformers, mitigating their quadratic computational cost. However, the
application of Parameter-Efficient Fine-Tuning (PEFT) methods to SSMs remains
largely unexplored. In particular, prompt-based methods like Prompt Tuning and
Prefix-Tuning, which are widely used in Transformers, do not perform well on
SSMs. To address this, we propose state-based methods as a superior alternative
to prompt-based methods. This new family of methods naturally stems from the
architectural characteristics of SSMs. State-based methods adjust state-related
features directly instead of depending on external prompts. Furthermore, we
introduce a novel state-based PEFT method: State-offset Tuning. At every
timestep, our method directly affects the state at the current step, leading to
more effective adaptation. Through extensive experiments across diverse
datasets, we demonstrate the effectiveness of our method. Code is available at
https://github.com/furiosa-ai/ssm-state-tuning.
NeuGrasp: Generalizable Neural Surface Reconstruction with Background
  Priors for Material-Agnostic Object Grasp Detection
Authors: name: Qingyu Fan, name: Yinghao Cai, name: Chao Li, name: Wenzhe He, name: Xudong Zheng, name: Tao Lu, name: Bin Liang, name: Shuo Wang
Published: 2025-03-05 13:57:37

Robotic grasping in scenes with transparent and specular objects presents
great challenges for methods relying on accurate depth information. In this
paper, we introduce NeuGrasp, a neural surface reconstruction method that
leverages background priors for material-agnostic grasp detection. NeuGrasp
integrates transformers and global prior volumes to aggregate multi-view
features with spatial encoding, enabling robust surface reconstruction in
narrow and sparse viewing conditions. By focusing on foreground objects through
residual feature enhancement and refining spatial perception with an
occupancy-prior volume, NeuGrasp excels in handling objects with transparent
and specular surfaces. Extensive experiments in both simulated and real-world
scenarios show that NeuGrasp outperforms state-of-the-art methods in grasping
while maintaining comparable reconstruction quality. More details are available
at https://neugrasp.github.io/.
FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation
  for Feature Implementation
Authors: name: Wei Li, name: Xin Zhang, name: Zhongxin Guo, name: Shaoguang Mao, name: Wen Luo, name: Guangyue Peng, name: Yangyu Huang, name: Houfeng Wang, name: Scarlett Li
Published: 2025-03-09 16:11:57

Implementing new features in repository-level codebases is a crucial
application of code generation models. However, current benchmarks lack a
dedicated evaluation framework for this capability. To fill this gap, we
introduce FEA-Bench, a benchmark designed to assess the ability of large
language models (LLMs) to perform incremental development within code
repositories. We collect pull requests from 83 GitHub repositories and use
rule-based and intent-based filtering to construct task instances focused on
new feature development. Each task instance containing code changes is paired
with relevant unit test files to ensure that the solution can be verified. The
feature implementation requires LLMs to simultaneously possess code completion
capabilities for new components and code editing abilities for other relevant
parts in the code repository, providing a more comprehensive evaluation method
of LLMs' automated software engineering capabilities. Experimental results show
that LLMs perform significantly worse in the FEA-Bench, highlighting
considerable challenges in such repository-level incremental code development.
Words or Vision: Do Vision-Language Models Have Blind Faith in Text?
Authors: name: Ailin Deng, name: Tri Cao, name: Zhirui Chen, name: Bryan Hooi
Published: 2025-03-04 02:21:07

Vision-Language Models (VLMs) excel in integrating visual and textual
information for vision-centric tasks, but their handling of inconsistencies
between modalities is underexplored. We investigate VLMs' modality preferences
when faced with visual data and varied textual inputs in vision-centered
settings. By introducing textual variations to four vision-centric tasks and
evaluating ten Vision-Language Models (VLMs), we discover a ``blind faith
in text'' phenomenon: VLMs disproportionately trust textual data over visual
data when inconsistencies arise, leading to significant performance drops under
corrupted text and raising safety concerns. We analyze factors influencing this
text bias, including instruction prompts, language model size, text relevance,
token order, and the interplay between visual and textual certainty. While
certain factors, such as scaling up the language model size, slightly mitigate
text bias, others like token order can exacerbate it due to positional biases
inherited from language models. To address this issue, we explore supervised
fine-tuning with text augmentation and demonstrate its effectiveness in
reducing text bias. Additionally, we provide a theoretical analysis suggesting
that the blind faith in text phenomenon may stem from an imbalance of pure text
and multi-modal data during training. Our findings highlight the need for
balanced training and careful consideration of modality interactions in VLMs to
enhance their robustness and reliability in handling multi-modal data
inconsistencies.
SafeArena: Evaluating the Safety of Autonomous Web Agents
Authors: name: Ada Defne Tur, name: Nicholas Meade, name: Xing Han Lù, name: Alejandra Zambrano, name: Arkil Patel, name: Esin Durmus, name: Spandana Gella, name: Karolina Stańczak, name: Siva Reddy
Published: 2025-03-06 20:43:14

LLM-based agents are becoming increasingly proficient at solving web-based
tasks. With this capability comes a greater risk of misuse for malicious
purposes, such as posting misinformation in an online forum or selling illicit
substances on a website. To evaluate these risks, we propose SafeArena, the
first benchmark to focus on the deliberate misuse of web agents. SafeArena
comprises 250 safe and 250 harmful tasks across four websites. We classify the
harmful tasks into five harm categories -- misinformation, illegal activity,
harassment, cybercrime, and social bias, designed to assess realistic misuses
of web agents. We evaluate leading LLM-based web agents, including GPT-4o,
Claude-3.5 Sonnet, Qwen-2-VL 72B, and Llama-3.2 90B, on our benchmark. To
systematically assess their susceptibility to harmful tasks, we introduce the
Agent Risk Assessment framework that categorizes agent behavior across four
risk levels. We find agents are surprisingly compliant with malicious requests,
with GPT-4o and Qwen-2 completing 34.7% and 27.3% of harmful requests,
respectively. Our findings highlight the urgent need for safety alignment
procedures for web agents. Our benchmark is available here:
https://safearena.github.io