Orionnnn/hub-stats huggingface dataset:
DreamActor-M1: Holistic, Expressive and Robust Human Image Animation
  with Hybrid Guidance
Authors: name: Yuxuan Luo, name: Zhengkun Rong, name: Lizhen Wang, name: Longhao Zhang, name: Tianshu Hu, name: Yongming Zhu
Published: 2025-04-02 13:30:32

While recent image-based human animation methods achieve realistic body and
facial motion synthesis, critical gaps remain in fine-grained holistic
controllability, multi-scale adaptability, and long-term temporal coherence,
which leads to their lower expressiveness and robustness. We propose a
diffusion transformer (DiT) based framework, DreamActor-M1, with hybrid
guidance to overcome these limitations. For motion guidance, our hybrid control
signals that integrate implicit facial representations, 3D head spheres, and 3D
body skeletons achieve robust control of facial expressions and body movements,
while producing expressive and identity-preserving animations. For scale
adaptation, to handle various body poses and image scales ranging from
portraits to full-body views, we employ a progressive training strategy using
data with varying resolutions and scales. For appearance guidance, we integrate
motion patterns from sequential frames with complementary visual references,
ensuring long-term temporal coherence for unseen regions during complex
movements. Experiments demonstrate that our method outperforms the
state-of-the-art works, delivering expressive results for portraits,
upper-body, and full-body generation with robust long-term consistency. Project
Page: https://grisoon.github.io/DreamActor-M1/.
AnimeGamer: Infinite Anime Life Simulation with Next Game State
  Prediction
Authors: name: Junhao Cheng, name: Yuying Ge, name: Yixiao Ge, name: Jing Liao, name: Ying Shan
Published: 2025-04-01 17:57:18

Recent advancements in image and video synthesis have opened up new promise
in generative games. One particularly intriguing application is transforming
characters from anime films into interactive, playable entities. This allows
players to immerse themselves in the dynamic anime world as their favorite
characters for life simulation through language instructions. Such games are
defined as infinite game since they eliminate predetermined boundaries and
fixed gameplay rules, where players can interact with the game world through
open-ended language and experience ever-evolving storylines and environments.
Recently, a pioneering approach for infinite anime life simulation employs
large language models (LLMs) to translate multi-turn text dialogues into
language instructions for image generation. However, it neglects historical
visual context, leading to inconsistent gameplay. Furthermore, it only
generates static images, failing to incorporate the dynamics necessary for an
engaging gaming experience. In this work, we propose AnimeGamer, which is built
upon Multimodal Large Language Models (MLLMs) to generate each game state,
including dynamic animation shots that depict character movements and updates
to character states, as illustrated in Figure 1. We introduce novel
action-aware multimodal representations to represent animation shots, which can
be decoded into high-quality video clips using a video diffusion model. By
taking historical animation shot representations as context and predicting
subsequent representations, AnimeGamer can generate games with contextual
consistency and satisfactory dynamics. Extensive evaluations using both
automated metrics and human evaluations demonstrate that AnimeGamer outperforms
existing methods in various aspects of the gaming experience. Codes and
checkpoints are available at https://github.com/TencentARC/AnimeGamer.
Articulated Kinematics Distillation from Video Diffusion Models
Authors: name: Xuan Li, name: Qianli Ma, name: Tsung-Yi Lin, name: Yongxin Chen, name: Chenfanfu Jiang, name: Ming-Yu Liu, name: Donglai Xiang
Published: 2025-04-01 21:37:57

We present Articulated Kinematics Distillation (AKD), a framework for
generating high-fidelity character animations by merging the strengths of
skeleton-based animation and modern generative models. AKD uses a
skeleton-based representation for rigged 3D assets, drastically reducing the
Degrees of Freedom (DoFs) by focusing on joint-level control, which allows for
efficient, consistent motion synthesis. Through Score Distillation Sampling
(SDS) with pre-trained video diffusion models, AKD distills complex,
articulated motions while maintaining structural integrity, overcoming
challenges faced by 4D neural deformation fields in preserving shape
consistency. This approach is naturally compatible with physics-based
simulation, ensuring physically plausible interactions. Experiments show that
AKD achieves superior 3D consistency and motion quality compared with existing
works on text-to-4D generation. Project page:
https://research.nvidia.com/labs/dir/akd/
PaperBench: Evaluating AI's Ability to Replicate AI Research
Authors: name: Giulio Starace, name: Oliver Jaffe, name: Dane Sherburn, name: James Aung, name: Jun Shern Chan, name: Leon Maksin, name: Rachel Dias, name: Evan Mays, name: Benjamin Kinsella, name: Wyatt Thompson, name: Johannes Heidecke, name: Amelia Glaese, name: Tejal Patwardhan
Published: 2025-04-02 15:55:24

We introduce PaperBench, a benchmark evaluating the ability of AI agents to
replicate state-of-the-art AI research. Agents must replicate 20 ICML 2024
Spotlight and Oral papers from scratch, including understanding paper
contributions, developing a codebase, and successfully executing experiments.
For objective evaluation, we develop rubrics that hierarchically decompose each
replication task into smaller sub-tasks with clear grading criteria. In total,
PaperBench contains 8,316 individually gradable tasks. Rubrics are co-developed
with the author(s) of each ICML paper for accuracy and realism. To enable
scalable evaluation, we also develop an LLM-based judge to automatically grade
replication attempts against rubrics, and assess our judge's performance by
creating a separate benchmark for judges. We evaluate several frontier models
on PaperBench, finding that the best-performing tested agent, Claude 3.5 Sonnet
(New) with open-source scaffolding, achieves an average replication score of
21.0\%. Finally, we recruit top ML PhDs to attempt a subset of PaperBench,
finding that models do not yet outperform the human baseline. We
https://github.com/openai/preparedness{open-source our code} to
facilitate future research in understanding the AI engineering capabilities of
AI agents.
Improved Visual-Spatial Reasoning via R1-Zero-Like Training
Authors: name: Zhenyi Liao, name: Qingsong Xie, name: Yanhao Zhang, name: Zijian Kong, name: Haonan Lu, name: Zhenyu Yang, name: Zhijie Deng
Published: 2025-04-01 15:11:11

Increasing attention has been placed on improving the reasoning capacities of
multi-modal large language models (MLLMs). As the cornerstone for AI agents
that function in the physical realm, video-based visual-spatial intelligence
(VSI) emerges as one of the most pivotal reasoning capabilities of MLLMs. This
work conducts a first, in-depth study on improving the visual-spatial reasoning
of MLLMs via R1-Zero-like training. Technically, we first identify that the
visual-spatial reasoning capacities of small- to medium-sized Qwen2-VL models
cannot be activated via Chain of Thought (CoT) prompts. We then incorporate
GRPO training for improved visual-spatial reasoning, using the carefully
curated VSI-100k dataset, following DeepSeek-R1-Zero. During the investigation,
we identify the necessity to keep the KL penalty (even with a small value) in
GRPO. With just 120 GPU hours, our vsGRPO-2B model, fine-tuned from
Qwen2-VL-2B, can outperform the base model by 12.1% and surpass GPT-4o.
Moreover, our vsGRPO-7B model, fine-tuned from Qwen2-VL-7B, achieves
performance comparable to that of the best open-source model
LLaVA-NeXT-Video-72B. Additionally, we compare vsGRPO to supervised fine-tuning
and direct preference optimization baselines and observe strong performance
superiority. The code and dataset will be available soon.
LSNet: See Large, Focus Small
Authors: name: Ao Wang, name: Hui Chen, name: Zijia Lin, name: Jungong Han, name: Guiguang Ding
Published: 2025-03-29 16:00:54

Vision network designs, including Convolutional Neural Networks and Vision
Transformers, have significantly advanced the field of computer vision. Yet,
their complex computations pose challenges for practical deployments,
particularly in real-time applications. To tackle this issue, researchers have
explored various lightweight and efficient network designs. However, existing
lightweight models predominantly leverage self-attention mechanisms and
convolutions for token mixing. This dependence brings limitations in
effectiveness and efficiency in the perception and aggregation processes of
lightweight networks, hindering the balance between performance and efficiency
under limited computational budgets. In this paper, we draw inspiration from
the dynamic heteroscale vision ability inherent in the efficient human vision
system and propose a ``See Large, Focus Small'' strategy for lightweight vision
network design. We introduce LS (Large-Small) convolution,
which combines large-kernel perception and small-kernel aggregation. It can
efficiently capture a wide range of perceptual information and achieve precise
feature aggregation for dynamic and complex visual representations, thus
enabling proficient processing of visual information. Based on LS convolution,
we present LSNet, a new family of lightweight models. Extensive experiments
demonstrate that LSNet achieves superior performance and efficiency over
existing lightweight networks in various vision tasks. Codes and models are
available at https://github.com/jameslahm/lsnet.
MergeVQ: A Unified Framework for Visual Generation and Representation
  with Disentangled Token Merging and Quantization
Authors: name: Siyuan Li, name: Luyuan Zhang, name: Zedong Wang, name: Juanxi Tian, name: Cheng Tan, name: Zicheng Liu, name: Chang Yu, name: Qingsong Xie, name: Haonan Lu, name: Haoqian Wang, name: Zhen Lei
Published: 2025-04-01 17:39:19

Masked Image Modeling (MIM) with Vector Quantization (VQ) has achieved great
success in both self-supervised pre-training and image generation. However,
most existing methods struggle to address the trade-off in shared latent space
for generation quality vs. representation learning and efficiency. To push the
limits of this paradigm, we propose MergeVQ, which incorporates token merging
techniques into VQ-based generative models to bridge the gap between image
generation and visual representation learning in a unified architecture. During
pre-training, MergeVQ decouples top-k semantics from latent space with the
token merge module after self-attention blocks in the encoder for subsequent
Look-up Free Quantization (LFQ) and global alignment and recovers their
fine-grained details through cross-attention in the decoder for reconstruction.
As for the second-stage generation, we introduce MergeAR, which performs KV
Cache compression for efficient raster-order prediction. Extensive experiments
on ImageNet verify that MergeVQ as an AR generative model achieves competitive
performance in both visual representation learning and image generation tasks
while maintaining favorable token efficiency and inference speed. The code and
model will be available at https://apexgen-x.github.io/MergeVQ.
ScholarCopilot: Training Large Language Models for Academic Writing with
  Accurate Citations
Authors: name: Yubo Wang, name: Xueguang Ma, name: Ping Nie, name: Huaye Zeng, name: Zhiheng Lyu, name: Yuxuan Zhang, name: Benjamin Schneider, name: Yi Lu, name: Xiang Yue, name: Wenhu Chen
Published: 2025-04-01 14:12:14

Academic writing requires both coherent text generation and precise citation
of relevant literature. Although recent Retrieval-Augmented Generation (RAG)
systems have significantly improved factual accuracy in general-purpose text
generation, their capacity to adequately support professional academic writing
remains limited. In this work, we introduce ScholarCopilot, a unified framework
designed to enhance existing large language models for generating professional
academic articles with accurate and contextually relevant citations.
ScholarCopilot dynamically determines when to retrieve scholarly references by
generating a retrieval token [RET], and then utilizes its representation to
look up relevant citations from a database. The retrieved references are fed
into the model to augment the generation process. We jointly optimize both the
generation and citation tasks within a single framework to increase efficiency.
Trained on 500K papers from arXiv, our model achieves a top-1 retrieval
accuracy of 40.1% on our evaluation dataset, outperforming baselines such as
E5-Mistral-7B-Instruct (15.0%) and BM25 (9.8%). On a dataset of 1,000 academic
writing samples, ScholarCopilot scores 16.2/25 in generation quality (measured
across relevance, coherence, academic rigor, completeness, and innovation),
surpassing models with 10x more parameters such as Qwen-2.5-72B-Instruct
(15.8/25). Human studies also confirm ScholarCopilot's superior performance in
citation recall, writing efficiency, and overall user experience, confirming
the effectiveness of our approach.
Safeguarding Vision-Language Models: Mitigating Vulnerabilities to
  Gaussian Noise in Perturbation-based Attacks
Authors: name: Jiawei Wang, name: Yushen Zuo, name: Yuanjun Chai, name: Zhendong Liu, name: Yichen Fu, name: Yichun Feng, name: Kin-man Lam
Published: 2025-04-02 02:35:19

Vision-Language Models (VLMs) extend the capabilities of Large Language
Models (LLMs) by incorporating visual information, yet they remain vulnerable
to jailbreak attacks, especially when processing noisy or corrupted images.
Although existing VLMs adopt security measures during training to mitigate such
attacks, vulnerabilities associated with noise-augmented visual inputs are
overlooked. In this work, we identify that missing noise-augmented training
causes critical security gaps: many VLMs are susceptible to even simple
perturbations such as Gaussian noise. To address this challenge, we propose
Robust-VLGuard, a multimodal safety dataset with aligned / misaligned
image-text pairs, combined with noise-augmented fine-tuning that reduces attack
success rates while preserving functionality of VLM. For stronger
optimization-based visual perturbation attacks, we propose DiffPure-VLM,
leveraging diffusion models to convert adversarial perturbations into
Gaussian-like noise, which can be defended by VLMs with noise-augmented safety
fine-tuning. Experimental results demonstrate that the distribution-shifting
property of diffusion model aligns well with our fine-tuned VLMs, significantly
mitigating adversarial perturbations across varying intensities. The dataset
and code are available at https://github.com/JarvisUSTC/DiffPure-RobustVLM.
YourBench: Easy Custom Evaluation Sets for Everyone
Authors: name: Sumuk Shashidhar, name: Clémentine Fourrier, name: Alina Lozovskia, name: Thomas Wolf, name: Gokhan Tur, name: Dilek Hakkani-Tür
Published: 2025-04-02 15:40:24

Evaluating large language models (LLMs) effectively remains a critical
bottleneck, as traditional static benchmarks suffer from saturation and
contamination, while human evaluations are costly and slow. This hinders timely
or domain-specific assessment, crucial for real-world applications. We
introduce YourBench, a novel, open-source framework that addresses these
limitations by enabling dynamic, automated generation of reliable, up-to-date,
and domain-tailored benchmarks cheaply and without manual annotation, directly
from user-provided documents. We demonstrate its efficacy by replicating 7
diverse MMLU subsets using minimal source text, achieving this for under 15 USD
in total inference costs while perfectly preserving the relative model
performance rankings (Spearman Rho = 1) observed on the original benchmark. To
ensure that YourBench generates data grounded in provided input instead of
relying on posterior parametric knowledge in models, we also introduce
Tempora-0325, a novel dataset of over 7K diverse documents, published
exclusively after March 2025. Our comprehensive analysis spans 26 SoTA models
from 7 major families across varying scales (3-671B parameters) to validate the
quality of generated evaluations through rigorous algorithmic checks (e.g.,
citation grounding) and human assessments. We release the YourBench library,
the Tempora-0325 dataset, 150k+ question answer pairs based on Tempora and all
evaluation and inference traces to facilitate reproducible research and empower
the community to generate bespoke benchmarks on demand, fostering more relevant
and trustworthy LLM evaluation.
Landscape of Thoughts: Visualizing the Reasoning Process of Large
  Language Models
Authors: name: Zhanke Zhou, name: Zhaocheng Zhu, name: Xuan Li, name: Mikhail Galkin, name: Xiao Feng, name: Sanmi Koyejo, name: Jian Tang, name: Bo Han
Published: 2025-03-28 06:09:51

Numerous applications of large language models (LLMs) rely on their ability
to perform step-by-step reasoning. However, the reasoning behavior of LLMs
remains poorly understood, posing challenges to research, development, and
safety. To address this gap, we introduce landscape of thoughts-the first
visualization tool for users to inspect the reasoning paths of chain-of-thought
and its derivatives on any multi-choice dataset. Specifically, we represent the
states in a reasoning path as feature vectors that quantify their distances to
all answer choices. These features are then visualized in two-dimensional plots
using t-SNE. Qualitative and quantitative analysis with the landscape of
thoughts effectively distinguishes between strong and weak models, correct and
incorrect answers, as well as different reasoning tasks. It also uncovers
undesirable reasoning patterns, such as low consistency and high uncertainty.
Additionally, users can adapt our tool to a model that predicts the property
they observe. We showcase this advantage by adapting our tool to a lightweight
verifier that evaluates the correctness of reasoning paths. The code is
publicly available at: https://github.com/tmlr-group/landscape-of-thoughts.
Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards
  for Reasoning-Enhanced Text-to-SQL
Authors: name: Mohammadreza Pourreza, name: Shayan Talaei, name: Ruoxi Sun, name: Xingchen Wan, name: Hailong Li, name: Azalia Mirhoseini, name: Amin Saberi, name: Sercan "O. Arik
Published: 2025-03-29 17:29:30

Text-to-SQL is a challenging task involving multiple reasoning-intensive
subtasks, including natural language understanding, database schema
comprehension, and precise SQL query formulation. Existing approaches often
rely on handcrafted reasoning paths with inductive biases that can limit their
overall effectiveness. Motivated by the recent success of reasoning-enhanced
models such as DeepSeek R1 and OpenAI o1, which effectively leverage
reward-driven self-exploration to enhance reasoning capabilities and
generalization, we propose a novel set of partial rewards tailored specifically
for the Text-to-SQL task. Our reward set includes schema-linking, AI feedback,
n-gram similarity, and syntax check, explicitly designed to address the reward
sparsity issue prevalent in reinforcement learning (RL). Leveraging group
relative policy optimization (GRPO), our approach explicitly encourages large
language models (LLMs) to develop intrinsic reasoning skills necessary for
accurate SQL query generation. With models of different sizes, we demonstrate
that RL-only training with our proposed rewards consistently achieves higher
accuracy and superior generalization compared to supervised fine-tuning (SFT).
Remarkably, our RL-trained 14B-parameter model significantly outperforms larger
proprietary models, e.g. o3-mini by 4% and Gemini-1.5-Pro-002 by 3% on the BIRD
benchmark. These highlight the efficacy of our proposed RL-training framework
with partial rewards for enhancing both accuracy and reasoning capabilities in
Text-to-SQL tasks.
MB-ORES: A Multi-Branch Object Reasoner for Visual Grounding in Remote
  Sensing
Authors: name: Karim Radouane, name: Hanane Azzag, name: Mustapha lebbah
Published: 2025-03-31 15:36:41

We propose a unified framework that integrates object detection (OD) and
visual grounding (VG) for remote sensing (RS) imagery. To support conventional
OD and establish an intuitive prior for VG task, we fine-tune an open-set
object detector using referring expression data, framing it as a partially
supervised OD task. In the first stage, we construct a graph representation of
each image, comprising object queries, class embeddings, and proposal
locations. Then, our task-aware architecture processes this graph to perform
the VG task. The model consists of: (i) a multi-branch network that integrates
spatial, visual, and categorical features to generate task-aware proposals, and
(ii) an object reasoning network that assigns probabilities across proposals,
followed by a soft selection mechanism for final referring object localization.
Our model demonstrates superior performance on the OPT-RSVG and DIOR-RSVG
datasets, achieving significant improvements over state-of-the-art methods
while retaining classical OD capabilities. The code will be available in our
repository: https://github.com/rd20karim/MB-ORES.
OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming
  Video Contexts
Authors: name: Yuxuan Wang, name: Yueqian Wang, name: Bo Chen, name: Tong Wu, name: Dongyan Zhao, name: Zilong Zheng
Published: 2025-03-29 02:46:58

The rapid advancement of multi-modal language models (MLLMs) like GPT-4o has
propelled the development of Omni language models, designed to process and
proactively respond to continuous streams of multi-modal data. Despite their
potential, evaluating their real-world interactive capabilities in streaming
video contexts remains a formidable challenge. In this work, we introduce
OmniMMI, a comprehensive multi-modal interaction benchmark tailored for
OmniLLMs in streaming video contexts. OmniMMI encompasses over 1,121 videos and
2,290 questions, addressing two critical yet underexplored challenges in
existing video benchmarks: streaming video understanding and proactive
reasoning, across six distinct subtasks. Moreover, we propose a novel
framework, Multi-modal Multiplexing Modeling (M4), designed to enable an
inference-efficient streaming model that can see, listen while generating.
Towards Trustworthy GUI Agents: A Survey
Authors: name: Yucheng Shi, name: Wenhao Yu, name: Wenlin Yao, name: Wenhu Chen, name: Ninghao Liu
Published: 2025-03-30 13:26:00

GUI agents, powered by large foundation models, can interact with digital
interfaces, enabling various applications in web automation, mobile navigation,
and software testing. However, their increasing autonomy has raised critical
concerns about their security, privacy, and safety. This survey examines the
trustworthiness of GUI agents in five critical dimensions: security
vulnerabilities, reliability in dynamic environments, transparency and
explainability, ethical considerations, and evaluation methodologies. We also
identify major challenges such as vulnerability to adversarial attacks,
cascading failure modes in sequential decision-making, and a lack of realistic
evaluation benchmarks. These issues not only hinder real-world deployment but
also call for comprehensive mitigation strategies beyond task success. As GUI
agents become more widespread, establishing robust safety standards and
responsible development practices is essential. This survey provides a
foundation for advancing trustworthy GUI agents through systematic
understanding and future research.