Orionnnn/hub-stats huggingface dataset:
Animate Anyone 2: High-Fidelity Character Image Animation with
  Environment Affordance
Authors: name: Li Hu, name: Guangyuan Wang, name: Zhen Shen, name: Xin Gao, name: Dechao Meng, name: Lian Zhuo, name: Peng Zhang, name: Bang Zhang, name: Liefeng Bo
Published: 2025-02-10 04:20:11

Recent character image animation methods based on diffusion models, such as
Animate Anyone, have made significant progress in generating consistent and
generalizable character animations. However, these approaches fail to produce
reasonable associations between characters and their environments. To address
this limitation, we introduce Animate Anyone 2, aiming to animate characters
with environment affordance. Beyond extracting motion signals from source
video, we additionally capture environmental representations as conditional
inputs. The environment is formulated as the region with the exclusion of
characters and our model generates characters to populate these regions while
maintaining coherence with the environmental context. We propose a
shape-agnostic mask strategy that more effectively characterizes the
relationship between character and environment. Furthermore, to enhance the
fidelity of object interactions, we leverage an object guider to extract
features of interacting objects and employ spatial blending for feature
injection. We also introduce a pose modulation strategy that enables the model
to handle more diverse motion patterns. Experimental results demonstrate the
superior performance of the proposed method.
BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large
  Language Models
Authors: name: Xu Huang, name: Wenhao Zhu, name: Hanxu Hu, name: Conghui He, name: Lei Li, name: Shujian Huang, name: Fei Yuan
Published: 2025-02-11 08:17:19

Previous multilingual benchmarks focus primarily on simple understanding
tasks, but for large language models(LLMs), we emphasize proficiency in
instruction following, reasoning, long context understanding, code generation,
and so on. However, measuring these advanced capabilities across languages is
underexplored. To address the disparity, we introduce BenchMAX, a multi-way
multilingual evaluation benchmark that allows for fair comparisons of these
important abilities across languages. To maintain high quality, three distinct
native-speaking annotators independently annotate each sample within all tasks
after the data was machine-translated from English into 16 other languages.
Additionally, we present a novel translation challenge stemming from dataset
construction. Extensive experiments on BenchMAX reveal varying effectiveness of
core capabilities across languages, highlighting performance gaps that cannot
be bridged by simply scaling up model size. BenchMAX serves as a comprehensive
multilingual evaluation platform, providing a promising test bed to promote the
development of multilingual language models. The dataset and code are publicly
accessible.
Mediator: Memory-efficient LLM Merging with Less Parameter Conflicts and
  Uncertainty Based Routing
Authors: name: Kunfeng Lai, name: Zhenheng Tang, name: Xinglin Pan, name: Peijie Dong, name: Xiang Liu, name: Haolan Chen, name: Li Shen, name: Bo Li, name: Xiaowen Chu
Published: 2025-02-06 11:26:30

Model merging aggregates Large Language Models (LLMs) finetuned on different
tasks into a stronger one. However, parameter conflicts between models leads to
performance degradation in averaging. While model routing addresses this issue
by selecting individual models during inference, it imposes excessive storage
and compute costs, and fails to leverage the common knowledge from different
models. In this work, we observe that different layers exhibit varying levels
of parameter conflicts. Building on this insight, we average layers with
minimal parameter conflicts and use a novel task-level expert routing for
layers with significant conflicts. To further reduce storage costs, inspired by
task arithmetic sparsity, we decouple multiple fine-tuned experts into a dense
expert and several sparse experts. Considering the out-of-distribution samples,
we select and merge appropriate experts based on the task uncertainty of the
input data. We conduct extensive experiments on both LLaMA and Qwen with
varying parameter scales, and evaluate on real-world reasoning tasks. Results
demonstrate that our method consistently achieves significant performance
improvements while requiring less system cost compared to existing methods.
MetaSC: Test-Time Safety Specification Optimization for Language Models
Authors: name: Víctor Gallego
Published: 2025-02-11 22:06:25

We propose a novel dynamic safety framework that optimizes language model
(LM) safety reasoning at inference time without modifying model weights.
Building on recent advances in self-critique methods, our approach leverages a
meta-critique mechanism that iteratively updates safety prompts-termed
specifications-to drive the critique and revision process adaptively. This
test-time optimization not only improves performance against adversarial
jailbreak requests but also in diverse general safety-related tasks, such as
avoiding moral harm or pursuing honest responses. Our empirical evaluations
across several language models demonstrate that dynamically optimized safety
prompts yield significantly higher safety scores compared to fixed system
prompts and static self-critique defenses. Code to be released at
https://github.com/vicgalle/meta-self-critique.git .
WorldGUI: Dynamic Testing for Comprehensive Desktop GUI Automation
Authors: name: Henry Hengyuan Zhao, name: Difei Gao, name: Mike Zheng Shou
Published: 2025-02-12 01:06:10

Current GUI agents have achieved outstanding performance in GUI element
grounding. However, planning remains highly challenging, especially due to
sensitivity to the initial state of the environment. Specifically, slight
differences in the initial state-such as the target software not being open or
the interface not being in its default state-often lead to planning errors.
This issue is widespread in real user scenarios, but existing benchmarks fail
to evaluate it. In this paper, we present WorldGUI, a novel GUI benchmark that
designs GUI tasks with various initial states to simulate real computer-user
interactions. The benchmark spans a wide range of tasks across 10 popular
software applications, including PowerPoint, VSCode, and Adobe Acrobat. In
addition, to address the challenges of dynamic GUI automation tasks, we propose
GUI-Thinker, a holistic framework, leveraging a critique mechanism, that
effectively manages the unpredictability and complexity of GUI interactions.
Experimental results demonstrate that GUI-Thinker significantly outperforms
Claude-3.5 (Computer Use) by 14.9% in success rate on WorldGUI tasks. This
improvement underscores the effectiveness of our critical-thinking-based
framework in enhancing GUI automation.
Towards Trustworthy Retrieval Augmented Generation for Large Language
  Models: A Survey
Authors: name: Bo Ni, name: Zheyuan Liu, name: Leyao Wang, name: Yongjia Lei, name: Yuying Zhao, name: Xueqi Cheng, name: Qingkai Zeng, name: Luna Dong, name: Yinglong Xia, name: Krishnaram Kenthapadi, name: Ryan Rossi, name: Franck Dernoncourt, name: Md Mehrab Tanjim, name: Nesreen Ahmed, name: Xiaorui Liu, name: Wenqi Fan, name: Erik Blasch, name: Yu Wang, name: Meng Jiang, name: Tyler Derr
Published: 2025-02-08 06:50:47

Retrieval-Augmented Generation (RAG) is an advanced technique designed to
address the challenges of Artificial Intelligence-Generated Content (AIGC). By
integrating context retrieval into content generation, RAG provides reliable
and up-to-date external knowledge, reduces hallucinations, and ensures relevant
context across a wide range of tasks. However, despite RAG's success and
potential, recent studies have shown that the RAG paradigm also introduces new
risks, including robustness issues, privacy concerns, adversarial attacks, and
accountability issues. Addressing these risks is critical for future
applications of RAG systems, as they directly impact their trustworthiness.
Although various methods have been developed to improve the trustworthiness of
RAG methods, there is a lack of a unified perspective and framework for
research in this topic. Thus, in this paper, we aim to address this gap by
providing a comprehensive roadmap for developing trustworthy RAG systems. We
place our discussion around five key perspectives: reliability, privacy,
safety, fairness, explainability, and accountability. For each perspective, we
present a general framework and taxonomy, offering a structured approach to
understanding the current challenges, evaluating existing solutions, and
identifying promising future research directions. To encourage broader adoption
and innovation, we also highlight the downstream applications where trustworthy
RAG systems have a significant impact.
NoLiMa: Long-Context Evaluation Beyond Literal Matching
Authors: name: Ali Modarressi, name: Hanieh Deilamsalehy, name: Franck Dernoncourt, name: Trung Bui, name: Ryan A. Rossi, name: Seunghyun Yoon, name: Hinrich Schütze
Published: 2025-02-07 18:49:46

Recent large language models (LLMs) support long contexts ranging from 128K
to 1M tokens. A popular method for evaluating these capabilities is the
needle-in-a-haystack (NIAH) test, which involves retrieving a "needle"
(relevant information) from a "haystack" (long irrelevant context). Extensions
of this approach include increasing distractors, fact chaining, and in-context
reasoning. However, in these benchmarks, models can exploit existing literal
matches between the needle and haystack to simplify the task. To address this,
we introduce NoLiMa, a benchmark extending NIAH with a carefully designed
needle set, where questions and needles have minimal lexical overlap, requiring
models to infer latent associations to locate the needle within the haystack.
We evaluate 12 popular LLMs that claim to support contexts of at least 128K
tokens. While they perform well in short contexts (<1K), performance degrades
significantly as context length increases. At 32K, for instance, 10 models drop
below 50% of their strong short-length baselines. Even GPT-4o, one of the
top-performing exceptions, experiences a reduction from an almost-perfect
baseline of 99.3% to 69.7%. Our analysis suggests these declines stem from the
increased difficulty the attention mechanism faces in longer contexts when
literal matches are absent, making it harder to retrieve relevant information.
TextAtlas5M: A Large-scale Dataset for Dense Text Image Generation
Authors: name: Alex Jinpeng Wang, name: Dongxing Mao, name: Jiawei Zhang, name: Weiming Han, name: Zhuobai Dong, name: Linjie Li, name: Yiqi Lin, name: Zhengyuan Yang, name: Libo Qin, name: Fuwei Zhang, name: Lijuan Wang, name: Min Li
Published: 2025-02-11 18:59:19

Text-conditioned image generation has gained significant attention in recent
years and are processing increasingly longer and comprehensive text prompt. In
everyday life, dense and intricate text appears in contexts like
advertisements, infographics, and signage, where the integration of both text
and visuals is essential for conveying complex information. However, despite
these advances, the generation of images containing long-form text remains a
persistent challenge, largely due to the limitations of existing datasets,
which often focus on shorter and simpler text. To address this gap, we
introduce TextAtlas5M, a novel dataset specifically designed to evaluate
long-text rendering in text-conditioned image generation. Our dataset consists
of 5 million long-text generated and collected images across diverse data
types, enabling comprehensive evaluation of large-scale generative models on
long-text image generation. We further curate 3000 human-improved test set
TextAtlasEval across 3 data domains, establishing one of the most extensive
benchmarks for text-conditioned generation. Evaluations suggest that the
TextAtlasEval benchmarks present significant challenges even for the most
advanced proprietary models (e.g. GPT4o with DallE-3), while their open-source
counterparts show an even larger performance gap. These evidences position
TextAtlas5M as a valuable dataset for training and evaluating future-generation
text-conditioned image generation models.
Light-A-Video: Training-free Video Relighting via Progressive Light
  Fusion
Authors: name: Yujie Zhou, name: Jiazi Bu, name: Pengyang Ling, name: Pan Zhang, name: Tong Wu, name: Qidong Huang, name: Jinsong Li, name: Xiaoyi Dong, name: Yuhang Zang, name: Yuhang Cao, name: Anyi Rao, name: Jiaqi Wang, name: Li Niu
Published: 2025-02-12 17:24:19

Recent advancements in image relighting models, driven by large-scale
datasets and pre-trained diffusion models, have enabled the imposition of
consistent lighting. However, video relighting still lags, primarily due to the
excessive training costs and the scarcity of diverse, high-quality video
relighting datasets. A simple application of image relighting models on a
frame-by-frame basis leads to several issues: lighting source inconsistency and
relighted appearance inconsistency, resulting in flickers in the generated
videos. In this work, we propose Light-A-Video, a training-free approach to
achieve temporally smooth video relighting. Adapted from image relighting
models, Light-A-Video introduces two key techniques to enhance lighting
consistency. First, we design a Consistent Light Attention (CLA) module, which
enhances cross-frame interactions within the self-attention layers to stabilize
the generation of the background lighting source. Second, leveraging the
physical principle of light transport independence, we apply linear blending
between the source video's appearance and the relighted appearance, using a
Progressive Light Fusion (PLF) strategy to ensure smooth temporal transitions
in illumination. Experiments show that Light-A-Video improves the temporal
consistency of relighted video while maintaining the image quality, ensuring
coherent lighting transitions across frames. Project page:
https://bujiazi.github.io/light-a-video.github.io/.
LASP-2: Rethinking Sequence Parallelism for Linear Attention and Its
  Hybrid
Authors: name: Weigao Sun, name: Disen Lan, name: Yiran Zhong, name: Xiaoye Qu, name: Yu Cheng
Published: 2025-02-11 14:01:39

Linear sequence modeling approaches, such as linear attention, provide
advantages like linear-time training and constant-memory inference over
sequence lengths. However, existing sequence parallelism (SP) methods are
either not optimized for the right-product-first feature of linear attention or
use a ring-style communication strategy, which results in lower computation
parallelism, limits their scalability for longer sequences in distributed
systems. In this paper, we introduce LASP-2, a new SP method to enhance both
communication and computation parallelism when training linear attention
transformer models with very-long input sequences. Compared to previous work
LASP, LASP-2 rethinks the minimal communication requirement for SP on linear
attention layers, reorganizes the whole communication-computation workflow of
LASP. In this way, only one single AllGather collective communication is needed
on intermediate memory states, whose sizes are independent of the sequence
length, leading to significant improvements of both communication and
computation parallelism, as well as their overlap. Additionally, we extend
LASP-2 to LASP-2H by applying similar communication redesign to standard
attention modules, offering an efficient SP solution for hybrid models that
blend linear and standard attention layers. Our evaluation on a Linear-Llama3
model, a variant of Llama3 with linear attention replacing standard attention,
demonstrates the effectiveness of LASP-2 and LASP-2H. Specifically, LASP-2
achieves training speed improvements of 15.2% over LASP and 36.6% over Ring
Attention, with a sequence length of 2048K across 64 GPUs. The Code is released
as a part of: https://github.com/OpenSparseLLMs/Linear-MoE.
LLM Pretraining with Continuous Concepts
Authors: name: Jihoon Tack, name: Jack Lanchantin, name: Jane Yu, name: Andrew Cohen, name: Ilia Kulikov, name: Janice Lan, name: Shibo Hao, name: Yuandong Tian, name: Jason Weston, name: Xian Li
Published: 2025-02-12 16:00:11

Next token prediction has been the standard training objective used in large
language model pretraining. Representations are learned as a result of
optimizing for token-level perplexity. We propose Continuous Concept Mixing
(CoCoMix), a novel pretraining framework that combines discrete next token
prediction with continuous concepts. Specifically, CoCoMix predicts continuous
concepts learned from a pretrained sparse autoencoder and mixes them into the
model's hidden state by interleaving with token hidden representations. Through
experiments on multiple benchmarks, including language modeling and downstream
reasoning tasks, we show that CoCoMix is more sample efficient and consistently
outperforms standard next token prediction, knowledge distillation and
inserting pause tokens. We find that combining both concept learning and
interleaving in an end-to-end framework is critical to performance gains.
Furthermore, CoCoMix enhances interpretability and steerability by allowing
direct inspection and modification of the predicted concept, offering a
transparent way to guide the model's internal reasoning process.
Distillation Scaling Laws
Authors: name: Dan Busbridge, name: Amitis Shidani, name: Floris Weers, name: Jason Ramapuram, name: Etai Littwin, name: Russ Webb
Published: 2025-02-12 17:52:47

We provide a distillation scaling law that estimates distilled model
performance based on a compute budget and its allocation between the student
and teacher. Our findings reduce the risks associated with using distillation
at scale; compute allocation for both the teacher and student models can now be
done to maximize student performance. We provide compute optimal distillation
recipes for when 1) a teacher exists, or 2) a teacher needs training. If many
students are to be distilled, or a teacher already exists, distillation
outperforms supervised pretraining until a compute level which grows
predictably with student size. If one student is to be distilled and a teacher
also needs training, supervised learning should be done instead. Additionally,
we provide insights across our large scale study of distillation, which
increase our understanding of distillation and inform experimental design.
SARChat-Bench-2M: A Multi-Task Vision-Language Benchmark for SAR Image
  Interpretation
Authors: name: Zhiming Ma, name: Xiayang Xiao, name: Sihao Dong, name: Peidong Wang, name: HaiPeng Wang, name: Qingyun Pan
Published: 2025-02-12 07:19:36

In the field of synthetic aperture radar (SAR) remote sensing image
interpretation, although Vision language models (VLMs) have made remarkable
progress in natural language processing and image understanding, their
applications remain limited in professional domains due to insufficient domain
expertise. This paper innovatively proposes the first large-scale multimodal
dialogue dataset for SAR images, named SARChat-2M, which contains approximately
2 million high-quality image-text pairs, encompasses diverse scenarios with
detailed target annotations. This dataset not only supports several key tasks
such as visual understanding and object detection tasks, but also has unique
innovative aspects: this study develop a visual-language dataset and benchmark
for the SAR domain, enabling and evaluating VLMs' capabilities in SAR image
interpretation, which provides a paradigmatic framework for constructing
multimodal datasets across various remote sensing vertical domains. Through
experiments on 16 mainstream VLMs, the effectiveness of the dataset has been
fully verified, and the first multi-task dialogue benchmark in the SAR field
has been successfully established. The project will be released at
https://github.com/JimmyMa99/SARChat, aiming to promote the in-depth
development and wide application of SAR visual language models.
CineMaster: A 3D-Aware and Controllable Framework for Cinematic
  Text-to-Video Generation
Authors: name: Qinghe Wang, name: Yawen Luo, name: Xiaoyu Shi, name: Xu Jia, name: Huchuan Lu, name: Tianfan Xue, name: Xintao Wang, name: Pengfei Wan, name: Di Zhang, name: Kun Gai
Published: 2025-02-12 18:55:36

In this work, we present CineMaster, a novel framework for 3D-aware and
controllable text-to-video generation. Our goal is to empower users with
comparable controllability as professional film directors: precise placement of
objects within the scene, flexible manipulation of both objects and camera in
3D space, and intuitive layout control over the rendered frames. To achieve
this, CineMaster operates in two stages. In the first stage, we design an
interactive workflow that allows users to intuitively construct 3D-aware
conditional signals by positioning object bounding boxes and defining camera
movements within the 3D space. In the second stage, these control
signals--comprising rendered depth maps, camera trajectories and object class
labels--serve as the guidance for a text-to-video diffusion model, ensuring to
generate the user-intended video content. Furthermore, to overcome the scarcity
of in-the-wild datasets with 3D object motion and camera pose annotations, we
carefully establish an automated data annotation pipeline that extracts 3D
bounding boxes and camera trajectories from large-scale video data. Extensive
qualitative and quantitative experiments demonstrate that CineMaster
significantly outperforms existing methods and implements prominent 3D-aware
text-to-video generation. Project page: https://cinemaster-dev.github.io/.
Next Block Prediction: Video Generation via Semi-Autoregressive Modeling
Authors: name: Shuhuai Ren, name: Shuming Ma, name: Xu Sun, name: Furu Wei
Published: 2025-02-11 17:57:53

Next-Token Prediction (NTP) is a de facto approach for autoregressive (AR)
video generation, but it suffers from suboptimal unidirectional dependencies
and slow inference speed. In this work, we propose a semi-autoregressive
(semi-AR) framework, called Next-Block Prediction (NBP), for video generation.
By uniformly decomposing video content into equal-sized blocks (e.g., rows or
frames), we shift the generation unit from individual tokens to blocks,
allowing each token in the current block to simultaneously predict the
corresponding token in the next block. Unlike traditional AR modeling, our
framework employs bidirectional attention within each block, enabling tokens to
capture more robust spatial dependencies. By predicting multiple tokens in
parallel, NBP models significantly reduce the number of generation steps,
leading to faster and more efficient inference. Our model achieves FVD scores
of 103.3 on UCF101 and 25.5 on K600, outperforming the vanilla NTP model by an
average of 4.4. Furthermore, thanks to the reduced number of inference steps,
the NBP model generates 8.89 frames (128x128 resolution) per second, achieving
an 11x speedup. We also explored model scales ranging from 700M to 3B
parameters, observing significant improvements in generation quality, with FVD
scores dropping from 103.3 to 55.3 on UCF101 and from 25.5 to 19.5 on K600,
demonstrating the scalability of our approach.