Orionnnn/hub-stats huggingface dataset:
Hyperbolic Safety-Aware Vision-Language Models
Authors: name: Tobia Poppi, name: Tejaswi Kasarla, name: Pascal Mettes, name: Lorenzo Baraldi, name: Rita Cucchiara
Published: 2025-03-15 13:18:04

Addressing the retrieval of unsafe content from vision-language models such
as CLIP is an important step towards real-world integration. Current efforts
have relied on unlearning techniques that try to erase the model's knowledge of
unsafe concepts. While effective in reducing unwanted outputs, unlearning
limits the model's capacity to discern between safe and unsafe content. In this
work, we introduce a novel approach that shifts from unlearning to an awareness
paradigm by leveraging the inherent hierarchical properties of the hyperbolic
space. We propose to encode safe and unsafe content as an entailment hierarchy,
where both are placed in different regions of hyperbolic space. Our HySAC,
Hyperbolic Safety-Aware CLIP, employs entailment loss functions to model the
hierarchical and asymmetrical relations between safe and unsafe image-text
pairs. This modelling, ineffective in standard vision-language models due to
their reliance on Euclidean embeddings, endows the model with awareness of
unsafe content, enabling it to serve as both a multimodal unsafe classifier and
a flexible content retriever, with the option to dynamically redirect unsafe
queries toward safer alternatives or retain the original output. Extensive
experiments show that our approach not only enhances safety recognition but
also establishes a more adaptable and interpretable framework for content
moderation in vision-language models. Our source code is available at
https://github.com/aimagelab/HySAC.
Pensez: Less Data, Better Reasoning -- Rethinking French LLM
Authors: name: Huy Hoang Ha
Published: 2025-03-17 19:09:11

Large language models (LLMs) have demonstrated remarkable capabilities in
various natural language processing tasks. However, achieving strong
performance in specialized domains like mathematical reasoning and non-English
languages often requires extensive training on massive datasets. This paper
investigates a contrasting approach: strategic fine-tuning on a small,
high-quality, bilingual (English-French) dataset to enhance both the reasoning
capabilities and French language proficiency of a large language model. Rather
than relying on scale, we explore the hypothesis that targeted data curation
and optimized training can achieve competitive, or even superior, performance.
We demonstrate, through targeted supervised fine-tuning (SFT) on only 2,000
carefully selected samples, significant improvements in mathematical reasoning.
Specifically, Pensez 7B exhibits an increase in accuracy of the base model up
to 20% on the AIME25 and a 12% increase on a French MATH level 5 benchmark.
These results challenge the prevailing assumption that massive datasets are
aprerequisite for strong reasoning performance in LLMs, highlighting the
potential of strategic data curation and optimized fine-tuning for enhancing
both specialized skills and multilingual capabilities. Our findings have
implications for the efficient development of high-performing, multilingual
LLMs, especially in resource-constrained scenarios.
Concat-ID: Towards Universal Identity-Preserving Video Synthesis
Authors: name: Yong Zhong, name: Zhuoyi Yang, name: Jiayan Teng, name: Xiaotao Gu, name: Chongxuan Li
Published: 2025-03-18 11:17:32

We present Concat-ID, a unified framework for identity-preserving video
generation. Concat-ID employs Variational Autoencoders to extract image
features, which are concatenated with video latents along the sequence
dimension, leveraging solely 3D self-attention mechanisms without the need for
additional modules. A novel cross-video pairing strategy and a multi-stage
training regimen are introduced to balance identity consistency and facial
editability while enhancing video naturalness. Extensive experiments
demonstrate Concat-ID's superiority over existing methods in both single and
multi-identity generation, as well as its seamless scalability to multi-subject
scenarios, including virtual try-on and background-controllable generation.
Concat-ID establishes a new benchmark for identity-preserving video synthesis,
providing a versatile and scalable solution for a wide range of applications.
Temporal Consistency for LLM Reasoning Process Error Identification
Authors: name: Jiacheng Guo, name: Yue Wu, name: Jiahao Qiu, name: Kaixuan Huang, name: Xinzhe Juan, name: Ling Yang, name: Mengdi Wang
Published: 2025-03-18 17:58:28

Verification is crucial for effective mathematical reasoning. We present a
new temporal consistency method where verifiers iteratively refine their
judgments based on the previous assessment. Unlike one-round verification or
multi-model debate approaches, our method leverages consistency in a sequence
of self-reflection actions to improve verification accuracy. Empirical
evaluations across diverse mathematical process error identification benchmarks
(Mathcheck, ProcessBench, and PRM800K) show consistent performance improvements
over baseline methods. When applied to the recent DeepSeek R1 distilled models,
our method demonstrates strong performance, enabling 7B/8B distilled models to
outperform all 70B/72B models and GPT-4o on ProcessBench. Notably, the
distilled 14B model with our method achieves performance comparable to
Deepseek-R1. Our codes are available at
https://github.com/jcguo123/Temporal-Consistency
Aligning Multimodal LLM with Human Preference: A Survey
Authors: name: Tao Yu, name: Yi-Fan Zhang, name: Chaoyou Fu, name: Junkang Wu, name: Jinda Lu, name: Kun Wang, name: Xingyu Lu, name: Yunhang Shen, name: Guibin Zhang, name: Dingjie Song, name: Yibo Yan, name: Tianlong Xu, name: Qingsong Wen, name: Zhang Zhang, name: Yan Huang, name: Liang Wang, name: Tieniu Tan
Published: 2025-03-18 17:59:56

Large language models (LLMs) can handle a wide variety of general tasks with
simple prompts, without the need for task-specific training. Multimodal Large
Language Models (MLLMs), built upon LLMs, have demonstrated impressive
potential in tackling complex tasks involving visual, auditory, and textual
data. However, critical issues related to truthfulness, safety, o1-like
reasoning, and alignment with human preference remain insufficiently addressed.
This gap has spurred the emergence of various alignment algorithms, each
targeting different application scenarios and optimization goals. Recent
studies have shown that alignment algorithms are a powerful approach to
resolving the aforementioned challenges. In this paper, we aim to provide a
comprehensive and systematic review of alignment algorithms for MLLMs.
Specifically, we explore four key aspects: (1) the application scenarios
covered by alignment algorithms, including general image understanding,
multi-image, video, and audio, and extended multimodal applications; (2) the
core factors in constructing alignment datasets, including data sources, model
responses, and preference annotations; (3) the benchmarks used to evaluate
alignment algorithms; and (4) a discussion of potential future directions for
the development of alignment algorithms. This work seeks to help researchers
organize current advancements in the field and inspire better alignment
methods. The project page of this paper is available at
https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Alignment.
Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion
  Transformers via In-Context Reflection
Authors: name: Shufan Li, name: Konstantinos Kallidromitis, name: Akash Gokul, name: Arsh Koneru, name: Yusuke Kato, name: Kazuki Kozuka, name: Aditya Grover
Published: 2025-03-15 21:58:12

The predominant approach to advancing text-to-image generation has been
training-time scaling, where larger models are trained on more data using
greater computational resources. While effective, this approach is
computationally expensive, leading to growing interest in inference-time
scaling to improve performance. Currently, inference-time scaling for
text-to-image diffusion models is largely limited to best-of-N sampling, where
multiple images are generated per prompt and a selection model chooses the best
output. Inspired by the recent success of reasoning models like DeepSeek-R1 in
the language domain, we introduce an alternative to naive best-of-N sampling by
equipping text-to-image Diffusion Transformers with in-context reflection
capabilities. We propose Reflect-DiT, a method that enables Diffusion
Transformers to refine their generations using in-context examples of
previously generated images alongside textual feedback describing necessary
improvements. Instead of passively relying on random sampling and hoping for a
better result in a future generation, Reflect-DiT explicitly tailors its
generations to address specific aspects requiring enhancement. Experimental
results demonstrate that Reflect-DiT improves performance on the GenEval
benchmark (+0.19) using SANA-1.0-1.6B as a base model. Additionally, it
achieves a new state-of-the-art score of 0.81 on GenEval while generating only
20 samples per prompt, surpassing the previous best score of 0.80, which was
obtained using a significantly larger model (SANA-1.5-4.8B) with 2048 samples
under the best-of-N approach.
CapArena: Benchmarking and Analyzing Detailed Image Captioning in the
  LLM Era
Authors: name: Kanzhi Cheng, name: Wenpo Song, name: Jiaxin Fan, name: Zheng Ma, name: Qiushi Sun, name: Fangzhi Xu, name: Chenyang Yan, name: Nuo Chen, name: Jianbing Zhang, name: Jiajun Chen
Published: 2025-03-16 02:56:09

Image captioning has been a longstanding challenge in vision-language
research. With the rise of LLMs, modern Vision-Language Models (VLMs) generate
detailed and comprehensive image descriptions. However, benchmarking the
quality of such captions remains unresolved. This paper addresses two key
questions: (1) How well do current VLMs actually perform on image captioning,
particularly compared to humans? We built CapArena, a platform with over 6000
pairwise caption battles and high-quality human preference votes. Our
arena-style evaluation marks a milestone, showing that leading models like
GPT-4o achieve or even surpass human performance, while most open-source models
lag behind. (2) Can automated metrics reliably assess detailed caption quality?
Using human annotations from CapArena, we evaluate traditional and recent
captioning metrics, as well as VLM-as-a-Judge. Our analysis reveals that while
some metrics (e.g., METEOR) show decent caption-level agreement with humans,
their systematic biases lead to inconsistencies in model ranking. In contrast,
VLM-as-a-Judge demonstrates robust discernment at both the caption and model
levels. Building on these insights, we release CapArena-Auto, an accurate and
efficient automated benchmark for detailed captioning, achieving 94.3%
correlation with human rankings at just $4 per test. Data and resources will be
open-sourced at https://caparena.github.io.
Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM
Authors: name: Xinyu Fang, name: Zhijian Chen, name: Kai Lan, name: Shengyuan Ding, name: Yingji Liang, name: Xiangyu Zhao, name: Farong Wen, name: Zicheng Zhang, name: Guofeng Zhang, name: Haodong Duan, name: Kai Chen, name: Dahua Lin
Published: 2025-03-18 17:51:34

Creativity is a fundamental aspect of intelligence, involving the ability to
generate novel and appropriate solutions across diverse contexts. While Large
Language Models (LLMs) have been extensively evaluated for their creative
capabilities, the assessment of Multimodal Large Language Models (MLLMs) in
this domain remains largely unexplored. To address this gap, we introduce
Creation-MMBench, a multimodal benchmark specifically designed to evaluate the
creative capabilities of MLLMs in real-world, image-based tasks. The benchmark
comprises 765 test cases spanning 51 fine-grained tasks. To ensure rigorous
evaluation, we define instance-specific evaluation criteria for each test case,
guiding the assessment of both general response quality and factual consistency
with visual inputs. Experimental results reveal that current open-source MLLMs
significantly underperform compared to proprietary models in creative tasks.
Furthermore, our analysis demonstrates that visual fine-tuning can negatively
impact the base LLM's creative abilities. Creation-MMBench provides valuable
insights for advancing MLLM creativity and establishes a foundation for future
improvements in multimodal generative intelligence. Full data and evaluation
code is released on https://github.com/open-compass/Creation-MMBench.
Measuring AI Ability to Complete Long Tasks
Authors: name: Thomas Kwa, name: Ben West, name: Joel Becker, name: Amy Deng, name: Katharyn Garcia, name: Max Hasin, name: Sami Jawhar, name: Megan Kinniment, name: Nate Rush, name: Sydney Von Arx, name: Ryan Bloom, name: Thomas Broadley, name: Haoxing Du, name: Brian Goodrich, name: Nikola Jurkovic, name: Luke Harold Miles, name: Seraphina Nix, name: Tao Lin, name: Neev Parikh, name: David Rein, name: Lucas Jun Koba Sato, name: Hjalmar Wijk, name: Daniel M. Ziegler, name: Elizabeth Barnes, name: Lawrence Chan
Published: 2025-03-18 17:59:31

Despite rapid progress on AI benchmarks, the real-world meaning of benchmark
performance remains unclear. To quantify the capabilities of AI systems in
terms of human capabilities, we propose a new metric: 50%-task-completion time
horizon. This is the time humans typically take to complete tasks that AI
models can complete with 50% success rate. We first timed humans with relevant
domain expertise on a combination of RE-Bench, HCAST, and 66 novel shorter
tasks. On these tasks, current frontier AI models such as Claude 3.7 Sonnet
have a 50% time horizon of around 50 minutes. Furthermore, frontier AI time
horizon has been doubling approximately every seven months since 2019, though
the trend may have accelerated in 2024. The increase in AI models' time
horizons seems to be primarily driven by greater reliability and ability to
adapt to mistakes, combined with better logical reasoning and tool use
capabilities. We discuss the limitations of our results -- including their
degree of external validity -- and the implications of increased autonomy for
dangerous capabilities. If these results generalize to real-world software
tasks, extrapolation of this trend predicts that within 5 years, AI systems
will be capable of automating many software tasks that currently take humans a
month.
DeepPerception: Advancing R1-like Cognitive Visual Perception in MLLMs
  for Knowledge-Intensive Visual Grounding
Authors: name: Xinyu Ma, name: Ziyang Ding, name: Zhicong Luo, name: Chi Chen, name: Zonghao Guo, name: Derek F. Wong, name: Xiaoyi Feng, name: Maosong Sun
Published: 2025-03-17 04:06:34

Human experts excel at fine-grained visual discrimination by leveraging
domain knowledge to refine perceptual features, a capability that remains
underdeveloped in current Multimodal Large Language Models (MLLMs). Despite
possessing vast expert-level knowledge, MLLMs struggle to integrate reasoning
into visual perception, often generating direct responses without deeper
analysis. To bridge this gap, we introduce knowledge-intensive visual grounding
(KVG), a novel visual grounding task that requires both fine-grained perception
and domain-specific knowledge integration. To address the challenges of KVG, we
propose DeepPerception, an MLLM enhanced with cognitive visual perception
capabilities. Our approach consists of (1) an automated data synthesis pipeline
that generates high-quality, knowledge-aligned training samples, and (2) a
two-stage training framework combining supervised fine-tuning for cognitive
reasoning scaffolding and reinforcement learning to optimize
perception-cognition synergy. To benchmark performance, we introduce KVG-Bench
a comprehensive dataset spanning 10 domains with 1.3K manually curated test
cases. Experimental results demonstrate that DeepPerception significantly
outperforms direct fine-tuning, achieving +8.08\% accuracy improvements on
KVG-Bench and exhibiting +4.60\% superior cross-domain generalization over
baseline approaches. Our findings highlight the importance of integrating
cognitive processes into MLLMs for human-like visual perception and open new
directions for multimodal reasoning research. The data, codes, and models are
released at https://github.com/thunlp/DeepPerception.
Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal
  Control
Authors: name: NVIDIA, name: Hassan Abu Alhaija, name: Jose Alvarez, name: Maciej Bala, name: Tiffany Cai, name: Tianshi Cao, name: Liz Cha, name: Joshua Chen, name: Mike Chen, name: Francesco Ferroni, name: Sanja Fidler, name: Dieter Fox, name: Yunhao Ge, name: Jinwei Gu, name: Ali Hassani, name: Michael Isaev, name: Pooya Jannaty, name: Shiyi Lan, name: Tobias Lasser, name: Huan Ling, name: Ming-Yu Liu, name: Xian Liu, name: Yifan Lu, name: Alice Luo, name: Qianli Ma, name: Hanzi Mao, name: Fabio Ramos, name: Xuanchi Ren, name: Tianchang Shen, name: Shitao Tang, name: Ting-Chun Wang, name: Jay Wu, name: Jiashu Xu, name: Stella Xu, name: Kevin Xie, name: Yuchong Ye, name: Xiaodong Yang, name: Xiaohui Zeng, name: Yu Zeng
Published: 2025-03-18 17:57:54

We introduce Cosmos-Transfer, a conditional world generation model that can
generate world simulations based on multiple spatial control inputs of various
modalities such as segmentation, depth, and edge. In the design, the spatial
conditional scheme is adaptive and customizable. It allows weighting different
conditional inputs differently at different spatial locations. This enables
highly controllable world generation and finds use in various world-to-world
transfer use cases, including Sim2Real. We conduct extensive evaluations to
analyze the proposed model and demonstrate its applications for Physical AI,
including robotics Sim2Real and autonomous vehicle data enrichment. We further
demonstrate an inference scaling strategy to achieve real-time world generation
with an NVIDIA GB200 NVL72 rack. To help accelerate research development in the
field, we open-source our models and code at
https://github.com/nvidia-cosmos/cosmos-transfer1.
DAPO: An Open-Source LLM Reinforcement Learning System at Scale
Authors: name: Qiying Yu, name: Zheng Zhang, name: Ruofei Zhu, name: Yufeng Yuan, name: Xiaochen Zuo, name: Yu Yue, name: Tiantian Fan, name: Gaohong Liu, name: Lingjun Liu, name: Xin Liu, name: Haibin Lin, name: Zhiqi Lin, name: Bole Ma, name: Guangming Sheng, name: Yuxuan Tong, name: Chi Zhang, name: Mofan Zhang, name: Wang Zhang, name: Hang Zhu, name: Jinhua Zhu, name: Jiaze Chen, name: Jiangjie Chen, name: Chengyi Wang, name: Hongli Yu, name: Weinan Dai, name: Yuxuan Song, name: Xiangpeng Wei, name: Hao Zhou, name: Jingjing Liu, name: Wei-Ying Ma, name: Ya-Qin Zhang, name: Lin Yan, name: Mu Qiao, name: Yonghui Wu, name: Mingxuan Wang
Published: 2025-03-18 17:49:06

Inference scaling empowers LLMs with unprecedented reasoning ability, with
reinforcement learning as the core technique to elicit complex reasoning.
However, key technical details of state-of-the-art reasoning LLMs are concealed
(such as in OpenAI o1 blog and DeepSeek R1 technical report), thus the
community still struggles to reproduce their RL training results. We propose
the Decoupled Clip and Dynamic sAmpling
Policy Optimization (DAPO) algorithm, and
fully open-source a state-of-the-art large-scale RL system that achieves 50
points on AIME 2024 using Qwen2.5-32B base model. Unlike previous works that
withhold training details, we introduce four key techniques of our algorithm
that make large-scale LLM RL a success. In addition, we open-source our
training code, which is built on the verl framework, along with a carefully
curated and processed dataset. These components of our open-source system
enhance reproducibility and support future research in large-scale LLM RL.
RWKV-7 "Goose" with Expressive Dynamic State Evolution
Authors: name: Bo Peng, name: Ruichong Zhang, name: Daniel Goldstein, name: Eric Alcaide, name: Haowen Hou, name: Janna Lu, name: William Merrill, name: Guangyu Song, name: Kaifeng Tan, name: Saiteja Utpala, name: Nathan Wilce, name: Johan S. Wind, name: Tianyi Wu, name: Daniel Wuttke, name: Christian Zhou-Zheng
Published: 2025-03-18 17:31:05

We present RWKV-7 "Goose", a new sequence modeling architecture, along with
pre-trained language models that establish a new state-of-the-art in downstream
performance at the 3 billion parameter scale on multilingual tasks, and match
current SoTA English language performance despite being trained on dramatically
fewer tokens than other top 3B models. Nevertheless, RWKV-7 models require only
constant memory usage and constant inference time per token. RWKV-7 introduces
a newly generalized formulation of the delta rule with vector-valued gating and
in-context learning rates, as well as a relaxed value replacement rule. We show
that RWKV-7 can perform state tracking and recognize all regular languages,
while retaining parallelizability of training. This exceeds the capabilities of
Transformers under standard complexity conjectures, which are limited to
TC^0. To demonstrate RWKV-7's language modeling capability, we also
present an extended open source 3.1 trillion token multilingual corpus, and
train four RWKV-7 models ranging from 0.19 billion to 2.9 billion parameters on
this dataset.
  To foster openness, reproduction, and adoption, we release our models and
dataset component listing at https://huggingface.co/RWKV, and our training and
inference code at https://github.com/RWKV/RWKV-LM all under the Apache 2.0
License.
Infinite Mobility: Scalable High-Fidelity Synthesis of Articulated
  Objects via Procedural Generation
Authors: name: Xinyu Lian, name: Zichao Yu, name: Ruiming Liang, name: Yitong Wang, name: Li Ray Luo, name: Kaixu Chen, name: Yuanzhen Zhou, name: Qihong Tang, name: Xudong Xu, name: Zhaoyang Lyu, name: Bo Dai, name: Jiangmiao Pang
Published: 2025-03-17 17:53:56

Large-scale articulated objects with high quality are desperately needed for
multiple tasks related to embodied AI. Most existing methods for creating
articulated objects are either data-driven or simulation based, which are
limited by the scale and quality of the training data or the fidelity and heavy
labour of the simulation. In this paper, we propose Infinite Mobility, a novel
method for synthesizing high-fidelity articulated objects through procedural
generation. User study and quantitative evaluation demonstrate that our method
can produce results that excel current state-of-the-art methods and are
comparable to human-annotated datasets in both physics property and mesh
quality. Furthermore, we show that our synthetic data can be used as training
data for generative models, enabling next-step scaling up. Code is available at
https://github.com/Intern-Nexus/Infinite-Mobility