Orionnnn/hub-stats huggingface dataset:
DreamO: A Unified Framework for Image Customization
Authors: name: Chong Mou, name: Yanze Wu, name: Wenxu Wu, name: Zinan Guo, name: Pengze Zhang, name: Yufeng Cheng, name: Yiming Luo, name: Fei Ding, name: Shiwen Zhang, name: Xinghui Li, name: Mengtian Li, name: Songtao Zhao, name: Jian Zhang, name: Qian He, name: Xinglong Wu
Published: 2025-04-23 17:41:44

Recently, extensive research on image customization (e.g., identity, subject,
style, background, etc.) demonstrates strong customization capabilities in
large-scale generative models. However, most approaches are designed for
specific tasks, restricting their generalizability to combine different types
of condition. Developing a unified framework for image customization remains an
open challenge. In this paper, we present DreamO, an image customization
framework designed to support a wide range of tasks while facilitating seamless
integration of multiple conditions. Specifically, DreamO utilizes a diffusion
transformer (DiT) framework to uniformly process input of different types.
During training, we construct a large-scale training dataset that includes
various customization tasks, and we introduce a feature routing constraint to
facilitate the precise querying of relevant information from reference images.
Additionally, we design a placeholder strategy that associates specific
placeholders with conditions at particular positions, enabling control over the
placement of conditions in the generated results. Moreover, we employ a
progressive training strategy consisting of three stages: an initial stage
focused on simple tasks with limited data to establish baseline consistency, a
full-scale training stage to comprehensively enhance the customization
capabilities, and a final quality alignment stage to correct quality biases
introduced by low-quality data. Extensive experiments demonstrate that the
proposed DreamO can effectively perform various image customization tasks with
high quality and flexibly integrate different types of control conditions.
VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal
  Large Language Models
Authors: name: Weiye Xu, name: Jiahao Wang, name: Weiyun Wang, name: Zhe Chen, name: Wengang Zhou, name: Aijun Yang, name: Lewei Lu, name: Houqiang Li, name: Xiaohua Wang, name: Xizhou Zhu, name: Wenhai Wang, name: Jifeng Dai, name: Jinguo Zhu
Published: 2025-04-21 17:59:53

Visual reasoning is a core component of human intelligence and a critical
capability for advanced multimodal models. Yet current reasoning evaluations of
multimodal large language models (MLLMs) often rely on text descriptions and
allow language-based reasoning shortcuts, failing to measure genuine
vision-centric reasoning. To address this, we introduce VisuLogic: a benchmark
of 1,000 human-verified problems across six categories (e.g., quantitative
shifts, spatial relations, attribute comparisons). These various types of
questions can be evaluated to assess the visual reasoning capabilities of MLLMs
from multiple perspectives. We evaluate leading MLLMs on this benchmark and
analyze their results to identify common failure modes. Most models score below
30% accuracy-only slightly above the 25% random baseline and far below the
51.4% achieved by humans-revealing significant gaps in visual reasoning.
Furthermore, we provide a supplementary training dataset and a
reinforcement-learning baseline to support further progress.
Trillion 7B Technical Report
Authors: name: Sungjun Han, name: Juyoung Suk, name: Suyeong An, name: Hyungguk Kim, name: Kyuseok Kim, name: Wonsuk Yang, name: Seungtaek Choi, name: Jamin Shin
Published: 2025-04-21 20:54:44

We introduce Trillion-7B, the most token-efficient Korean-centric
multilingual LLM available. Our novel Cross-lingual Document Attention (XLDA)
mechanism enables highly efficient and effective knowledge transfer from
English to target languages like Korean and Japanese. Combined with optimized
data mixtures, language-specific filtering, and tailored tokenizer
construction, Trillion-7B achieves competitive performance while dedicating
only 10\% of its 2T training tokens to multilingual data and requiring just
59.4K H100 GPU hours (\$148K) for full training. Comprehensive evaluations
across 27 benchmarks in four languages demonstrate Trillion-7B's robust
multilingual performance and exceptional cross-lingual consistency.
Pre-DPO: Improving Data Utilization in Direct Preference Optimization
  Using a Guiding Reference Model
Authors: name: Junshu Pan, name: Wei Shen, name: Shulin Huang, name: Qiji Zhou, name: Yue Zhang
Published: 2025-04-22 12:39:30

Direct Preference Optimization (DPO) simplifies reinforcement learning from
human feedback (RLHF) for large language models (LLMs) by directly optimizing
human preferences without an explicit reward model. We find that during DPO
training, the reference model plays the role of a data weight adjuster.
However, the common practice of initializing the policy and reference models
identically in DPO can lead to inefficient data utilization and impose a
performance ceiling. Meanwhile, the lack of a reference model in Simple
Preference Optimization (SimPO) reduces training robustness and necessitates
stricter conditions to prevent catastrophic forgetting. In this work, we
propose Pre-DPO, a simple yet effective DPO-based training paradigm that
enhances preference optimization performance by leveraging a guiding reference
model. This reference model provides foresight into the optimal policy state
achievable through the training preference data, serving as a guiding mechanism
that adaptively assigns higher weights to samples more suitable for the model
and lower weights to those less suitable. Extensive experiments on AlpacaEval
2.0 and Arena-Hard v0.1 benchmarks demonstrate that Pre-DPO consistently
improves the performance of both DPO and SimPO, without relying on external
models or additional data.
Personalized Text-to-Image Generation with Auto-Regressive Models
Authors: name: Kaiyue Sun, name: Xian Liu, name: Yao Teng, name: Xihui Liu
Published: 2025-04-17 17:58:26

Personalized image synthesis has emerged as a pivotal application in
text-to-image generation, enabling the creation of images featuring specific
subjects in diverse contexts. While diffusion models have dominated this
domain, auto-regressive models, with their unified architecture for text and
image modeling, remain underexplored for personalized image generation. This
paper investigates the potential of optimizing auto-regressive models for
personalized image synthesis, leveraging their inherent multimodal capabilities
to perform this task. We propose a two-stage training strategy that combines
optimization of text embeddings and fine-tuning of transformer layers. Our
experiments on the auto-regressive model demonstrate that this method achieves
comparable subject fidelity and prompt following to the leading diffusion-based
personalization methods. The results highlight the effectiveness of
auto-regressive models in personalized image generation, offering a new
direction for future research in this area.
CAPTURe: Evaluating Spatial Reasoning in Vision Language Models via
  Occluded Object Counting
Authors: name: Atin Pothiraj, name: Elias Stengel-Eskin, name: Jaemin Cho, name: Mohit Bansal
Published: 2025-04-21 23:38:43

Recognizing and reasoning about occluded (partially or fully hidden) objects
is vital to understanding visual scenes, as occlusions frequently occur in
real-world environments and act as obstacles for spatial comprehension. To test
models' ability to reason about multiple occluded objects, we introduce a novel
task, Counting Amodally for Patterns Through Unseen REgions (CAPTURe), which
requires a model to count objects arranged in a pattern by inferring how the
pattern continues behind an occluder (an object which blocks parts of the
scene). CAPTURe requires both recognizing visual patterns and reasoning, making
it a useful testbed for evaluating vision-language models (VLMs) on whether
they understand occluded patterns and possess spatial understanding skills. By
requiring models to reason about occluded objects, CAPTURe also tests VLMs'
ability to form world models that would allow them to fill in missing
information. CAPTURe consists of two parts: (1) CAPTURe-real, with manually
filtered images of real objects in patterns and (2) CAPTURe-synthetic, a
controlled diagnostic with generated patterned images. We evaluate four strong
VLMs (GPT-4o, Intern-VL2, Molmo, and Qwen2-VL) on CAPTURe, finding that models
struggle to count on both occluded and unoccluded patterns. Crucially, we find
that models perform worse with occlusion, suggesting that VLMs are also
deficient in inferring unseen spatial relationships: even the strongest VLMs
like GPT-4o fail to count with occlusion. In contrast, we find that humans
achieve very little error on CAPTURe. We also find that providing auxiliary
information of occluded object locations increases performance, underscoring
that the model error comes both from an inability to handle occlusion as well
as difficulty counting in images.
DiffVox: A Differentiable Model for Capturing and Analysing Professional
  Effects Distributions
Authors: name: Chin-Yun Yu, name: Marco A. Martínez-Ramírez, name: Junghyun Koo, name: Ben Hayes, name: Wei-Hsiang Liao, name: György Fazekas, name: Yuki Mitsufuji
Published: 2025-04-20 20:52:58

This study introduces a novel and interpretable model, DiffVox, for matching
vocal effects in music production. DiffVox, short for ``Differentiable Vocal
Fx", integrates parametric equalisation, dynamic range control, delay, and
reverb with efficient differentiable implementations to enable gradient-based
optimisation for parameter estimation. Vocal presets are retrieved from two
datasets, comprising 70 tracks from MedleyDB and 365 tracks from a private
collection. Analysis of parameter correlations highlights strong relationships
between effects and parameters, such as the high-pass and low-shelf filters
often behaving together to shape the low end, and the delay time correlates
with the intensity of the delayed signals. Principal component analysis reveals
connections to McAdams' timbre dimensions, where the most crucial component
modulates the perceived spaciousness while the secondary components influence
spectral brightness. Statistical testing confirms the non-Gaussian nature of
the parameter distribution, highlighting the complexity of the vocal effects
space. These initial findings on the parameter distributions set the foundation
for future research in vocal effects modelling and automatic mixing. Our source
code and datasets are accessible at https://github.com/SonyResearch/diffvox.
IPBench: Benchmarking the Knowledge of Large Language Models in
  Intellectual Property
Authors: name: Qiyao Wang, name: Guhong Chen, name: Hongbo Wang, name: Huaren Liu, name: Minghui Zhu, name: Zhifei Qin, name: Linwei Li, name: Yilin Yue, name: Shiqiang Wang, name: Jiayan Li, name: Yihang Wu, name: Ziqiang Liu, name: Longze Chen, name: Run Luo, name: Liyang Fan, name: Jiaming Li, name: Lei Zhang, name: Kan Xu, name: Hongfei Lin, name: Hamid Alinejad-Rokny, name: Shiwen Ni, name: Yuan Lin, name: Min Yang
Published: 2025-04-22 02:00:41

Intellectual Property (IP) is a unique domain that integrates technical and
legal knowledge, making it inherently complex and knowledge-intensive. As large
language models (LLMs) continue to advance, they show great potential for
processing IP tasks, enabling more efficient analysis, understanding, and
generation of IP-related content. However, existing datasets and benchmarks
either focus narrowly on patents or cover limited aspects of the IP field,
lacking alignment with real-world scenarios. To bridge this gap, we introduce
the first comprehensive IP task taxonomy and a large, diverse bilingual
benchmark, IPBench, covering 8 IP mechanisms and 20 tasks. This benchmark is
designed to evaluate LLMs in real-world intellectual property applications,
encompassing both understanding and generation. We benchmark 16 LLMs, ranging
from general-purpose to domain-specific models, and find that even the
best-performing model achieves only 75.8% accuracy, revealing substantial room
for improvement. Notably, open-source IP and law-oriented models lag behind
closed-source general-purpose models. We publicly release all data and code of
IPBench and will continue to update it with additional IP-related tasks to
better reflect real-world challenges in the intellectual property domain.
LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making
  Abilities
Authors: name: Thomas Schmied, name: Jörg Bornschein, name: Jordi Grau-Moya, name: Markus Wulfmeier, name: Razvan Pascanu
Published: 2025-04-22 17:57:14

The success of Large Language Models (LLMs) has sparked interest in various
agentic applications. A key hypothesis is that LLMs, leveraging common sense
and Chain-of-Thought (CoT) reasoning, can effectively explore and efficiently
solve complex domains. However, LLM agents have been found to suffer from
sub-optimal exploration and the knowing-doing gap, the inability to effectively
act on knowledge present in the model. In this work, we systematically study
why LLMs perform sub-optimally in decision-making scenarios. In particular, we
closely examine three prevalent failure modes: greediness, frequency bias, and
the knowing-doing gap. We propose mitigation of these shortcomings by
fine-tuning via Reinforcement Learning (RL) on self-generated CoT rationales.
Our experiments across multi-armed bandits, contextual bandits, and
Tic-tac-toe, demonstrate that RL fine-tuning enhances the decision-making
abilities of LLMs by increasing exploration and narrowing the knowing-doing
gap. Finally, we study both classic exploration mechanisms, such as
epsilon-greedy, and LLM-specific approaches, such as self-correction and
self-consistency, to enable more effective fine-tuning of LLMs for
decision-making.
Kuwain 1.5B: An Arabic SLM via Language Injection
Authors: name: Khalil Hennara, name: Sara Chrouf, name: Mohamed Motaism Hamed, name: Zeina Aldallal, name: Omar Hadid, name: Safwan AlModhayan
Published: 2025-04-21 14:17:25

Enhancing existing models with new knowledge is a crucial aspect of AI
development. This paper introduces a novel method for integrating a new
language into a large language model (LLM). Our approach successfully
incorporates a previously unseen target language into an existing LLM without
compromising its prior knowledge. We trained a tiny model with 1.5 billion
parameters named Kuwain by injecting the Arabic language into a small
open-source model mainly trained in English. Our method demonstrates
significant improvements in Arabic language performance, with an average 8%
improvement across various benchmarks, while retaining the model's existing
knowledge with a minimum amount of the original model's data. This offers a
cost-effective alternative to training a comprehensive model in both English
and Arabic. The results highlight the potential for efficient, targeted
language model expansion without extensive retraining or resource-intensive
processes.
BookWorld: From Novels to Interactive Agent Societies for Creative Story
  Generation
Authors: name: Yiting Ran, name: Xintao Wang, name: Tian Qiu, name: Jiaqing Liang, name: Yanghua Xiao, name: Deqing Yang
Published: 2025-04-20 08:56:27

Recent advances in large language models (LLMs) have enabled social
simulation through multi-agent systems. Prior efforts focus on agent societies
created from scratch, assigning agents with newly defined personas. However,
simulating established fictional worlds and characters remain largely
underexplored, despite its significant practical value. In this paper, we
introduce BookWorld, a comprehensive system for constructing and simulating
book-based multi-agent societies. BookWorld's design covers comprehensive
real-world intricacies, including diverse and dynamic characters, fictional
worldviews, geographical constraints and changes, e.t.c. BookWorld enables
diverse applications including story generation, interactive games and social
simulation, offering novel ways to extend and explore beloved fictional works.
Through extensive experiments, we demonstrate that BookWorld generates
creative, high-quality stories while maintaining fidelity to the source books,
surpassing previous methods with a win rate of 75.36%. The code of this paper
can be found at the project page: https://bookworld2025.github.io/.
WALL-E 2.0: World Alignment by NeuroSymbolic Learning improves World
  Model-based LLM Agents
Authors: name: Siyu Zhou, name: Tianyi Zhou, name: Yijun Yang, name: Guodong Long, name: Deheng Ye, name: Jing Jiang, name: Chengqi Zhang
Published: 2025-04-22 10:58:27

Can we build accurate world models out of large language models (LLMs)? How
can world models benefit LLM agents? The gap between the prior knowledge of
LLMs and the specified environment's dynamics usually bottlenecks LLMs'
performance as world models. To bridge the gap, we propose a training-free
"world alignment" that learns an environment's symbolic knowledge complementary
to LLMs. The symbolic knowledge covers action rules, knowledge graphs, and
scene graphs, which are extracted by LLMs from exploration trajectories and
encoded into executable codes to regulate LLM agents' policies. We further
propose an RL-free, model-based agent "WALL-E 2.0" through the model-predictive
control (MPC) framework. Unlike classical MPC requiring costly optimization on
the fly, we adopt an LLM agent as an efficient look-ahead optimizer of future
steps' actions by interacting with the neurosymbolic world model. While the LLM
agent's strong heuristics make it an efficient planner in MPC, the quality of
its planned actions is also secured by the accurate predictions of the aligned
world model. They together considerably improve learning efficiency in a new
environment. On open-world challenges in Mars (Minecraft like) and ALFWorld
(embodied indoor environments), WALL-E 2.0 significantly outperforms existing
methods, e.g., surpassing baselines in Mars by 16.1%-51.6% of success rate and
by at least 61.7% in score. In ALFWorld, it achieves a new record 98% success
rate after only 4 iterations.
TTRL: Test-Time Reinforcement Learning
Authors: name: Yuxin Zuo, name: Kaiyan Zhang, name: Shang Qu, name: Li Sheng, name: Xuekai Zhu, name: Biqing Qi, name: Youbang Sun, name: Ganqu Cui, name: Ning Ding, name: Bowen Zhou
Published: 2025-04-22 17:59:56

This paper investigates Reinforcement Learning (RL) on data without explicit
labels for reasoning tasks in Large Language Models (LLMs). The core challenge
of the problem is reward estimation during inference while not having access to
ground-truth information. While this setting appears elusive, we find that
common practices in Test-Time Scaling (TTS), such as majority voting, yield
surprisingly effective rewards suitable for driving RL training. In this work,
we introduce Test-Time Reinforcement Learning (TTRL), a novel method for
training LLMs using RL on unlabeled data. TTRL enables self-evolution of LLMs
by utilizing the priors in the pre-trained models. Our experiments demonstrate
that TTRL consistently improves performance across a variety of tasks and
models. Notably, TTRL boosts the pass@1 performance of Qwen-2.5-Math-7B by
approximately 159% on the AIME 2024 with only unlabeled test data. Furthermore,
although TTRL is only supervised by the Maj@N metric, TTRL has demonstrated
performance to consistently surpass the upper limit of the initial model, and
approach the performance of models trained directly on test data with
ground-truth labels. Our experimental findings validate the general
effectiveness of TTRL across various tasks, and highlight TTRL's potential for
broader tasks and domains. GitHub: https://github.com/PRIME-RL/TTRL
CheXWorld: Exploring Image World Modeling for Radiograph Representation
  Learning
Authors: name: Yang Yue, name: Yulin Wang, name: Chenxin Tao, name: Pan Liu, name: Shiji Song, name: Gao Huang
Published: 2025-04-18 17:50:43

Humans can develop internal world models that encode common sense knowledge,
telling them how the world works and predicting the consequences of their
actions. This concept has emerged as a promising direction for establishing
general-purpose machine-learning models in recent preliminary works, e.g., for
visual representation learning. In this paper, we present CheXWorld, the first
effort towards a self-supervised world model for radiographic images.
Specifically, our work develops a unified framework that simultaneously models
three aspects of medical knowledge essential for qualified radiologists,
including 1) local anatomical structures describing the fine-grained
characteristics of local tissues (e.g., architectures, shapes, and textures);
2) global anatomical layouts describing the global organization of the human
body (e.g., layouts of organs and skeletons); and 3) domain variations that
encourage CheXWorld to model the transitions across different appearance
domains of radiographs (e.g., varying clarity, contrast, and exposure caused by
collecting radiographs from different hospitals, devices, or patients).
Empirically, we design tailored qualitative and quantitative analyses,
revealing that CheXWorld successfully captures these three dimensions of
medical knowledge. Furthermore, transfer learning experiments across eight
medical image classification and segmentation benchmarks showcase that
CheXWorld significantly outperforms existing SSL methods and large-scale
medical foundation models. Code & pre-trained models are available at
https://github.com/LeapLabTHU/CheXWorld.
IV-Bench: A Benchmark for Image-Grounded Video Perception and Reasoning
  in Multimodal LLMs
Authors: name: David Ma, name: Yuanxing Zhang, name: Jincheng Ren, name: Jarvis Guo, name: Yifan Yao, name: Zhenlin Wei, name: Zhenzhu Yang, name: Zhongyuan Peng, name: Boyu Feng, name: Jun Ma, name: Xiao Gu, name: Zhoufutu Wen, name: King Zhu, name: Yancheng He, name: Meng Cao, name: Shiwen Ni, name: Jiaheng Liu, name: Wenhao Huang, name: Ge Zhang, name: Xiaojie Jin
Published: 2025-04-21 19:53:44

Existing evaluation frameworks for Multimodal Large Language Models (MLLMs)
primarily focus on image reasoning or general video understanding tasks,
largely overlooking the significant role of image context in video
comprehension. To bridge this gap, we propose IV-Bench, the first comprehensive
benchmark for evaluating Image-Grounded Video Perception and Reasoning.
IV-Bench consists of 967 videos paired with 2,585 meticulously annotated
image-text queries across 13 tasks (7 perception and 6 reasoning tasks) and 5
representative categories. Extensive evaluations of state-of-the-art
open-source (e.g., InternVL2.5, Qwen2.5-VL) and closed-source (e.g., GPT-4o,
Gemini2-Flash and Gemini2-Pro) MLLMs demonstrate that current models
substantially underperform in image-grounded video Perception and Reasoning,
merely achieving at most 28.9% accuracy. Further analysis reveals key factors
influencing model performance on IV-Bench, including inference pattern, frame
number, and resolution. Additionally, through a simple data synthesis approach,
we demonstratethe challenges of IV- Bench extend beyond merely aligning the
data format in the training proecss. These findings collectively provide
valuable insights for future research. Our codes and data are released in
https://github.com/multimodal-art-projection/IV-Bench.