Orionnnn/hub-stats huggingface dataset:
DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perception
Authors: name: Junjie Wang, name: Bin Chen, name: Yulin Li, name: Bin Kang, name: Yichi Chen, name: Zhuotao Tian
Published: 2025-05-07 13:46:34

Dense visual prediction tasks have been constrained by their reliance on
predefined categories, limiting their applicability in real-world scenarios
where visual concepts are unbounded. While Vision-Language Models (VLMs) like
CLIP have shown promise in open-vocabulary tasks, their direct application to
dense prediction often leads to suboptimal performance due to limitations in
local feature representation. In this work, we present our observation that
CLIP's image tokens struggle to effectively aggregate information from
spatially or semantically related regions, resulting in features that lack
local discriminability and spatial consistency. To address this issue, we
propose DeCLIP, a novel framework that enhances CLIP by decoupling the
self-attention module to obtain ``content'' and ``context'' features
respectively. The ``content'' features are aligned with image crop
representations to improve local discriminability, while ``context'' features
learn to retain the spatial correlations under the guidance of vision
foundation models, such as DINO. Extensive experiments demonstrate that DeCLIP
significantly outperforms existing methods across multiple open-vocabulary
dense prediction tasks, including object detection and semantic segmentation.
Code is available at magenta{https://github.com/xiaomoguhz/DeCLIP}.
Insights into DeepSeek-V3: Scaling Challenges and Reflections on
  Hardware for AI Architectures
Authors: name: Chenggang Zhao, name: Chengqi Deng, name: Chong Ruan, name: Damai Dai, name: Huazuo Gao, name: Jiashi Li, name: Liyue Zhang, name: Panpan Huang, name: Shangyan Zhou, name: Shirong Ma, name: Wenfeng Liang, name: Ying He, name: Yuqing Wang, name: Yuxuan Liu, name: Y. X. Wei
Published: 2025-05-14 12:39:03

The rapid scaling of large language models (LLMs) has unveiled critical
limitations in current hardware architectures, including constraints in memory
capacity, computational efficiency, and interconnection bandwidth. DeepSeek-V3,
trained on 2,048 NVIDIA H800 GPUs, demonstrates how hardware-aware model
co-design can effectively address these challenges, enabling cost-efficient
training and inference at scale. This paper presents an in-depth analysis of
the DeepSeek-V3/R1 model architecture and its AI infrastructure, highlighting
key innovations such as Multi-head Latent Attention (MLA) for enhanced memory
efficiency, Mixture of Experts (MoE) architectures for optimized
computation-communication trade-offs, FP8 mixed-precision training to unlock
the full potential of hardware capabilities, and a Multi-Plane Network Topology
to minimize cluster-level network overhead. Building on the hardware
bottlenecks encountered during DeepSeek-V3's development, we engage in a
broader discussion with academic and industry peers on potential future
hardware directions, including precise low-precision computation units,
scale-up and scale-out convergence, and innovations in low-latency
communication fabrics. These insights underscore the critical role of hardware
and model co-design in meeting the escalating demands of AI workloads, offering
a practical blueprint for innovation in next-generation AI systems.
VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large
  Video Language Models
Authors: name: Pritam Sarkar, name: Ali Etemad
Published: 2025-05-13 11:35:58

Despite recent advances in video understanding, the capabilities of Large
Video Language Models (LVLMs) to perform video-based causal reasoning remains
underexplored, largely due to the absence of relevant and dedicated benchmarks
for evaluating causal reasoning in visually grounded and goal-driven settings.
To fill this gap, we introduce a novel benchmark named Video-based long-form
Causal Reasoning (VCRBench). We create VCRBench using procedural videos of
simple everyday activities, where the steps are deliberately shuffled with each
clip capturing a key causal event, to test whether LVLMs can identify, reason
about, and correctly sequence the events needed to accomplish a specific goal.
Moreover, the benchmark is carefully designed to prevent LVLMs from exploiting
linguistic shortcuts, as seen in multiple-choice or binary QA formats, while
also avoiding the challenges associated with evaluating open-ended QA. Our
evaluation of state-of-the-art LVLMs on VCRBench suggests that these models
struggle with video-based long-form causal reasoning, primarily due to their
difficulty in modeling long-range causal dependencies directly from visual
observations. As a simple step toward enabling such capabilities, we propose
Recognition-Reasoning Decomposition (RRD), a modular approach that breaks
video-based causal reasoning into two sub-tasks of video recognition and causal
reasoning. Our experiments on VCRBench show that RRD significantly boosts
accuracy on VCRBench, with gains of up to 25.2%. Finally, our thorough analysis
reveals interesting insights, for instance, that LVLMs primarily rely on
language knowledge for complex video-based long-form causal reasoning tasks.
BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture,
  Training and Dataset
Authors: name: Jiuhai Chen, name: Zhiyang Xu, name: Xichen Pan, name: Yushi Hu, name: Can Qin, name: Tom Goldstein, name: Lifu Huang, name: Tianyi Zhou, name: Saining Xie, name: Silvio Savarese, name: Le Xue, name: Caiming Xiong, name: Ran Xu
Published: 2025-05-14 17:11:07

Unifying image understanding and generation has gained growing attention in
recent research on multimodal models. Although design choices for image
understanding have been extensively studied, the optimal model architecture and
training recipe for a unified framework with image generation remain
underexplored. Motivated by the strong potential of autoregressive and
diffusion models for high-quality generation and scalability, we conduct a
comprehensive study of their use in unified multimodal settings, with emphasis
on image representations, modeling objectives, and training strategies.
Grounded in these investigations, we introduce a novel approach that employs a
diffusion transformer to generate semantically rich CLIP image features, in
contrast to conventional VAE-based representations. This design yields both
higher training efficiency and improved generative quality. Furthermore, we
demonstrate that a sequential pretraining strategy for unified models-first
training on image understanding and subsequently on image generation-offers
practical advantages by preserving image understanding capability while
developing strong image generation ability. Finally, we carefully curate a
high-quality instruction-tuning dataset BLIP3o-60k for image generation by
prompting GPT-4o with a diverse set of captions covering various scenes,
objects, human gestures, and more. Building on our innovative model design,
training recipe, and datasets, we develop BLIP3-o, a suite of state-of-the-art
unified multimodal models. BLIP3-o achieves superior performance across most of
the popular benchmarks spanning both image understanding and generation tasks.
To facilitate future research, we fully open-source our models, including code,
model weights, training scripts, and pretraining and instruction tuning
datasets.
Tests as Prompt: A Test-Driven-Development Benchmark for LLM Code
  Generation
Authors: name: Yi Cui
Published: 2025-05-13 23:47:12

We introduce WebApp1K, a novel benchmark for evaluating large language models
(LLMs) in test-driven development (TDD) tasks, where test cases serve as both
prompt and verification for code generation. Unlike traditional approaches
relying on natural language prompts, our benchmark emphasizes the ability of
LLMs to interpret and implement functionality directly from test cases,
reflecting real-world software development practices. Comprising 1000 diverse
challenges across 20 application domains, the benchmark evaluates LLMs on their
ability to generate compact, functional code under the constraints of context
length and multi-feature complexity. Our findings highlight instruction
following and in-context learning as critical capabilities for TDD success,
surpassing the importance of general coding proficiency or pretraining
knowledge. Through comprehensive evaluation of 19 frontier models, we reveal
performance bottlenecks, such as instruction loss in long prompts, and provide
a detailed error analysis spanning multiple root causes. This work underscores
the practical value of TDD-specific benchmarks and lays the foundation for
advancing LLM capabilities in rigorous, application-driven coding scenarios.
Fast Text-to-Audio Generation with Adversarial Post-Training
Authors: name: Zachary Novack, name: Zach Evans, name: Zack Zukowski, name: Josiah Taylor, name: CJ Carr, name: Julian Parker, name: Adnan Al-Sinan, name: Gian Marco Iodice, name: Julian McAuley, name: Taylor Berg-Kirkpatrick, name: Jordi Pons
Published: 2025-05-13 02:25:47

Text-to-audio systems, while increasingly performant, are slow at inference
time, thus making their latency unpractical for many creative applications. We
present Adversarial Relativistic-Contrastive (ARC) post-training, the first
adversarial acceleration algorithm for diffusion/flow models not based on
distillation. While past adversarial post-training methods have struggled to
compare against their expensive distillation counterparts, ARC post-training is
a simple procedure that (1) extends a recent relativistic adversarial
formulation to diffusion/flow post-training and (2) combines it with a novel
contrastive discriminator objective to encourage better prompt adherence. We
pair ARC post-training with a number optimizations to Stable Audio Open and
build a model capable of generating approx12s of 44.1kHz stereo audio in
approx75ms on an H100, and approx7s on a mobile edge-device, the fastest
text-to-audio model to our knowledge.
TRAIL: Trace Reasoning and Agentic Issue Localization
Authors: name: Darshan Deshpande, name: Varun Gangal, name: Hersh Mehta, name: Jitin Krishnan, name: Anand Kannappan, name: Rebecca Qian
Published: 2025-05-13 14:55:31

The increasing adoption of agentic workflows across diverse domains brings a
critical need to scalably and systematically evaluate the complex traces these
systems generate. Current evaluation methods depend on manual, domain-specific
human analysis of lengthy workflow traces - an approach that does not scale
with the growing complexity and volume of agentic outputs. Error analysis in
these settings is further complicated by the interplay of external tool outputs
and language model reasoning, making it more challenging than traditional
software debugging. In this work, we (1) articulate the need for robust and
dynamic evaluation methods for agentic workflow traces, (2) introduce a formal
taxonomy of error types encountered in agentic systems, and (3) present a set
of 148 large human-annotated traces (TRAIL) constructed using this taxonomy and
grounded in established agentic benchmarks. To ensure ecological validity, we
curate traces from both single and multi-agent systems, focusing on real-world
applications such as software engineering and open-world information retrieval.
Our evaluations reveal that modern long context LLMs perform poorly at trace
debugging, with the best Gemini-2.5-pro model scoring a mere 11% on TRAIL. Our
dataset and code are made publicly available to support and accelerate future
research in scalable evaluation for agentic workflows.
Bring Reason to Vision: Understanding Perception and Reasoning through
  Model Merging
Authors: name: Shiqi Chen, name: Jinghan Zhang, name: Tongyao Zhu, name: Wei Liu, name: Siyang Gao, name: Miao Xiong, name: Manling Li, name: Junxian He
Published: 2025-05-08 17:56:23

Vision-Language Models (VLMs) combine visual perception with the general
capabilities, such as reasoning, of Large Language Models (LLMs). However, the
mechanisms by which these two abilities can be combined and contribute remain
poorly understood. In this work, we explore to compose perception and reasoning
through model merging that connects parameters of different models. Unlike
previous works that often focus on merging models of the same kind, we propose
merging models across modalities, enabling the incorporation of the reasoning
capabilities of LLMs into VLMs. Through extensive experiments, we demonstrate
that model merging offers a successful pathway to transfer reasoning abilities
from LLMs to VLMs in a training-free manner. Moreover, we utilize the merged
models to understand the internal mechanism of perception and reasoning and how
merging affects it. We find that perception capabilities are predominantly
encoded in the early layers of the model, whereas reasoning is largely
facilitated by the middle-to-late layers. After merging, we observe that all
layers begin to contribute to reasoning, whereas the distribution of perception
abilities across layers remains largely unchanged. These observations shed
light on the potential of model merging as a tool for multimodal integration
and interpretation.
Memorization-Compression Cycles Improve Generalization
Authors: name: Fangyuan Yu
Published: 2025-05-13 16:37:54

We prove theoretically that generalization improves not only through data
scaling but also by compressing internal representations. To operationalize
this insight, we introduce the Information Bottleneck Language Modeling (IBLM)
objective, which reframes language modeling as a constrained optimization
problem: minimizing representation entropy subject to optimal prediction
performance. Empirically, we observe an emergent memorization-compression cycle
during LLM pretraining, evidenced by oscillation positive/negative gradient
alignment between cross-entropy and Matrix-Based Entropy (MBE), a measure of
representation entropy. This pattern closely mirrors the predictive-compressive
trade-off prescribed by IBLM and also parallels the biological alternation
between awake learning and sleep consolidation. Motivated by this observation,
we propose Gated Phase Transition (GAPT), a training algorithm that adaptively
switches between memorization and compression phases. When applied to GPT-2
pretraining on FineWeb dataset, GAPT reduces MBE by 50% and improves
cross-entropy by 4.8%. GAPT improves OOD generalizatino by 35% in a pretraining
task on arithmetic multiplication. In a setting designed to simulate
catastrophic forgetting, GAPT reduces interference by compressing and
separating representations, achieving a 97% improvement in separation -
paralleling the functional role of sleep consolidation.
Aya Vision: Advancing the Frontier of Multilingual Multimodality
Authors: name: Saurabh Dash, name: Yiyang Nan, name: John Dang, name: Arash Ahmadian, name: Shivalika Singh, name: Madeline Smith, name: Bharat Venkitesh, name: Vlad Shmyhlo, name: Viraat Aryabumi, name: Walter Beller-Morales, name: Jeremy Pekmez, name: Jason Ozuzu, name: Pierre Richemond, name: Acyr Locatelli, name: Nick Frosst, name: Phil Blunsom, name: Aidan Gomez, name: Ivan Zhang, name: Marzieh Fadaee, name: Manoj Govindassamy, name: Sudip Roy, name: Matthias Gallé, name: Beyza Ermis, name: Ahmet Üstün, name: Sara Hooker
Published: 2025-05-13 17:03:48

Building multimodal language models is fundamentally challenging: it requires
aligning vision and language modalities, curating high-quality instruction
data, and avoiding the degradation of existing text-only capabilities once
vision is introduced. These difficulties are further magnified in the
multilingual setting, where the need for multimodal data in different languages
exacerbates existing data scarcity, machine translation often distorts meaning,
and catastrophic forgetting is more pronounced. To address the aforementioned
challenges, we introduce novel techniques spanning both data and modeling.
First, we develop a synthetic annotation framework that curates high-quality,
diverse multilingual multimodal instruction data, enabling Aya Vision models to
produce natural, human-preferred responses to multimodal inputs across many
languages. Complementing this, we propose a cross-modal model merging technique
that mitigates catastrophic forgetting, effectively preserving text-only
capabilities while simultaneously enhancing multimodal generative performance.
Aya-Vision-8B achieves best-in-class performance compared to strong multimodal
models such as Qwen-2.5-VL-7B, Pixtral-12B, and even much larger
Llama-3.2-90B-Vision. We further scale this approach with Aya-Vision-32B, which
outperforms models more than twice its size, such as Molmo-72B and
LLaMA-3.2-90B-Vision. Our work advances multilingual progress on the
multi-modal frontier, and provides insights into techniques that effectively
bend the need for compute while delivering extremely high performance.
AM-Thinking-v1: Advancing the Frontier of Reasoning at 32B Scale
Authors: name: Yunjie Ji, name: Xiaoyu Tian, name: Sitong Zhao, name: Haotian Wang, name: Shuaiting Chen, name: Yiping Peng, name: Han Zhao, name: Xiangang Li
Published: 2025-05-13 07:41:15

We present AM-Thinking-v1, a 32B dense language model that advances the
frontier of reasoning, embodying the collaborative spirit of open-source
innovation. Outperforming DeepSeek-R1 and rivaling leading Mixture-of-Experts
(MoE) models like Qwen3-235B-A22B and Seed1.5-Thinking, AM-Thinking-v1 achieves
impressive scores of 85.3 on AIME 2024, 74.4 on AIME 2025, and 70.3 on
LiveCodeBench, showcasing state-of-the-art mathematical and coding capabilities
among open-source models of similar scale.
  Built entirely from the open-source Qwen2.5-32B base model and publicly
available queries, AM-Thinking-v1 leverages a meticulously crafted
post-training pipeline - combining supervised fine-tuning and reinforcement
learning - to deliver exceptional reasoning capabilities. This work
demonstrates that the open-source community can achieve high performance at the
32B scale, a practical sweet spot for deployment and fine-tuning. By striking a
balance between top-tier performance and real-world usability, we hope
AM-Thinking-v1 inspires further collaborative efforts to harness mid-scale
models, pushing reasoning boundaries while keeping accessibility at the core of
innovation. We have open-sourced our model on
https://huggingface.co/a-m-team/AM-Thinking-v1{Hugging Face}.
Optimizing Retrieval-Augmented Generation: Analysis of Hyperparameter
  Impact on Performance and Efficiency
Authors: name: Adel Ammar, name: Anis Koubaa, name: Omer Nacar, name: Wadii Boulila
Published: 2025-05-13 11:13:27

Large language models achieve high task performance yet often hallucinate or
rely on outdated knowledge. Retrieval-augmented generation (RAG) addresses
these gaps by coupling generation with external search. We analyse how
hyperparameters influence speed and quality in RAG systems, covering Chroma and
Faiss vector stores, chunking policies, cross-encoder re-ranking, and
temperature, and we evaluate six metrics: faithfulness, answer correctness,
answer relevancy, context precision, context recall, and answer similarity.
Chroma processes queries 13% faster, whereas Faiss yields higher retrieval
precision, revealing a clear speed-accuracy trade-off. Naive fixed-length
chunking with small windows and minimal overlap outperforms semantic
segmentation while remaining the quickest option. Re-ranking provides modest
gains in retrieval quality yet increases runtime by roughly a factor of 5, so
its usefulness depends on latency constraints. These results help practitioners
balance computational cost and accuracy when tuning RAG systems for
transparent, up-to-date responses. Finally, we re-evaluate the top
configurations with a corrective RAG workflow and show that their advantages
persist when the model can iteratively request additional evidence. We obtain a
near-perfect context precision (99%), which demonstrates that RAG systems can
achieve extremely high retrieval accuracy with the right combination of
hyperparameters, with significant implications for applications where retrieval
quality directly impacts downstream task performance, such as clinical decision
support in healthcare.
Advancing Arabic Reverse Dictionary Systems: A Transformer-Based
  Approach with Dataset Construction Guidelines
Authors: name: Serry Sibaee, name: Samar Ahmed, name: Abdullah Al Harbi, name: Omer Nacar, name: Adel Ammar, name: Yasser Habashi, name: Wadii Boulila
Published: 2025-04-30 09:56:36

This study addresses the critical gap in Arabic natural language processing
by developing an effective Arabic Reverse Dictionary (RD) system that enables
users to find words based on their descriptions or meanings. We present a novel
transformer-based approach with a semi-encoder neural network architecture
featuring geometrically decreasing layers that achieves state-of-the-art
results for Arabic RD tasks. Our methodology incorporates a comprehensive
dataset construction process and establishes formal quality standards for
Arabic lexicographic definitions. Experiments with various pre-trained models
demonstrate that Arabic-specific models significantly outperform general
multilingual embeddings, with ARBERTv2 achieving the best ranking score
(0.0644). Additionally, we provide a formal abstraction of the reverse
dictionary task that enhances theoretical understanding and develop a modular,
extensible Python library (RDTL) with configurable training pipelines. Our
analysis of dataset quality reveals important insights for improving Arabic
definition construction, leading to eight specific standards for building
high-quality reverse dictionary resources. This work contributes significantly
to Arabic computational linguistics and provides valuable tools for language
learning, academic writing, and professional communication in Arabic.
MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable
  Speaker Encoder
Authors: name: Bowen Zhang, name: Congchao Guo, name: Geng Yang, name: Hang Yu, name: Haozhe Zhang, name: Heidi Lei, name: Jialong Mai, name: Junjie Yan, name: Kaiyue Yang, name: Mingqi Yang, name: Peikai Huang, name: Ruiyang Jin, name: Sitan Jiang, name: Weihua Cheng, name: Yawei Li, name: Yichen Xiao, name: Yiying Zhou, name: Yongmao Zhang, name: Yuan Lu, name: Yucen He
Published: 2025-05-12 14:25:20

We introduce MiniMax-Speech, an autoregressive Transformer-based
Text-to-Speech (TTS) model that generates high-quality speech. A key innovation
is our learnable speaker encoder, which extracts timbre features from a
reference audio without requiring its transcription. This enables
MiniMax-Speech to produce highly expressive speech with timbre consistent with
the reference in a zero-shot manner, while also supporting one-shot voice
cloning with exceptionally high similarity to the reference voice. In addition,
the overall quality of the synthesized audio is enhanced through the proposed
Flow-VAE. Our model supports 32 languages and demonstrates excellent
performance across multiple objective and subjective evaluations metrics.
Notably, it achieves state-of-the-art (SOTA) results on objective voice cloning
metrics (Word Error Rate and Speaker Similarity) and has secured the top
position on the public TTS Arena leaderboard. Another key strength of
MiniMax-Speech, granted by the robust and disentangled representations from the
speaker encoder, is its extensibility without modifying the base model,
enabling various applications such as: arbitrary voice emotion control via
LoRA; text to voice (T2V) by synthesizing timbre features directly from text
description; and professional voice cloning (PVC) by fine-tuning timbre
features with additional data. We encourage readers to visit
https://minimax-ai.github.io/tts_tech_report for more examples.