Orionnnn/hub-stats huggingface dataset:
Kimi-Audio Technical Report
Authors: name: KimiTeam, name: Ding Ding, name: Zeqian Ju, name: Yichong Leng, name: Songxiang Liu, name: Tong Liu, name: Zeyu Shang, name: Kai Shen, name: Wei Song, name: Xu Tan, name: Heyi Tang, name: Zhengtao Wang, name: Chu Wei, name: Yifei Xin, name: Xinran Xu, name: Jianwei Yu, name: Yutao Zhang, name: Xinyu Zhou, name: Y. Charles, name: Jun Chen, name: Yanru Chen, name: Yulun Du, name: Weiran He, name: Zhenxing Hu, name: Guokun Lai, name: Qingcheng Li, name: Yangyang Liu, name: Weidong Sun, name: Jianzhou Wang, name: Yuzhi Wang, name: Yuefeng Wu, name: Yuxin Wu, name: Dongchao Yang, name: Hao Yang, name: Ying Yang, name: Zhilin Yang, name: Aoxiong Yin, name: Ruibin Yuan, name: Yutong Zhang, name: Zaida Zhou
Published: 2025-04-25 15:31:46

We present Kimi-Audio, an open-source audio foundation model that excels in
audio understanding, generation, and conversation. We detail the practices in
building Kimi-Audio, including model architecture, data curation, training
recipe, inference deployment, and evaluation. Specifically, we leverage a
12.5Hz audio tokenizer, design a novel LLM-based architecture with continuous
features as input and discrete tokens as output, and develop a chunk-wise
streaming detokenizer based on flow matching. We curate a pre-training dataset
that consists of more than 13 million hours of audio data covering a wide range
of modalities including speech, sound, and music, and build a pipeline to
construct high-quality and diverse post-training data. Initialized from a
pre-trained LLM, Kimi-Audio is continual pre-trained on both audio and text
data with several carefully designed tasks, and then fine-tuned to support a
diverse of audio-related tasks. Extensive evaluation shows that Kimi-Audio
achieves state-of-the-art performance on a range of audio benchmarks including
speech recognition, audio understanding, audio question answering, and speech
conversation. We release the codes, model checkpoints, as well as the
evaluation toolkits in https://github.com/MoonshotAI/Kimi-Audio.
Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing
  Efficiency Through Vocabulary Adaptation
Authors: name: Luca Moroni, name: Giovanni Puccetti, name: Pere-Lluis Huguet Cabot, name: Andrei Stefan Bejgu, name: Edoardo Barba, name: Alessio Miaschi, name: Felice Dell'Orletta, name: Andrea Esuli, name: Roberto Navigli
Published: 2025-04-23 18:12:27

The number of pretrained Large Language Models (LLMs) is increasing steadily,
though the majority are designed predominantly for the English language. While
state-of-the-art LLMs can handle other languages, due to language contamination
or some degree of multilingual pretraining data, they are not optimized for
non-English languages, leading to inefficient encoding (high token "fertility")
and slower inference speed. In this work, we thoroughly compare a variety of
vocabulary adaptation techniques for optimizing English LLMs for the Italian
language, and put forward Semantic Alignment Vocabulary Adaptation (SAVA), a
novel method that leverages neural mapping for vocabulary substitution. SAVA
achieves competitive performance across multiple downstream tasks, enhancing
grounded alignment strategies. We adapt two LLMs: Mistral-7b-v0.1, reducing
token fertility by 25\%, and Llama-3.1-8B, optimizing the vocabulary and
reducing the number of parameters by 1 billion. We show that, following the
adaptation of the vocabulary, these models can recover their performance with a
relatively limited stage of continual training on the target language. Finally,
we test the capabilities of the adapted models on various multi-choice and
generative tasks.
VideoVista-CulturalLingo: 360^circ Horizons-Bridging Cultures,
  Languages, and Domains in Video Comprehension
Authors: name: Xinyu Chen, name: Yunxin Li, name: Haoyuan Shi, name: Baotian Hu, name: Wenhan Luo, name: Yaowei Wang, name: Min Zhang
Published: 2025-04-23 13:47:30

Assessing the video comprehension capabilities of multimodal AI systems can
effectively measure their understanding and reasoning abilities. Most video
evaluation benchmarks are limited to a single language, typically English, and
predominantly feature videos rooted in Western cultural contexts. In this
paper, we present VideoVista-CulturalLingo, the first video evaluation
benchmark designed to bridge cultural, linguistic, and domain divide in video
comprehension. Our work differs from existing benchmarks in the following ways:
1) Cultural diversity, incorporating cultures from China, North America, and
Europe; 2) Multi-linguistics, with questions presented in Chinese and
English-two of the most widely spoken languages; and 3) Broad domain, featuring
videos sourced from hundreds of human-created domains. VideoVista-CulturalLingo
contains 1,389 videos and 3,134 QA pairs, and we have evaluated 24 recent
open-source or proprietary video large models. From the experiment results, we
observe that: 1) Existing models perform worse on Chinese-centric questions
than Western-centric ones, particularly those related to Chinese history; 2)
Current open-source models still exhibit limitations in temporal understanding,
especially in the Event Localization task, achieving a maximum score of only
45.2%; 3) Mainstream models demonstrate strong performance in general
scientific questions, while open-source models demonstrate weak performance in
mathematics.
Subject-driven Video Generation via Disentangled Identity and Motion
Authors: name: Daneul Kim, name: Jingxu Zhang, name: Wonjoon Jin, name: Sunghyun Cho, name: Qi Dai, name: Jaesik Park, name: Chong Luo
Published: 2025-04-23 06:48:31

We propose to train a subject-driven customized video generation model
through decoupling the subject-specific learning from temporal dynamics in
zero-shot without additional tuning. A traditional method for video
customization that is tuning-free often relies on large, annotated video
datasets, which are computationally expensive and require extensive annotation.
In contrast to the previous approach, we introduce the use of an image
customization dataset directly on training video customization models,
factorizing the video customization into two folds: (1) identity injection
through image customization dataset and (2) temporal modeling preservation with
a small set of unannotated videos through the image-to-video training method.
Additionally, we employ random image token dropping with randomized image
initialization during image-to-video fine-tuning to mitigate the copy-and-paste
issue. To further enhance learning, we introduce stochastic switching during
joint optimization of subject-specific and temporal features, mitigating
catastrophic forgetting. Our method achieves strong subject consistency and
scalability, outperforming existing video customization models in zero-shot
settings, demonstrating the effectiveness of our framework.
DianJin-R1: Evaluating and Enhancing Financial Reasoning in Large
  Language Models
Authors: name: Jie Zhu, name: Qian Chen, name: Huaixia Dou, name: Junhui Li, name: Lifan Guo, name: Feng Chen, name: Chi Zhang
Published: 2025-04-22 09:01:04

Effective reasoning remains a core challenge for large language models (LLMs)
in the financial domain, where tasks often require domain-specific knowledge,
precise numerical calculations, and strict adherence to compliance rules. We
propose DianJin-R1, a reasoning-enhanced framework designed to address these
challenges through reasoning-augmented supervision and reinforcement learning.
Central to our approach is DianJin-R1-Data, a high-quality dataset constructed
from CFLUE, FinQA, and a proprietary compliance corpus (Chinese Compliance
Check, CCC), combining diverse financial reasoning scenarios with verified
annotations. Our models, DianJin-R1-7B and DianJin-R1-32B, are fine-tuned from
Qwen2.5-7B-Instruct and Qwen2.5-32B-Instruct using a structured format that
generates both reasoning steps and final answers. To further refine reasoning
quality, we apply Group Relative Policy Optimization (GRPO), a reinforcement
learning method that incorporates dual reward signals: one encouraging
structured outputs and another rewarding answer correctness. We evaluate our
models on five benchmarks: three financial datasets (CFLUE, FinQA, and CCC) and
two general reasoning benchmarks (MATH-500 and GPQA-Diamond). Experimental
results show that DianJin-R1 models consistently outperform their non-reasoning
counterparts, especially on complex financial tasks. Moreover, on the
real-world CCC dataset, our single-call reasoning models match or even surpass
the performance of multi-agent systems that require significantly more
computational cost. These findings demonstrate the effectiveness of DianJin-R1
in enhancing financial reasoning through structured supervision and
reward-aligned learning, offering a scalable and practical solution for
real-world applications.
The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs
Authors: name: Piotr Nawrot, name: Robert Li, name: Renjie Huang, name: Sebastian Ruder, name: Kelly Marchisio, name: Edoardo M. Ponti
Published: 2025-04-24 17:39:25

Sparse attention offers a promising strategy to extend long-context
capabilities in Transformer LLMs, yet its viability, its efficiency-accuracy
trade-offs, and systematic scaling studies remain unexplored. To address this
gap, we perform a careful comparison of training-free sparse attention methods
at varying model scales, sequence lengths, and sparsity levels on a diverse
collection of long-sequence tasks-including novel ones that rely on natural
language while remaining controllable and easy to evaluate. Based on our
experiments, we report a series of key findings: 1) an isoFLOPS analysis
reveals that for very long sequences, larger and highly sparse models are
preferable to smaller and dense ones. 2) The level of sparsity attainable while
statistically guaranteeing accuracy preservation is higher during decoding than
prefilling, and correlates with model size in the former. 3) There is no clear
strategy that performs best across tasks and phases, with different units of
sparsification or budget adaptivity needed for different scenarios. Even
moderate sparsity levels often result in significant performance degradation on
at least one task, highlighting that sparse attention is not a universal
solution. 4) We introduce and validate novel scaling laws specifically tailored
for sparse attention, providing evidence that our findings are likely to hold
true beyond our range of experiments. Through these insights, we demonstrate
that sparse attention is a key tool to enhance the capabilities of Transformer
LLMs for processing longer sequences, but requires careful evaluation of
trade-offs for performance-sensitive applications.
Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning
Authors: name: Chris, name: Yichen Wei, name: Yi Peng, name: Xiaokun Wang, name: Weijie Qiu, name: Wei Shen, name: Tianyidan Xie, name: Jiangbo Pei, name: Jianhao Zhang, name: Yunzhuo Hao, name: Xuchen Song, name: Yang Liu, name: Yahui Zhou
Published: 2025-04-23 12:24:10

We present Skywork R1V2, a next-generation multimodal reasoning model and a
major leap forward from its predecessor, Skywork R1V. At its core, R1V2
introduces a hybrid reinforcement learning paradigm that harmonizes
reward-model guidance with rule-based strategies, thereby addressing the
long-standing challenge of balancing sophisticated reasoning capabilities with
broad generalization. To further enhance training efficiency, we propose the
Selective Sample Buffer (SSB) mechanism, which effectively counters the
``Vanishing Advantages'' dilemma inherent in Group Relative Policy Optimization
(GRPO) by prioritizing high-value samples throughout the optimization process.
Notably, we observe that excessive reinforcement signals can induce visual
hallucinations--a phenomenon we systematically monitor and mitigate through
calibrated reward thresholds throughout the training process. Empirical results
affirm the exceptional capability of R1V2, with benchmark-leading performances
such as 62.6 on OlympiadBench, 79.0 on AIME2024, 63.6 on LiveCodeBench, and
74.0 on MMMU. These results underscore R1V2's superiority over existing
open-source models and demonstrate significant progress in closing the
performance gap with premier proprietary systems, including Gemini 2.5 and
OpenAI o4-mini. The Skywork R1V2 model weights have been publicly released to
promote openness and reproducibility
https://huggingface.co/Skywork/Skywork-R1V2-38B.
DC-SAM: In-Context Segment Anything in Images and Videos via Dual
  Consistency
Authors: name: Mengshi Qi, name: Pengfei Zhu, name: Xiangtai Li, name: Xiaoyang Bi, name: Lu Qi, name: Huadong Ma, name: Ming-Hsuan Yang
Published: 2025-04-16 13:41:59

Given a single labeled example, in-context segmentation aims to segment
corresponding objects. This setting, known as one-shot segmentation in few-shot
learning, explores the segmentation model's generalization ability and has been
applied to various vision tasks, including scene understanding and image/video
editing. While recent Segment Anything Models have achieved state-of-the-art
results in interactive segmentation, these approaches are not directly
applicable to in-context segmentation. In this work, we propose the Dual
Consistency SAM (DC-SAM) method based on prompt-tuning to adapt SAM and SAM2
for in-context segmentation of both images and videos. Our key insights are to
enhance the features of the SAM's prompt encoder in segmentation by providing
high-quality visual prompts. When generating a mask prior, we fuse the SAM
features to better align the prompt encoder. Then, we design a cycle-consistent
cross-attention on fused features and initial visual prompts. Next, a
dual-branch design is provided by using the discriminative positive and
negative prompts in the prompt encoder. Furthermore, we design a simple
mask-tube training strategy to adopt our proposed dual consistency method into
the mask tube. Although the proposed DC-SAM is primarily designed for images,
it can be seamlessly extended to the video domain with the support of SAM2.
Given the absence of in-context segmentation in the video domain, we manually
curate and construct the first benchmark from existing video segmentation
datasets, named In-Context Video Object Segmentation (IC-VOS), to better assess
the in-context capability of the model. Extensive experiments demonstrate that
our method achieves 55.5 (+1.4) mIoU on COCO-20i, 73.0 (+1.1) mIoU on
PASCAL-5i, and a J&F score of 71.52 on the proposed IC-VOS benchmark. Our
source code and benchmark are available at https://github.com/zaplm/DC-SAM.
BitNet v2: Native 4-bit Activations with Hadamard Transformation for
  1-bit LLMs
Authors: name: Hongyu Wang, name: Shuming Ma, name: Furu Wei
Published: 2025-04-25 15:17:52

Efficient deployment of 1-bit Large Language Models (LLMs) is hindered by
activation outliers, which complicate quantization to low bit-widths. We
introduce BitNet v2, a novel framework enabling native 4-bit activation
quantization for 1-bit LLMs. To tackle outliers in attention and feed-forward
network activations, we propose H-BitLinear, a module applying an online
Hadamard transformation prior to activation quantization. This transformation
smooths sharp activation distributions into more Gaussian-like forms, suitable
for low-bit representation. Experiments show BitNet v2 trained from scratch
with 8-bit activations matches BitNet b1.58 performance. Crucially, BitNet v2
achieves minimal performance degradation when trained with native 4-bit
activations, significantly reducing memory footprint and computational cost for
batched inference.
Can Large Language Models Help Multimodal Language Analysis? MMLA: A
  Comprehensive Benchmark
Authors: name: Hanlei Zhang, name: Zhuohang Li, name: Yeshuang Zhu, name: Hua Xu, name: Peiwu Wang, name: Haige Zhu, name: Jie Zhou, name: Jinchao Zhang
Published: 2025-04-23 05:25:13

Multimodal language analysis is a rapidly evolving field that leverages
multiple modalities to enhance the understanding of high-level semantics
underlying human conversational utterances. Despite its significance, little
research has investigated the capability of multimodal large language models
(MLLMs) to comprehend cognitive-level semantics. In this paper, we introduce
MMLA, a comprehensive benchmark specifically designed to address this gap. MMLA
comprises over 61K multimodal utterances drawn from both staged and real-world
scenarios, covering six core dimensions of multimodal semantics: intent,
emotion, dialogue act, sentiment, speaking style, and communication behavior.
We evaluate eight mainstream branches of LLMs and MLLMs using three methods:
zero-shot inference, supervised fine-tuning, and instruction tuning. Extensive
experiments reveal that even fine-tuned models achieve only about 60%~70%
accuracy, underscoring the limitations of current MLLMs in understanding
complex human language. We believe that MMLA will serve as a solid foundation
for exploring the potential of large language models in multimodal language
analysis and provide valuable resources to advance this field. The datasets and
code are open-sourced at https://github.com/thuiar/MMLA.
Towards Understanding Camera Motions in Any Video
Authors: name: Zhiqiu Lin, name: Siyuan Cen, name: Daniel Jiang, name: Jay Karhade, name: Hewei Wang, name: Chancharik Mitra, name: Tiffany Ling, name: Yuhan Huang, name: Sifan Liu, name: Mingyu Chen, name: Rushikesh Zawar, name: Xue Bai, name: Yilun Du, name: Chuang Gan, name: Deva Ramanan
Published: 2025-04-21 18:34:57

We introduce CameraBench, a large-scale dataset and benchmark designed to
assess and improve camera motion understanding. CameraBench consists of ~3,000
diverse internet videos, annotated by experts through a rigorous multi-stage
quality control process. One of our contributions is a taxonomy of camera
motion primitives, designed in collaboration with cinematographers. We find,
for example, that some motions like "follow" (or tracking) require
understanding scene content like moving subjects. We conduct a large-scale
human study to quantify human annotation performance, revealing that domain
expertise and tutorial-based training can significantly enhance accuracy. For
example, a novice may confuse zoom-in (a change of intrinsics) with translating
forward (a change of extrinsics), but can be trained to differentiate the two.
Using CameraBench, we evaluate Structure-from-Motion (SfM) and Video-Language
Models (VLMs), finding that SfM models struggle to capture semantic primitives
that depend on scene content, while VLMs struggle to capture geometric
primitives that require precise estimation of trajectories. We then fine-tune a
generative VLM on CameraBench to achieve the best of both worlds and showcase
its applications, including motion-augmented captioning, video question
answering, and video-text retrieval. We hope our taxonomy, benchmark, and
tutorials will drive future efforts towards the ultimate goal of understanding
camera motions in any video.
Dynamic Camera Poses and Where to Find Them
Authors: name: Chris Rockwell, name: Joseph Tung, name: Tsung-Yi Lin, name: Ming-Yu Liu, name: David F. Fouhey, name: Chen-Hsuan Lin
Published: 2025-04-24 17:59:56

Annotating camera poses on dynamic Internet videos at scale is critical for
advancing fields like realistic video generation and simulation. However,
collecting such a dataset is difficult, as most Internet videos are unsuitable
for pose estimation. Furthermore, annotating dynamic Internet videos present
significant challenges even for state-of-theart methods. In this paper, we
introduce DynPose-100K, a large-scale dataset of dynamic Internet videos
annotated with camera poses. Our collection pipeline addresses filtering using
a carefully combined set of task-specific and generalist models. For pose
estimation, we combine the latest techniques of point tracking, dynamic
masking, and structure-from-motion to achieve improvements over the
state-of-the-art approaches. Our analysis and experiments demonstrate that
DynPose-100K is both large-scale and diverse across several key attributes,
opening up avenues for advancements in various downstream applications.
DiMeR: Disentangled Mesh Reconstruction Model
Authors: name: Lutao Jiang, name: Jiantao Lin, name: Kanghao Chen, name: Wenhang Ge, name: Xin Yang, name: Yifan Jiang, name: Yuanhuiyi Lyu, name: Xu Zheng, name: Yingcong Chen
Published: 2025-04-24 15:39:20

With the advent of large-scale 3D datasets, feed-forward 3D generative
models, such as the Large Reconstruction Model (LRM), have gained significant
attention and achieved remarkable success. However, we observe that RGB images
often lead to conflicting training objectives and lack the necessary clarity
for geometry reconstruction. In this paper, we revisit the inductive biases
associated with mesh reconstruction and introduce DiMeR, a novel disentangled
dual-stream feed-forward model for sparse-view mesh reconstruction. The key
idea is to disentangle both the input and framework into geometry and texture
parts, thereby reducing the training difficulty for each part according to the
Principle of Occam's Razor. Given that normal maps are strictly consistent with
geometry and accurately capture surface variations, we utilize normal maps as
exclusive input for the geometry branch to reduce the complexity between the
network's input and output. Moreover, we improve the mesh extraction algorithm
to introduce 3D ground truth supervision. As for texture branch, we use RGB
images as input to obtain the textured mesh. Overall, DiMeR demonstrates robust
capabilities across various tasks, including sparse-view reconstruction,
single-image-to-3D, and text-to-3D. Numerous experiments show that DiMeR
significantly outperforms previous methods, achieving over 30% improvement in
Chamfer Distance on the GSO and OmniObject3D dataset.
Process Reward Models That Think
Authors: name: Muhammad Khalifa, name: Rishabh Agarwal, name: Lajanugen Logeswaran, name: Jaekyeom Kim, name: Hao Peng, name: Moontae Lee, name: Honglak Lee, name: Lu Wang
Published: 2025-04-23 15:44:54

Step-by-step verifiers -- also known as process reward models (PRMs) -- are a
key ingredient for test-time scaling. PRMs require step-level supervision,
making them expensive to train. This work aims to build data-efficient PRMs as
verbalized step-wise reward models that verify every step in the solution by
generating a verification chain-of-thought (CoT). We propose ThinkPRM, a long
CoT verifier fine-tuned on orders of magnitude fewer process labels than those
required by discriminative PRMs. Our approach capitalizes on the inherent
reasoning abilities of long CoT models, and outperforms LLM-as-a-Judge and
discriminative verifiers -- using only 1% of the process labels in PRM800K --
across several challenging benchmarks. Specifically, ThinkPRM beats the
baselines on ProcessBench, MATH-500, and AIME '24 under best-of-N selection and
reward-guided search. In an out-of-domain evaluation on a subset of
GPQA-Diamond and LiveCodeBench, our PRM surpasses discriminative verifiers
trained on the full PRM800K by 8% and 4.5%, respectively. Lastly, under the
same token budget, ThinkPRM scales up verification compute more effectively
compared to LLM-as-a-Judge, outperforming it by 7.2% on a subset of
ProcessBench. Our work highlights the value of generative, long CoT PRMs that
can scale test-time compute for verification while requiring minimal
supervision for training. Our code, data, and models will be released at
https://github.com/mukhal/thinkprm.