Orionnnn/hub-stats huggingface dataset:
Babel: Open Multilingual Large Language Models Serving Over 90% of
  Global Speakers
Authors: name: Yiran Zhao, name: Chaoqun Liu, name: Yue Deng, name: Jiahao Ying, name: Mahani Aljunied, name: Zhaodonghui Li, name: Lidong Bing, name: Hou Pong Chan, name: Yu Rong, name: Deli Zhao, name: Wenxuan Zhang
Published: 2025-03-02 11:53:55

Large language models (LLMs) have revolutionized natural language processing
(NLP), yet open-source multilingual LLMs remain scarce, with existing models
often limited in language coverage. Such models typically prioritize
well-resourced languages, while widely spoken but under-resourced languages are
often overlooked. To address this disparity, we introduce Babel, an
open multilingual LLM that covers the top 25 languages by number of speakers,
supports over 90% of the global population, and includes many languages
neglected by other open multilingual LLMs. Unlike traditional continue
pretraining approaches, Babel expands its parameter count through a layer
extension technique that elevates Babel's performance ceiling. We introduce two
variants: Babel-9B, designed for efficient inference and
fine-tuning, and Babel-83B, which sets a new standard for open
multilingual LLMs. Extensive evaluations on multilingual tasks demonstrate its
superior performance compared to open LLMs of comparable size. In addition,
using open-source supervised fine-tuning datasets, Babel achieves remarkable
performance, with Babel-9B-Chat leading among 10B-sized LLMs and Babel-83B-Chat
setting a new standard for multilingual tasks, reaching the same level of
commercial models.
Interact, Instruct to Improve: A LLM-Driven Parallel Actor-Reasoner
  Framework for Enhancing Autonomous Vehicle Interactions
Authors: name: Shiyu Fang, name: Jiaqi Liu, name: Chengkai Xu, name: Chen Lv, name: Peng Hang, name: Jian Sun
Published: 2025-03-01 14:15:52

Autonomous Vehicles (AVs) have entered the commercialization stage, but their
limited ability to interact and express intentions still poses challenges in
interactions with Human-driven Vehicles (HVs). Recent advances in large
language models (LLMs) enable bidirectional human-machine communication, but
the conflict between slow inference speed and the need for real-time
decision-making challenges practical deployment. To address these issues, this
paper introduces a parallel Actor-Reasoner framework designed to enable
explicit bidirectional AV-HV interactions across multiple scenarios. First, by
facilitating interactions between the LLM-driven Reasoner and heterogeneous
simulated HVs during training, an interaction memory database, referred to as
the Actor, is established. Then, by introducing the memory partition module and
the two-layer memory retrieval module, the Actor's ability to handle
heterogeneous HVs is significantly enhanced. Ablation studies and comparisons
with other decision-making methods demonstrate that the proposed Actor-Reasoner
framework significantly improves safety and efficiency. Finally, with the
combination of the external Human-Machine Interface (eHMI) information derived
from Reasoner's reasoning and the feasible action solutions retrieved from the
Actor, the effectiveness of the proposed Actor-Reasoner is confirmed in
multi-scenario field interactions. Our code is available at
https://github.com/FanGShiYuu/Actor-Reasoner.
ABC: Achieving Better Control of Multimodal Embeddings using VLMs
Authors: name: Benjamin Schneider, name: Florian Kerschbaum, name: Wenhu Chen
Published: 2025-03-01 03:29:02

Visual embedding models excel at zero-shot tasks like visual retrieval and
classification. However, these models cannot be used for tasks that contain
ambiguity or require user instruction. These tasks necessitate a multimodal
embedding model, which outputs embeddings that combine visual and natural
language input. Existing CLIP-based approaches embed images and text
independently, and fuse the result. We find that this results in weak
interactions between modalities, and poor user control over the representation.
We introduce ABC, an open-source multimodal embedding model that uses a
vision-language model backbone to deeply integrate image features with natural
language instructions. ABC achieves bestfor-size performance on MSCOCO
image-to-text retrieval and is the top performing model on classification and
VQA tasks in the Massive Multimodal Embedding Benchmark. With a strongly
unified vision-language representation, ABC can use natural language to solve
subtle and potentially ambiguous visual retrieval problems. To evaluate this
capability, we design CtrlBench, a benchmark that requires interleaving textual
instructions with image content for correct retrieval. ABC advances the state
of multimodal embeddings by offering high-quality representations and flexible
natural language control. Our model and datasets are available at our project
page.
KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for
  Coding
Authors: name: Zhangchen Xu, name: Yang Liu, name: Yueqin Yin, name: Mingyuan Zhou, name: Radha Poovendran
Published: 2025-03-04 19:17:36

We introduce KodCode, a synthetic dataset that addresses the persistent
challenge of acquiring high-quality, verifiable training data across diverse
difficulties and domains for training Large Language Models for coding.
Existing code-focused resources typically fail to ensure either the breadth of
coverage (e.g., spanning simple coding tasks to advanced algorithmic problems)
or verifiable correctness (e.g., unit tests). In contrast, KodCode comprises
question-solution-test triplets that are systematically validated via a
self-verification procedure. Our pipeline begins by synthesizing a broad range
of coding questions, then generates solutions and test cases with additional
attempts allocated to challenging problems. Finally, post-training data
synthesis is done by rewriting questions into diverse formats and generating
responses under a test-based reject sampling procedure from a reasoning model
(DeepSeek R1). This pipeline yields a large-scale, robust and diverse coding
dataset. KodCode is suitable for supervised fine-tuning and the paired unit
tests also provide great potential for RL tuning. Fine-tuning experiments on
coding benchmarks (HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench)
demonstrate that KodCode-tuned models achieve state-of-the-art performance,
surpassing models like Qwen2.5-Coder-32B-Instruct and
DeepSeek-R1-Distill-Llama-70B.
DoraCycle: Domain-Oriented Adaptation of Unified Generative Model in
  Multimodal Cycles
Authors: name: Rui Zhao, name: Weijia Mao, name: Mike Zheng Shou
Published: 2025-03-05 16:26:58

Adapting generative models to specific domains presents an effective solution
for satisfying specialized requirements. However, adapting to some complex
domains remains challenging, especially when these domains require substantial
paired data to capture the targeted distributions. Since unpaired data from a
single modality, such as vision or language, is more readily available, we
utilize the bidirectional mappings between vision and language learned by the
unified generative model to enable training on unpaired data for domain
adaptation. Specifically, we propose DoraCycle, which integrates two multimodal
cycles: text-to-image-to-text and image-to-text-to-image. The model is
optimized through cross-entropy loss computed at the cycle endpoints, where
both endpoints share the same modality. This facilitates self-evolution of the
model without reliance on annotated text-image pairs. Experimental results
demonstrate that for tasks independent of paired knowledge, such as
stylization, DoraCycle can effectively adapt the unified model using only
unpaired data. For tasks involving new paired knowledge, such as specific
identities, a combination of a small set of paired image-text examples and
larger-scale unpaired data is sufficient for effective domain-oriented
adaptation. The code will be released at https://github.com/showlab/DoraCycle.
Societal Alignment Frameworks Can Improve LLM Alignment
Authors: name: Karolina Stańczak, name: Nicholas Meade, name: Mehar Bhatia, name: Hattie Zhou, name: Konstantin Böttinger, name: Jeremy Barnes, name: Jason Stanley, name: Jessica Montgomery, name: Richard Zemel, name: Nicolas Papernot, name: Nicolas Chapados, name: Denis Therien, name: Timothy P. Lillicrap, name: Ana Marasović, name: Sylvie Delacroix, name: Gillian K. Hadfield, name: Siva Reddy
Published: 2025-02-27 13:26:07

Recent progress in large language models (LLMs) has focused on producing
responses that meet human expectations and align with shared values - a process
coined alignment. However, aligning LLMs remains challenging due to the
inherent disconnect between the complexity of human values and the narrow
nature of the technological approaches designed to address them. Current
alignment methods often lead to misspecified objectives, reflecting the broader
issue of incomplete contracts, the impracticality of specifying a contract
between a model developer, and the model that accounts for every scenario in
LLM alignment. In this paper, we argue that improving LLM alignment requires
incorporating insights from societal alignment frameworks, including social,
economic, and contractual alignment, and discuss potential solutions drawn from
these domains. Given the role of uncertainty within societal alignment
frameworks, we then investigate how it manifests in LLM alignment. We end our
discussion by offering an alternative view on LLM alignment, framing the
underspecified nature of its objectives as an opportunity rather than perfect
their specification. Beyond technical improvements in LLM alignment, we discuss
the need for participatory alignment interface designs.
Unified Video Action Model
Authors: name: Shuang Li, name: Yihuai Gao, name: Dorsa Sadigh, name: Shuran Song
Published: 2025-02-28 21:38:17

A unified video and action model holds significant promise for robotics,
where videos provide rich scene information for action prediction, and actions
provide dynamics information for video prediction. However, effectively
combining video generation and action prediction remains challenging, and
current video generation-based methods struggle to match the performance of
direct policy learning in action accuracy and inference speed. To bridge this
gap, we introduce the Unified Video Action model (UVA), which jointly optimizes
video and action predictions to achieve both high accuracy and efficient action
inference. The key lies in learning a joint video-action latent representation
and decoupling video-action decoding. The joint latent representation bridges
the visual and action domains, effectively modeling the relationship between
video and action sequences. Meanwhile, the decoupled decoding, powered by two
lightweight diffusion heads, enables high-speed action inference by bypassing
video generation during inference. Such a unified framework further enables
versatile functionality through masked input training. By selectively masking
actions or videos, a single model can tackle diverse tasks beyond policy
learning, such as forward and inverse dynamics modeling and video generation.
Via an extensive set of experiments, we demonstrate that UVA can serve as a
general-purpose solution for a wide range of robotics tasks, such as policy
learning, forward/inverse dynamics and video observation prediction, without
compromising performance compared to methods tailored for specific
applications. Results are best viewed on
https://unified-video-action-model.github.io/.
Q-Filters: Leveraging QK Geometry for Efficient KV Cache Compression
Authors: name: Nathan Godey, name: Alessio Devoto, name: Yu Zhao, name: Simone Scardapane, name: Pasquale Minervini, name: Éric de la Clergerie, name: Benoît Sagot
Published: 2025-03-04 17:37:49

Autoregressive language models rely on a Key-Value (KV) Cache, which avoids
re-computing past hidden states during generation, making it faster. As model
sizes and context lengths grow, the KV Cache becomes a significant memory
bottleneck, which calls for compression methods that limit its size during
generation. In this paper, we discover surprising properties of Query (Q) and
Key (K) vectors that allow us to efficiently approximate attention scores
without computing the attention maps. We propose Q-Filters, a training-free KV
Cache compression method that filters out less crucial Key-Value pairs based on
a single context-agnostic projection. Contrarily to many alternatives,
Q-Filters is compatible with FlashAttention, as it does not require direct
access to attention weights. Experimental results in long-context settings
demonstrate that Q-Filters is competitive with attention-based compression
methods such as SnapKV in retrieval tasks while consistently outperforming
efficient compression schemes such as Streaming-LLM in generation setups.
Notably, Q-Filters achieves a 99% accuracy in the needle-in-a-haystack task
with a x32 compression level while reducing the generation perplexity drop by
up to 65% in text generation compared to Streaming-LLM.
A Multimodal Symphony: Integrating Taste and Sound through Generative AI
Authors: name: Matteo Spanio, name: Massimiliano Zampini, name: Antonio Rodà, name: Franco Pierucci
Published: 2025-03-04 17:48:48

In recent decades, neuroscientific and psychological research has traced
direct relationships between taste and auditory perceptions. This article
explores multimodal generative models capable of converting taste information
into music, building on this foundational research. We provide a brief review
of the state of the art in this field, highlighting key findings and
methodologies. We present an experiment in which a fine-tuned version of a
generative music model (MusicGEN) is used to generate music based on detailed
taste descriptions provided for each musical piece. The results are promising:
according the participants' (n=111) evaluation, the fine-tuned model produces
music that more coherently reflects the input taste descriptions compared to
the non-fine-tuned model. This study represents a significant step towards
understanding and developing embodied interactions between AI, sound, and
taste, opening new possibilities in the field of generative AI. We release our
dataset, code and pre-trained model at: https://osf.io/xs5jy/.
Tabby: Tabular Data Synthesis with Language Models
Authors: name: Sonia Cromp, name: Satya Sai Srinath Namburi GNVV, name: Mohammed Alkhudhayri, name: Catherine Cao, name: Samuel Guo, name: Nicholas Roberts, name: Frederic Sala
Published: 2025-03-04 00:32:15

While advances in large language models (LLMs) have greatly improved the
quality of synthetic text data in recent years, synthesizing tabular data has
received relatively less attention. We address this disparity with Tabby, a
simple but powerful post-training modification to the standard Transformer
language model architecture, enabling its use for tabular dataset synthesis.
Tabby enables the representation of differences across columns using Gated
Mixture-of-Experts, with column-specific sets of parameters. Empirically, Tabby
results in data quality near or equal to that of real data. By pairing our
novel LLM table training technique, Plain, with Tabby, we observe up to a 44%
improvement in quality over previous methods. We also show that Tabby extends
beyond tables to more general structured data, reaching parity with real data
on a nested JSON dataset as well.
A Token-level Text Image Foundation Model for Document Understanding
Authors: name: Tongkun Guan, name: Zining Wang, name: Pei Fu, name: Zhengtao Guo, name: Wei Shen, name: Kai Zhou, name: Tiezhu Yue, name: Chen Duan, name: Hao Sun, name: Qianyi Jiang, name: Junfeng Luo, name: Xiaokang Yang
Published: 2025-03-04 06:05:33

In recent years, general visual foundation models (VFMs) have witnessed
increasing adoption, particularly as image encoders for popular multi-modal
large language models (MLLMs). However, without semantically fine-grained
supervision, these models still encounter fundamental prediction errors in the
context of downstream text-image-related tasks, i.e., perception, understanding
and reasoning with images containing small and dense texts. To bridge this gap,
we develop TokenOCR, the first token-level visual foundation model specifically
tailored for text-image-related tasks, designed to support a variety of
traditional downstream applications. To facilitate the pretraining of TokenOCR,
we also devise a high-quality data production pipeline that constructs the
first token-level image text dataset, TokenIT, comprising 20 million images and
1.8 billion token-mask pairs. Furthermore, leveraging this foundation with
exceptional image-as-text capability, we seamlessly replace previous VFMs with
TokenOCR to construct a document-level MLLM, TokenVL, for VQA-based document
understanding tasks. Finally, extensive experiments demonstrate the
effectiveness of TokenOCR and TokenVL. Code, datasets, and weights will be
available at https://token-family.github.io/TokenOCR_project.
Discrete-Time Hybrid Automata Learning: Legged Locomotion Meets
  Skateboarding
Authors: name: Hang Liu, name: Sangli Teng, name: Ben Liu, name: Wei Zhang, name: Maani Ghaffari
Published: 2025-03-03 18:59:23

This paper introduces Discrete-time Hybrid Automata Learning (DHAL), a
framework using on-policy Reinforcement Learning to identify and execute
mode-switching without trajectory segmentation or event function learning.
Hybrid dynamical systems, which include continuous flow and discrete mode
switching, can model robotics tasks like legged robot locomotion. Model-based
methods usually depend on predefined gaits, while model-free approaches lack
explicit mode-switching knowledge. Current methods identify discrete modes via
segmentation before regressing continuous flow, but learning high-dimensional
complex rigid body dynamics without trajectory labels or segmentation is a
challenging open problem. Our approach incorporates a beta policy distribution
and a multi-critic architecture to model contact-guided motions, exemplified by
a challenging quadrupedal robot skateboard task. We validate our method through
simulations and real-world tests, demonstrating robust performance in hybrid
dynamical systems.
Improve Representation for Imbalanced Regression through Geometric
  Constraints
Authors: name: Zijian Dong, name: Yilei Wu, name: Chongyao Chen, name: Yingtian Zou, name: Yichi Zhang, name: Juan Helen Zhou
Published: 2025-03-02 12:31:34

In representation learning, uniformity refers to the uniform feature
distribution in the latent space (i.e., unit hypersphere). Previous work has
shown that improving uniformity contributes to the learning of
under-represented classes. However, most of the previous work focused on
classification; the representation space of imbalanced regression remains
unexplored. Classification-based methods are not suitable for regression tasks
because they cluster features into distinct groups without considering the
continuous and ordered nature essential for regression. In a geometric aspect,
we uniquely focus on ensuring uniformity in the latent space for imbalanced
regression through two key losses: enveloping and homogeneity. The enveloping
loss encourages the induced trace to uniformly occupy the surface of a
hypersphere, while the homogeneity loss ensures smoothness, with
representations evenly spaced at consistent intervals. Our method integrates
these geometric principles into the data representations via a Surrogate-driven
Representation Learning (SRL) framework. Experiments with real-world regression
and operator learning tasks highlight the importance of uniformity in
imbalanced regression and validate the efficacy of our geometry-based loss
functions.
Q-Eval-100K: Evaluating Visual Quality and Alignment Level for
  Text-to-Vision Content
Authors: name: Zicheng Zhang, name: Tengchuan Kou, name: Shushi Wang, name: Chunyi Li, name: Wei Sun, name: Wei Wang, name: Xiaoyu Li, name: Zongyu Wang, name: Xuezhi Cao, name: Xiongkuo Min, name: Xiaohong Liu, name: Guangtao Zhai
Published: 2025-03-04 07:28:45

Evaluating text-to-vision content hinges on two crucial aspects: visual
quality and alignment. While significant progress has been made in developing
objective models to assess these dimensions, the performance of such models
heavily relies on the scale and quality of human annotations. According to
Scaling Law, increasing the number of human-labeled instances follows a
predictable pattern that enhances the performance of evaluation models.
Therefore, we introduce a comprehensive dataset designed to Evaluate Visual
quality and Alignment Level for text-to-vision content (Q-EVAL-100K), featuring
the largest collection of human-labeled Mean Opinion Scores (MOS) for the
mentioned two aspects. The Q-EVAL-100K dataset encompasses both text-to-image
and text-to-video models, with 960K human annotations specifically focused on
visual quality and alignment for 100K instances (60K images and 40K videos).
Leveraging this dataset with context prompt, we propose Q-Eval-Score, a unified
model capable of evaluating both visual quality and alignment with special
improvements for handling long-text prompt alignment. Experimental results
indicate that the proposed Q-Eval-Score achieves superior performance on both
visual quality and alignment, with strong generalization capabilities across
other benchmarks. These findings highlight the significant value of the
Q-EVAL-100K dataset. Data and codes will be available at
https://github.com/zzc-1998/Q-Eval.
IterPref: Focal Preference Learning for Code Generation via Iterative
  Debugging
Authors: name: Jie Wu, name: Haoling Li, name: Xin Zhang, name: Jianwen Luo, name: Yangyu Huang, name: Ruihang Chu, name: Yujiu Yang, name: Scarlett Li
Published: 2025-03-04 16:56:34

Preference learning enhances Code LLMs beyond supervised fine-tuning by
leveraging relative quality comparisons. Existing methods construct preference
pairs from
  candidates based on test case success, treating the higher pass rate sample
as positive and the lower as negative. However, this approach does not pinpoint
specific errors in the code, which prevents the model from learning more
informative error correction patterns, as aligning failing code as a whole
lacks the granularity needed to capture meaningful error-resolution
relationships. To address these issues, we propose IterPref, a new preference
alignment framework that mimics human iterative debugging to refine Code LLMs.
IterPref explicitly locates error regions and aligns the corresponding tokens
via a tailored DPO algorithm. To generate informative pairs, we introduce the
CodeFlow dataset, where samples are iteratively refined until passing tests,
with modifications capturing error corrections. Extensive experiments show that
a diverse suite of Code LLMs equipped with IterPref achieves significant
performance gains in code generation and improves on challenging tasks like
BigCodeBench. In-depth analysis reveals that IterPref yields fewer errors. Our
code and data will be made publicaly available.