Orionnnn/hub-stats huggingface dataset:
FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networks
Authors: name: Luca Della Libera, name: Francesco Paissan, name: Cem Subakan, name: Mirco Ravanelli
Published: 2025-02-06 19:24:50

Large language models have revolutionized natural language processing through
self-supervised pretraining on massive datasets. Inspired by this success,
researchers have explored adapting these methods to speech by discretizing
continuous audio into tokens using neural audio codecs. However, existing
approaches face limitations, including high bitrates, the loss of either
semantic or acoustic information, and the reliance on multi-codebook designs
when trying to capture both, which increases architectural complexity for
downstream tasks. To address these challenges, we introduce FocalCodec, an
efficient low-bitrate codec based on focal modulation that utilizes a single
binary codebook to compress speech between 0.16 and 0.65 kbps. FocalCodec
delivers competitive performance in speech resynthesis and voice conversion at
lower bitrates than the current state-of-the-art, while effectively handling
multilingual speech and noisy environments. Evaluation on downstream tasks
shows that FocalCodec successfully preserves sufficient semantic and acoustic
information, while also being well-suited for generative modeling. Demo
samples, code and checkpoints are available at
https://lucadellalib.github.io/focalcodec-web/.
Teaching Language Models to Critique via Reinforcement Learning
Authors: name: Zhihui Xie, name: Jie chen, name: Liyu Chen, name: Weichao Mao, name: Jingjing Xu, name: Lingpeng Kong
Published: 2025-02-05 02:18:46

Teaching large language models (LLMs) to critique and refine their outputs is
crucial for building systems that can iteratively improve, yet it is
fundamentally limited by the ability to provide accurate judgments and
actionable suggestions. In this work, we study LLM critics for code generation
and propose CTRL, a framework for Critic
Training via Reinforcement Learning, which
trains a critic model to generate feedback that maximizes correction
performance for a fixed generator model without human supervision. Our results
demonstrate that critics trained with CTRL significantly enhance
pass rates and mitigate compounding errors across both base and stronger
generator models. Furthermore, we show that these critic models act as accurate
generative reward models and enable test-time scaling through iterative
critique-revision, achieving up to 106.1% relative improvements across
challenging code generation benchmarks.
Magic 1-For-1: Generating One Minute Video Clips within One Minute
Authors: name: Hongwei Yi, name: Shitong Shao, name: Tian Ye, name: Jiantong Zhao, name: Qingyu Yin, name: Michael Lingelbach, name: Li Yuan, name: Yonghong Tian, name: Enze Xie, name: Daquan Zhou
Published: 2025-02-11 16:58:15

In this technical report, we present Magic 1-For-1 (Magic141), an efficient
video generation model with optimized memory consumption and inference latency.
The key idea is simple: factorize the text-to-video generation task into two
separate easier tasks for diffusion step distillation, namely text-to-image
generation and image-to-video generation. We verify that with the same
optimization algorithm, the image-to-video task is indeed easier to converge
over the text-to-video task. We also explore a bag of optimization tricks to
reduce the computational cost of training the image-to-video (I2V) models from
three aspects: 1) model convergence speedup by using a multi-modal prior
condition injection; 2) inference latency speed up by applying an adversarial
step distillation, and 3) inference memory cost optimization with parameter
sparsification. With those techniques, we are able to generate 5-second video
clips within 3 seconds. By applying a test time sliding window, we are able to
generate a minute-long video within one minute with significantly improved
visual quality and motion dynamics, spending less than 1 second for generating
1 second video clips on average. We conduct a series of preliminary
explorations to find out the optimal tradeoff between computational cost and
video quality during diffusion step distillation and hope this could be a good
foundation model for open-source explorations. The code and the model weights
are available at https://github.com/DA-Group-PKU/Magic-1-For-1.
Forget What You Know about LLMs Evaluations - LLMs are Like a Chameleon
Authors: name: Nurit Cohen-Inger, name: Yehonatan Elisha, name: Bracha Shapira, name: Lior Rokach, name: Seffi Cohen
Published: 2025-02-11 10:43:36

Large language models (LLMs) often appear to excel on public benchmarks, but
these high scores may mask an overreliance on dataset-specific surface cues
rather than true language understanding. We introduce the Chameleon Benchmark
Overfit Detector (C-BOD), a meta-evaluation framework that systematically
distorts benchmark prompts via a parametric transformation and detects
overfitting of LLMs. By rephrasing inputs while preserving their semantic
content and labels, C-BOD exposes whether a model's performance is driven by
memorized patterns. Evaluated on the MMLU benchmark using 26 leading LLMs, our
method reveals an average performance degradation of 2.15% under modest
perturbations, with 20 out of 26 models exhibiting statistically significant
differences. Notably, models with higher baseline accuracy exhibit larger
performance differences under perturbation, and larger LLMs tend to be more
sensitive to rephrasings indicating that both cases may overrely on fixed
prompt patterns. In contrast, the Llama family and models with lower baseline
accuracy show insignificant degradation, suggesting reduced dependency on
superficial cues. Moreover, C-BOD's dataset- and model-agnostic design allows
easy integration into training pipelines to promote more robust language
understanding. Our findings challenge the community to look beyond leaderboard
scores and prioritize resilience and generalization in LLM evaluation.
VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video
  Generation
Authors: name: Sixiao Zheng, name: Zimian Peng, name: Yanpeng Zhou, name: Yi Zhu, name: Hang Xu, name: Xiangru Huang, name: Yanwei Fu
Published: 2025-02-11 13:11:59

Recent image-to-video generation methods have demonstrated success in
enabling control over one or two visual elements, such as camera trajectory or
object motion. However, these methods are unable to offer control over multiple
visual elements due to limitations in data and network efficacy. In this paper,
we introduce VidCRAFT3, a novel framework for precise image-to-video generation
that enables control over camera motion, object motion, and lighting direction
simultaneously. To better decouple control over each visual element, we propose
the Spatial Triple-Attention Transformer, which integrates lighting direction,
text, and image in a symmetric way. Since most real-world video datasets lack
lighting annotations, we construct a high-quality synthetic video dataset, the
VideoLightingDirection (VLD) dataset. This dataset includes lighting direction
annotations and objects of diverse appearance, enabling VidCRAFT3 to
effectively handle strong light transmission and reflection effects.
Additionally, we propose a three-stage training strategy that eliminates the
need for training data annotated with multiple visual elements (camera motion,
object motion, and lighting direction) simultaneously. Extensive experiments on
benchmark datasets demonstrate the efficacy of VidCRAFT3 in producing
high-quality video content, surpassing existing state-of-the-art methods in
terms of control granularity and visual coherence. All code and data will be
publicly available. Project page: https://sixiaozheng.github.io/VidCRAFT3/.
CAD-Editor: A Locate-then-Infill Framework with Automated Training Data
  Synthesis for Text-Based CAD Editing
Authors: name: Yu Yuan, name: Shizhao Sun, name: Qi Liu, name: Jiang Bian
Published: 2025-02-06 11:57:14

Computer Aided Design (CAD) is indispensable across various industries.
Text-based CAD editing, which automates the modification of CAD models
based on textual instructions, holds great potential but remains underexplored.
Existing methods primarily focus on design variation generation or text-based
CAD generation, either lacking support for text-based control or neglecting
existing CAD models as constraints. We introduce CAD-Editor, the first
framework for text-based CAD editing. To address the challenge of demanding
triplet data with accurate correspondence for training, we propose an automated
data synthesis pipeline. This pipeline utilizes design variation models to
generate pairs of original and edited CAD models and employs Large
Vision-Language Models (LVLMs) to summarize their differences into editing
instructions. To tackle the composite nature of text-based CAD editing, we
propose a locate-then-infill framework that decomposes the task into two
focused sub-tasks: locating regions requiring modification and infilling these
regions with appropriate edits. Large Language Models (LLMs) serve as the
backbone for both sub-tasks, leveraging their capabilities in natural language
understanding and CAD knowledge. Experiments show that CAD-Editor achieves
superior performance both quantitatively and qualitatively.
Enhance-A-Video: Better Generated Video for Free
Authors: name: Yang Luo, name: Xuanlei Zhao, name: Mengzhao Chen, name: Kaipeng Zhang, name: Wenqi Shao, name: Kai Wang, name: Zhangyang Wang, name: Yang You
Published: 2025-02-11 12:22:35

DiT-based video generation has achieved remarkable results, but research into
enhancing existing models remains relatively unexplored. In this work, we
introduce a training-free approach to enhance the coherence and quality of
DiT-based generated videos, named Enhance-A-Video. The core idea is enhancing
the cross-frame correlations based on non-diagonal temporal attention
distributions. Thanks to its simple design, our approach can be easily applied
to most DiT-based video generation frameworks without any retraining or
fine-tuning. Across various DiT-based video generation models, our approach
demonstrates promising improvements in both temporal consistency and visual
quality. We hope this research can inspire future explorations in video
generation enhancement.
Auditing Prompt Caching in Language Model APIs
Authors: name: Chenchen Gu, name: Xiang Lisa Li, name: Rohith Kuditipudi, name: Percy Liang, name: Tatsunori Hashimoto
Published: 2025-02-11 18:58:04

Prompt caching in large language models (LLMs) results in data-dependent
timing variations: cached prompts are processed faster than non-cached prompts.
These timing differences introduce the risk of side-channel timing attacks. For
example, if the cache is shared across users, an attacker could identify cached
prompts from fast API response times to learn information about other users'
prompts. Because prompt caching may cause privacy leakage, transparency around
the caching policies of API providers is important. To this end, we develop and
conduct statistical audits to detect prompt caching in real-world LLM API
providers. We detect global cache sharing across users in seven API providers,
including OpenAI, resulting in potential privacy leakage about users' prompts.
Timing variations due to prompt caching can also result in leakage of
information about model architecture. Namely, we find evidence that OpenAI's
embedding model is a decoder-only Transformer, which was previously not
publicly known.
NatureLM: Deciphering the Language of Nature for Scientific Discovery
Authors: name: Yingce Xia, name: Peiran Jin, name: Shufang Xie, name: Liang He, name: Chuan Cao, name: Renqian Luo, name: Guoqing Liu, name: Yue Wang, name: Zequn Liu, name: Yuan-Jyue Chen, name: Zekun Guo, name: Yeqi Bai, name: Pan Deng, name: Yaosen Min, name: Ziheng Lu, name: Hongxia Hao, name: Han Yang, name: Jielan Li, name: Chang Liu, name: Jia Zhang, name: Jianwei Zhu, name: Kehan Wu, name: Wei Zhang, name: Kaiyuan Gao, name: Qizhi Pei, name: Qian Wang, name: Xixian Liu, name: Yanting Li, name: Houtian Zhu, name: Yeqing Lu, name: Mingqian Ma, name: Zun Wang, name: Tian Xie, name: Krzysztof Maziarz, name: Marwin Segler, name: Zhao Yang, name: Zilong Chen, name: Yu Shi, name: Shuxin Zheng, name: Lijun Wu, name: Chen Hu, name: Peggy Dai, name: Tie-Yan Liu, name: Haiguang Liu, name: Tao Qin
Published: 2025-02-11 13:08:03

Foundation models have revolutionized natural language processing and
artificial intelligence, significantly enhancing how machines comprehend and
generate human languages. Inspired by the success of these foundation models,
researchers have developed foundation models for individual scientific domains,
including small molecules, materials, proteins, DNA, and RNA. However, these
models are typically trained in isolation, lacking the ability to integrate
across different scientific domains. Recognizing that entities within these
domains can all be represented as sequences, which together form the "language
of nature", we introduce Nature Language Model (briefly, NatureLM), a
sequence-based science foundation model designed for scientific discovery.
Pre-trained with data from multiple scientific domains, NatureLM offers a
unified, versatile model that enables various applications including: (i)
generating and optimizing small molecules, proteins, RNA, and materials using
text instructions; (ii) cross-domain generation/design, such as
protein-to-molecule and protein-to-RNA generation; and (iii) achieving
state-of-the-art performance in tasks like SMILES-to-IUPAC translation and
retrosynthesis on USPTO-50k. NatureLM offers a promising generalist approach
for various scientific tasks, including drug discovery (hit
generation/optimization, ADMET optimization, synthesis), novel material design,
and the development of therapeutic proteins or nucleotides. We have developed
NatureLM models in different sizes (1 billion, 8 billion, and 46.7 billion
parameters) and observed a clear improvement in performance as the model size
increases.
Hephaestus: Improving Fundamental Agent Capabilities of Large Language
  Models through Continual Pre-Training
Authors: name: Yuchen Zhuang, name: Jingfeng Yang, name: Haoming Jiang, name: Xin Liu, name: Kewei Cheng, name: Sanket Lokegaonkar, name: Yifan Gao, name: Qing Ping, name: Tianyi Liu, name: Binxuan Huang, name: Zheng Li, name: Zhengyang Wang, name: Pei Chen, name: Ruijie Wang, name: Rongzhi Zhang, name: Nasser Zalmout, name: Priyanka Nigam, name: Bing Yin, name: Chao Zhang
Published: 2025-02-10 15:54:34

Due to the scarcity of agent-oriented pre-training data, LLM-based autonomous
agents typically rely on complex prompting or extensive fine-tuning, which
often fails to introduce new capabilities while preserving strong
generalizability. We introduce Hephaestus-Forge, the first large-scale
pre-training corpus designed to enhance the fundamental capabilities of LLM
agents in API function calling, intrinsic reasoning and planning, and adapting
to environmental feedback. Hephaestus-Forge comprises 103B agent-specific data
encompassing 76,537 APIs, including both tool documentation to introduce
knowledge of API functions and function calling trajectories to strengthen
intrinsic reasoning. To explore effective training protocols, we investigate
scaling laws to identify the optimal recipe in data mixing ratios. By continual
pre-training on Hephaestus-Forge, Hephaestus outperforms small- to medium-scale
open-source LLMs and rivals commercial LLMs on three agent benchmarks,
demonstrating the effectiveness of our pre-training corpus in enhancing
fundamental agentic capabilities and generalization of LLMs to new tasks or
environments.
Scaling Pre-training to One Hundred Billion Data for Vision Language
  Models
Authors: name: Xiao Wang, name: Ibrahim Alabdulmohsin, name: Daniel Salz, name: Zhe Li, name: Keran Rong, name: Xiaohua Zhai
Published: 2025-02-11 15:05:33

We provide an empirical investigation of the potential of pre-training
vision-language models on an unprecedented scale: 100 billion examples. We find
that model performance tends to saturate at this scale on many common
Western-centric classification and retrieval benchmarks, such as COCO Captions.
Nevertheless, tasks of cultural diversity achieve more substantial gains from
the 100-billion scale web data, thanks to its coverage of long-tail concepts.
Furthermore, we analyze the model's multilinguality and show gains in
low-resource languages as well. In addition, we observe that reducing the size
of the pretraining dataset via quality filters like using CLIP, typically used
to enhance performance, may inadvertently reduce the cultural diversity
represented even in large-scale datasets. Our results highlight that while
traditional benchmarks may not benefit significantly from scaling noisy, raw
web data to 100 billion examples, this data scale is vital for building truly
inclusive multimodal systems.
CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction
Authors: name: Junlong Li, name: Daya Guo, name: Dejian Yang, name: Runxin Xu, name: Yu Wu, name: Junxian He
Published: 2025-02-11 07:26:50

Reasoning is a fundamental capability of Large Language Models. While prior
research predominantly focuses on enhancing narrow skills like math or code
generation, improving performance on many other reasoning tasks remains
challenging due to sparse and fragmented training data. To address this issue,
we propose CodeI/O, a novel approach that systematically condenses diverse
reasoning patterns inherently embedded in contextually-grounded codes, through
transforming the original code into a code input-output prediction format. By
training models to predict inputs/outputs given code and test cases entirely in
natural language as Chain-of-Thought (CoT) rationales, we expose them to
universal reasoning primitives -- like logic flow planning, state-space
searching, decision tree traversal, and modular decomposition -- while
decoupling structured reasoning from code-specific syntax and preserving
procedural rigor. Experimental results demonstrate CodeI/O leads to consistent
improvements across symbolic, scientific, logic, math & numerical, and
commonsense reasoning tasks. By matching the existing ground-truth outputs or
re-executing the code with predicted inputs, we can verify each prediction and
further enhance the CoTs through multi-turn revision, resulting in CodeI/O++
and achieving higher performance. Our data and models are available at
https://github.com/hkust-nlp/CodeIO.
LLMs Can Easily Learn to Reason from Demonstrations Structure, not
  content, is what matters!
Authors: name: Dacheng Li, name: Shiyi Cao, name: Tyler Griggs, name: Shu Liu, name: Xiangxi Mo, name: Shishir G. Patil, name: Matei Zaharia, name: Joseph E. Gonzalez, name: Ion Stoica
Published: 2025-02-11 08:48:48

Large reasoning models (LRMs) tackle complex reasoning problems by following
long chain-of-thoughts (Long CoT) that incorporate reflection, backtracking,
and self-validation. However, the training techniques and data requirements to
elicit Long CoT remain poorly understood. In this work, we find that a Large
Language model (LLM) can effectively learn Long CoT reasoning through
data-efficient supervised fine-tuning (SFT) and parameter-efficient low-rank
adaptation (LoRA). With just 17k long CoT training samples, the
Qwen2.5-32B-Instruct model achieves significant improvements on a wide range of
math and coding benchmarks, including 56.7% (+40.0%) on AIME 2024 and 57.0%
(+8.1%) on LiveCodeBench, competitive to the proprietary o1-preview model's
score of 44.6% and 59.1%. More importantly, we find that the structure of Long
CoT is critical to the learning process, whereas the content of individual
reasoning steps has minimal impact. Perturbations affecting content, such as
training on incorrect samples or removing reasoning keywords, have little
impact on performance. In contrast, structural modifications that disrupt
logical consistency in the Long CoT, such as shuffling or deleting reasoning
steps, significantly degrade accuracy. For example, a model trained on Long CoT
samples with incorrect answers still achieves only 3.2% lower accuracy compared
to training with fully correct samples. These insights deepen our understanding
of how to elicit reasoning capabilities in LLMs and highlight key
considerations for efficiently training the next generation of reasoning
models. This is the academic paper of our previous released Sky-T1-32B-Preview
model. Codes are available at https://github.com/NovaSky-AI/SkyThought.
Competitive Programming with Large Reasoning Models
Authors: name: OpenAI, name: Ahmed El-Kishky, name: Alexander Wei, name: Andre Saraiva, name: Borys Minaev, name: Daniel Selsam, name: David Dohan, name: Francis Song, name: Hunter Lightman, name: Ignasi Clavera, name: Jakub Pachocki, name: Jerry Tworek, name: Lorenz Kuhn, name: Lukasz Kaiser, name: Mark Chen, name: Max Schwarzer, name: Mostafa Rohaninejad, name: Nat McAleese, name: o3 contributors, name: Oleg Mürk, name: Rhythm Garg, name: Rui Shu, name: Szymon Sidor, name: Vineet Kosaraju, name: Wenda Zhou
Published: 2025-02-03 23:00:15

We show that reinforcement learning applied to large language models (LLMs)
significantly boosts performance on complex coding and reasoning tasks.
Additionally, we compare two general-purpose reasoning models - OpenAI o1 and
an early checkpoint of o3 - with a domain-specific system, o1-ioi, which uses
hand-engineered inference strategies designed for competing in the 2024
International Olympiad in Informatics (IOI). We competed live at IOI 2024 with
o1-ioi and, using hand-crafted test-time strategies, placed in the 49th
percentile. Under relaxed competition constraints, o1-ioi achieved a gold
medal. However, when evaluating later models such as o3, we find that o3
achieves gold without hand-crafted domain-specific strategies or relaxed
constraints. Our findings show that although specialized pipelines such as
o1-ioi yield solid improvements, the scaled-up, general-purpose o3 model
surpasses those results without relying on hand-crafted inference heuristics.
Notably, o3 achieves a gold medal at the 2024 IOI and obtains a Codeforces
rating on par with elite human competitors. Overall, these results indicate
that scaling general-purpose reinforcement learning, rather than relying on
domain-specific techniques, offers a robust path toward state-of-the-art AI in
reasoning domains, such as competitive programming.
Forbidden Science: Dual-Use AI Challenge Benchmark and Scientific
  Refusal Tests
Authors: name: David Noever, name: Forrest McKee
Published: 2025-02-08 04:27:33

The development of robust safety benchmarks for large language models
requires open, reproducible datasets that can measure both appropriate refusal
of harmful content and potential over-restriction of legitimate scientific
discourse. We present an open-source dataset and testing framework for
evaluating LLM safety mechanisms across mainly controlled substance queries,
analyzing four major models' responses to systematically varied prompts. Our
results reveal distinct safety profiles: Claude-3.5-sonnet demonstrated the
most conservative approach with 73% refusals and 27% allowances, while Mistral
attempted to answer 100% of queries. GPT-3.5-turbo showed moderate restriction
with 10% refusals and 90% allowances, and Grok-2 registered 20% refusals and
80% allowances. Testing prompt variation strategies revealed decreasing
response consistency, from 85% with single prompts to 65% with five variations.
This publicly available benchmark enables systematic evaluation of the critical
balance between necessary safety restrictions and potential over-censorship of
legitimate scientific inquiry, while providing a foundation for measuring
progress in AI safety implementation. Chain-of-thought analysis reveals
potential vulnerabilities in safety mechanisms, highlighting the complexity of
implementing robust safeguards without unduly restricting desirable and valid
scientific discourse.