Orionnnn/hub-stats huggingface dataset:
A Multi-Modal AI Copilot for Single-Cell Analysis with Instruction
  Following
Authors: name: Yin Fang, name: Xinle Deng, name: Kangwei Liu, name: Ningyu Zhang, name: Jingyang Qian, name: Penghui Yang, name: Xiaohui Fan, name: Huajun Chen
Published: 2025-01-14 15:12:19

Large language models excel at interpreting complex natural language
instructions, enabling them to perform a wide range of tasks. In the life
sciences, single-cell RNA sequencing (scRNA-seq) data serves as the "language
of cellular biology", capturing intricate gene expression patterns at the
single-cell level. However, interacting with this "language" through
conventional tools is often inefficient and unintuitive, posing challenges for
researchers. To address these limitations, we present InstructCell, a
multi-modal AI copilot that leverages natural language as a medium for more
direct and flexible single-cell analysis. We construct a comprehensive
multi-modal instruction dataset that pairs text-based instructions with
scRNA-seq profiles from diverse tissues and species. Building on this, we
develop a multi-modal cell language architecture capable of simultaneously
interpreting and processing both modalities. InstructCell empowers researchers
to accomplish critical tasks-such as cell type annotation, conditional
pseudo-cell generation, and drug sensitivity prediction-using straightforward
natural language commands. Extensive evaluations demonstrate that InstructCell
consistently meets or exceeds the performance of existing single-cell
foundation models, while adapting to diverse experimental conditions. More
importantly, InstructCell provides an accessible and intuitive tool for
exploring complex single-cell data, lowering technical barriers and enabling
deeper biological insights.
MiniMax-01: Scaling Foundation Models with Lightning Attention
Authors: name: MiniMax, name: Aonian Li, name: Bangwei Gong, name: Bo Yang, name: Boji Shan, name: Chang Liu, name: Cheng Zhu, name: Chunhao Zhang, name: Congchao Guo, name: Da Chen, name: Dong Li, name: Enwei Jiao, name: Gengxin Li, name: Guojun Zhang, name: Haohai Sun, name: Houze Dong, name: Jiadai Zhu, name: Jiaqi Zhuang, name: Jiayuan Song, name: Jin Zhu, name: Jingtao Han, name: Jingyang Li, name: Junbin Xie, name: Junhao Xu, name: Junjie Yan, name: Kaishun Zhang, name: Kecheng Xiao, name: Kexi Kang, name: Le Han, name: Leyang Wang, name: Lianfei Yu, name: Liheng Feng, name: Lin Zheng, name: Linbo Chai, name: Long Xing, name: Meizhi Ju, name: Mingyuan Chi, name: Mozhi Zhang, name: Peikai Huang, name: Pengcheng Niu, name: Pengfei Li, name: Pengyu Zhao, name: Qi Yang, name: Qidi Xu, name: Qiexiang Wang, name: Qin Wang, name: Qiuhui Li, name: Ruitao Leng, name: Shengmin Shi, name: Shuqi Yu, name: Sichen Li, name: Songquan Zhu, name: Tao Huang, name: Tianrun Liang, name: Weigao Sun, name: Weixuan Sun, name: Weiyu Cheng, name: Wenkai Li, name: Xiangjun Song, name: Xiao Su, name: Xiaodong Han, name: Xinjie Zhang, name: Xinzhu Hou, name: Xu Min, name: Xun Zou, name: Xuyang Shen, name: Yan Gong, name: Yingjie Zhu, name: Yipeng Zhou, name: Yiran Zhong, name: Yongyi Hu, name: Yuanxiang Fan, name: Yue Yu, name: Yufeng Yang, name: Yuhao Li, name: Yunan Huang, name: Yunji Li, name: Yunpeng Huang, name: Yunzhi Xu, name: Yuxin Mao, name: Zehan Li, name: Zekang Li, name: Zewei Tao, name: Zewen Ying, name: Zhaoyang Cong, name: Zhen Qin, name: Zhenhua Fan, name: Zhihang Yu, name: Zhuo Jiang, name: Zijia Wu
Published: 2025-01-14 18:50:05

We introduce MiniMax-01 series, including MiniMax-Text-01 and MiniMax-VL-01,
which are comparable to top-tier models while offering superior capabilities in
processing longer contexts. The core lies in lightning attention and its
efficient scaling. To maximize computational capacity, we integrate it with
Mixture of Experts (MoE), creating a model with 32 experts and 456 billion
total parameters, of which 45.9 billion are activated for each token. We
develop an optimized parallel strategy and highly efficient
computation-communication overlap techniques for MoE and lightning attention.
This approach enables us to conduct efficient training and inference on models
with hundreds of billions of parameters across contexts spanning millions of
tokens. The context window of MiniMax-Text-01 can reach up to 1 million tokens
during training and extrapolate to 4 million tokens during inference at an
affordable cost. Our vision-language model, MiniMax-VL-01 is built through
continued training with 512 billion vision-language tokens. Experiments on both
standard and in-house benchmarks show that our models match the performance of
state-of-the-art models like GPT-4o and Claude-3.5-Sonnet while offering 20-32
times longer context window. We publicly release MiniMax-01 at
https://github.com/MiniMax-AI.
Evaluating Sample Utility for Data Selection by Mimicking Model Weights
Authors: name: Tzu-Heng Huang, name: Manjot Bilkhu, name: Frederic Sala, name: Javier Movellan
Published: 2025-01-12 04:28:14

Foundation models rely on large-scale web-crawled datasets, which frequently
contain noisy data, biases, and irrelevant content. Existing data selection
techniques typically use human heuristics, downstream evaluation datasets, or
specialized scoring models, and can overlook samples' utility in the training
process. Instead, we propose a new approach, Mimic Score, a data quality metric
that uses a pretrained reference model as a guide to assess the usefulness of
data samples for training a new model. It relies on the alignment between the
gradient of the new model parameters and the vector pointing toward the
reference model in weight space. Samples that misalign with this direction are
considered low-value and can be filtered out. Motivated by the Mimic score, we
develop Grad-Mimic, a data selection framework that identifies and prioritizes
useful samples, automating the selection process to create effective filters.
Empirically, using Mimic scores to guide model training results in consistent
performance gains across six image datasets and enhances the performance of
CLIP models. Moreover, Mimic scores and their associated filters improve upon
existing filtering methods and offer accurate estimation of dataset quality.
SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training
Authors: name: Tianjin Huang, name: Ziquan Zhu, name: Gaojie Jin, name: Lu Liu, name: Zhangyang Wang, name: Shiwei Liu
Published: 2025-01-12 15:21:22

Large Language Models (LLMs) have demonstrated exceptional performance across
diverse tasks, yet their training remains highly resource-intensive and
susceptible to critical challenges such as training instability. A predominant
source of this instability stems from gradient and loss spikes, which disrupt
the learning process, often leading to costly interventions like checkpoint
recovery and experiment restarts, further amplifying inefficiencies. This paper
presents a comprehensive investigation into gradient spikes observed during LLM
training, revealing their prevalence across multiple architectures and
datasets. Our analysis shows that these spikes can be up to 1000times larger
than typical gradients, substantially deteriorating model performance. To
address this issue, we propose Spike-Aware Adam with Momentum Reset SPAM, a
novel optimizer designed to counteract gradient spikes through momentum reset
and spike-aware gradient clipping. Extensive experiments, including both
pre-training and fine-tuning, demonstrate that SPAM consistently surpasses Adam
and its variants across various tasks, including (1) LLM pre-training from 60M
to 1B, (2) 4-bit LLM pre-training,(3) reinforcement learning, and (4) Time
Series Forecasting. Additionally, SPAM facilitates memory-efficient training by
enabling sparse momentum, where only a subset of momentum terms are maintained
and updated. When operating under memory constraints, SPAM outperforms
state-of-the-art memory-efficient optimizers such as GaLore and Adam-Mini. Our
work underscores the importance of mitigating gradient spikes in LLM training
and introduces an effective optimization strategy that enhances both training
stability and resource efficiency at scale. Code is available at
https://github.com/TianjinYellow/SPAM-Optimizer.git
BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and
  Vision-Language Models Derived from Scientific Literature
Authors: name: Alejandro Lozano, name: Min Woo Sun, name: James Burgess, name: Liangyu Chen, name: Jeffrey J Nirschl, name: Jeffrey Gu, name: Ivan Lopez, name: Josiah Aklilu, name: Austin Wolfgang Katzer, name: Collin Chiu, name: Anita Rau, name: Xiaohan Wang, name: Yuhui Zhang, name: Alfred Seunghoon Song, name: Robert Tibshirani, name: Serena Yeung-Levy
Published: 2025-01-13 09:58:03

The development of vision-language models (VLMs) is driven by large-scale and
diverse multimodal datasets. However, progress toward generalist biomedical
VLMs is limited by the lack of annotated, publicly accessible datasets across
biology and medicine. Existing efforts are restricted to narrow domains,
missing the full diversity of biomedical knowledge encoded in scientific
literature. To address this gap, we introduce BIOMEDICA, a scalable,
open-source framework to extract, annotate, and serialize the entirety of the
PubMed Central Open Access subset into an easy-to-use, publicly accessible
dataset.Our framework produces a comprehensive archive with over 24 million
unique image-text pairs from over 6 million articles. Metadata and
expert-guided annotations are also provided. We demonstrate the utility and
accessibility of our resource by releasing BMCA-CLIP, a suite of CLIP-style
models continuously pre-trained on the BIOMEDICA dataset via streaming,
eliminating the need to download 27 TB of data locally.On average, our models
achieve state-of-the-art performance across 40 tasks - spanning pathology,
radiology, ophthalmology, dermatology, surgery, molecular biology,
parasitology, and cell biology - excelling in zero-shot classification with a
6.56% average improvement (as high as 29.8% and 17.5% in dermatology and
ophthalmology, respectively), and stronger image-text retrieval, all while
using 10x less compute. To foster reproducibility and collaboration, we release
our codebase and dataset for the broader research community.
VideoAuteur: Towards Long Narrative Video Generation
Authors: name: Junfei Xiao, name: Feng Cheng, name: Lu Qi, name: Liangke Gui, name: Jiepeng Cen, name: Zhibei Ma, name: Alan Yuille, name: Lu Jiang
Published: 2025-01-10 18:52:11

Recent video generation models have shown promising results in producing
high-quality video clips lasting several seconds. However, these models face
challenges in generating long sequences that convey clear and informative
events, limiting their ability to support coherent narrations. In this paper,
we present a large-scale cooking video dataset designed to advance long-form
narrative generation in the cooking domain. We validate the quality of our
proposed dataset in terms of visual fidelity and textual caption accuracy using
state-of-the-art Vision-Language Models (VLMs) and video generation models,
respectively. We further introduce a Long Narrative Video Director to enhance
both visual and semantic coherence in generated videos and emphasize the role
of aligning visual embeddings to achieve improved overall video quality. Our
method demonstrates substantial improvements in generating visually detailed
and semantically aligned keyframes, supported by finetuning techniques that
integrate text and image embeddings within the video generation process.
Project page: https://videoauteur.github.io/
WebWalker: Benchmarking LLMs in Web Traversal
Authors: name: Jialong Wu, name: Wenbiao Yin, name: Yong Jiang, name: Zhenglin Wang, name: Zekun Xi, name: Runnan Fang, name: Deyu Zhou, name: Pengjun Xie, name: Fei Huang
Published: 2025-01-13 18:58:07

Retrieval-augmented generation (RAG) demonstrates remarkable performance
across tasks in open-domain question-answering. However, traditional search
engines may retrieve shallow content, limiting the ability of LLMs to handle
complex, multi-layered information. To address it, we introduce WebWalkerQA, a
benchmark designed to assess the ability of LLMs to perform web traversal. It
evaluates the capacity of LLMs to traverse a website's subpages to extract
high-quality data systematically. We propose WebWalker, which is a multi-agent
framework that mimics human-like web navigation through an explore-critic
paradigm. Extensive experimental results show that WebWalkerQA is challenging
and demonstrates the effectiveness of RAG combined with WebWalker, through the
horizontal and vertical integration in real-world scenarios.
ChemAgent: Self-updating Library in Large Language Models Improves
  Chemical Reasoning
Authors: name: Xiangru Tang, name: Tianyu Hu, name: Muyang Ye, name: Yanjun Shao, name: Xunjian Yin, name: Siru Ouyang, name: Wangchunshu Zhou, name: Pan Lu, name: Zhuosheng Zhang, name: Yilun Zhao, name: Arman Cohan, name: Mark Gerstein
Published: 2025-01-11 17:10:30

Chemical reasoning usually involves complex, multi-step processes that demand
precise calculations, where even minor errors can lead to cascading failures.
Furthermore, large language models (LLMs) encounter difficulties handling
domain-specific formulas, executing reasoning steps accurately, and integrating
code effectively when tackling chemical reasoning tasks. To address these
challenges, we present ChemAgent, a novel framework designed to improve the
performance of LLMs through a dynamic, self-updating library. This library is
developed by decomposing chemical tasks into sub-tasks and compiling these
sub-tasks into a structured collection that can be referenced for future
queries. Then, when presented with a new problem, ChemAgent retrieves and
refines pertinent information from the library, which we call memory,
facilitating effective task decomposition and the generation of solutions. Our
method designs three types of memory and a library-enhanced reasoning
component, enabling LLMs to improve over time through experience. Experimental
results on four chemical reasoning datasets from SciBench demonstrate that
ChemAgent achieves performance gains of up to 46% (GPT-4), significantly
outperforming existing methods. Our findings suggest substantial potential for
future applications, including tasks such as drug discovery and materials
science. Our code can be found at https://github.com/gersteinlab/chemagent
MinMo: A Multimodal Large Language Model for Seamless Voice Interaction
Authors: name: Qian Chen, name: Yafeng Chen, name: Yanni Chen, name: Mengzhe Chen, name: Yingda Chen, name: Chong Deng, name: Zhihao Du, name: Ruize Gao, name: Changfeng Gao, name: Zhifu Gao, name: Yabin Li, name: Xiang Lv, name: Jiaqing Liu, name: Haoneng Luo, name: Bin Ma, name: Chongjia Ni, name: Xian Shi, name: Jialong Tang, name: Hui Wang, name: Hao Wang, name: Wen Wang, name: Yuxuan Wang, name: Yunlan Xu, name: Fan Yu, name: Zhijie Yan, name: Yexin Yang, name: Baosong Yang, name: Xian Yang, name: Guanrou Yang, name: Tianyu Zhao, name: Qinglin Zhang, name: Shiliang Zhang, name: Nan Zhao, name: Pei Zhang, name: Chong Zhang, name: Jinren Zhou
Published: 2025-01-10 15:55:27

Recent advancements in large language models (LLMs) and multimodal
speech-text models have laid the groundwork for seamless voice interactions,
enabling real-time, natural, and human-like conversations. Previous models for
voice interactions are categorized as native and aligned. Native models
integrate speech and text processing in one framework but struggle with issues
like differing sequence lengths and insufficient pre-training. Aligned models
maintain text LLM capabilities but are often limited by small datasets and a
narrow focus on speech tasks. In this work, we introduce MinMo, a Multimodal
Large Language Model with approximately 8B parameters for seamless voice
interaction. We address the main limitations of prior aligned multimodal
models. We train MinMo through multiple stages of speech-to-text alignment,
text-to-speech alignment, speech-to-speech alignment, and duplex interaction
alignment, on 1.4 million hours of diverse speech data and a broad range of
speech tasks. After the multi-stage training, MinMo achieves state-of-the-art
performance across various benchmarks for voice comprehension and generation
while maintaining the capabilities of text LLMs, and also facilitates
full-duplex conversation, that is, simultaneous two-way communication between
the user and the system. Moreover, we propose a novel and simple voice decoder
that outperforms prior models in voice generation. The enhanced
instruction-following capabilities of MinMo supports controlling speech
generation based on user instructions, with various nuances including emotions,
dialects, and speaking rates, and mimicking specific voices. For MinMo, the
speech-to-text latency is approximately 100ms, full-duplex latency is
approximately 600ms in theory and 800ms in practice. The MinMo project web page
is https://funaudiollm.github.io/minmo, and the code and models will be
released soon.
Transformer^2: Self-adaptive LLMs
Authors: name: Qi Sun, name: Edoardo Cetin, name: Yujin Tang
Published: 2025-01-09 01:19:21

Self-adaptive large language models (LLMs) aim to solve the challenges posed
by traditional fine-tuning methods, which are often computationally intensive
and static in their ability to handle diverse tasks. We introduce \implname, a
novel self-adaptation framework that adapts LLMs for unseen tasks in real-time
by selectively adjusting only the singular components of their weight matrices.
During inference, \implname employs a two-pass mechanism: first, a dispatch
system identifies the task properties, and then task-specific "expert" vectors,
trained using reinforcement learning, are dynamically mixed to obtain targeted
behavior for the incoming prompt. Our method outperforms ubiquitous approaches
such as LoRA, with fewer parameters and greater efficiency. \implname
demonstrates versatility across different LLM architectures and modalities,
including vision-language tasks. \implname represents a significant leap
forward, offering a scalable, efficient solution for enhancing the adaptability
and task-specific performance of LLMs, paving the way for truly dynamic,
self-organizing AI systems.
The Lessons of Developing Process Reward Models in Mathematical
  Reasoning
Authors: name: Zhenru Zhang, name: Chujie Zheng, name: Yangzhen Wu, name: Beichen Zhang, name: Runji Lin, name: Bowen Yu, name: Dayiheng Liu, name: Jingren Zhou, name: Junyang Lin
Published: 2025-01-13 13:10:16

Process Reward Models (PRMs) emerge as a promising approach for process
supervision in mathematical reasoning of Large Language Models (LLMs), which
aim to identify and mitigate intermediate errors in the reasoning processes.
However, the development of effective PRMs faces significant challenges,
particularly in data annotation and evaluation methodologies. In this paper,
through extensive experiments, we demonstrate that commonly used Monte Carlo
(MC) estimation-based data synthesis for PRMs typically yields inferior
performance and generalization compared to LLM-as-a-judge and human annotation
methods. MC estimation relies on completion models to evaluate current-step
correctness, leading to inaccurate step verification. Furthermore, we identify
potential biases in conventional Best-of-N (BoN) evaluation strategies for
PRMs: (1) The unreliable policy models generate responses with correct answers
but flawed processes, leading to a misalignment between the evaluation criteria
of BoN and the PRM objectives of process verification. (2) The tolerance of
PRMs of such responses leads to inflated BoN scores. (3) Existing PRMs have a
significant proportion of minimum scores concentrated on the final answer
steps, revealing the shift from process to outcome-based assessment in BoN
Optimized PRMs. To address these challenges, we develop a consensus filtering
mechanism that effectively integrates MC estimation with LLM-as-a-judge and
advocates a more comprehensive evaluation framework that combines
response-level and step-level metrics. Based on the mechanisms, we
significantly improve both model performance and data efficiency in the BoN
evaluation and the step-wise error identification task. Finally, we release a
new state-of-the-art PRM that outperforms existing open-source alternatives and
provides practical guidelines for future research in building process
supervision models.
O1 Replication Journey -- Part 3: Inference-time Scaling for Medical
  Reasoning
Authors: name: Zhongzhen Huang, name: Gui Geng, name: Shengyi Hua, name: Zhen Huang, name: Haoyang Zou, name: Shaoting Zhang, name: Pengfei Liu, name: Xiaofan Zhang
Published: 2025-01-11 07:10:23

Building upon our previous investigations of O1 replication (Part 1: Journey
Learning [Qin et al., 2024] and Part 2: Distillation [Huang et al., 2024]),
this work explores the potential of inference-time scaling in large language
models (LLMs) for medical reasoning tasks, ranging from diagnostic
decision-making to treatment planning. Through extensive experiments on medical
benchmarks of varying complexity (MedQA, Medbullets, and JAMA Clinical
Challenges), our investigation reveals several key insights: (1) Increasing
inference time does lead to improved performance. With a modest training set of
500 samples, our model yields substantial performance improvements of 6%-11%.
(2) Task complexity directly correlates with the required length of reasoning
chains, confirming the necessity of extended thought processes for challenging
problems. (3) The differential diagnoses generated by our model adhere to the
principles of the hypothetico-deductive method, producing a list of potential
conditions that may explain a patient's symptoms and systematically narrowing
these possibilities by evaluating the evidence. These findings demonstrate the
promising synergy between inference-time scaling and journey learning in
advancing LLMs' real-world clinical reasoning capabilities.
Tensor Product Attention Is All You Need
Authors: name: Yifan Zhang, name: Yifeng Liu, name: Huizhuo Yuan, name: Zhen Qin, name: Yang Yuan, name: Quanquan Gu, name: Andrew Chi-Chih Yao
Published: 2025-01-11 03:37:10

Scaling language models to handle longer input sequences typically
necessitates large key-value (KV) caches, resulting in substantial memory
overhead during inference. In this paper, we propose Tensor Product Attention
(TPA), a novel attention mechanism that uses tensor decompositions to represent
queries, keys, and values compactly, significantly shrinking KV cache size at
inference time. By factorizing these representations into contextual low-rank
components (contextual factorization) and seamlessly integrating with RoPE, TPA
achieves improved model quality alongside memory efficiency. Based on TPA, we
introduce the Tensor ProducT ATTenTion Transformer (T6), a new model
architecture for sequence modeling. Through extensive empirical evaluation of
language modeling tasks, we demonstrate that T6 exceeds the performance of
standard Transformer baselines including MHA, MQA, GQA, and MLA across various
metrics, including perplexity and a range of renowned evaluation benchmarks.
Notably, TPAs memory efficiency enables the processing of significantly longer
sequences under fixed resource constraints, addressing a critical scalability
challenge in modern language models. The code is available at
https://github.com/tensorgi/T6.