大型语言模型列表

大型语言模型（LLM）是一种机器学习模型，专为语言生成等自然语言处理任务而设计。LLM 是具有许多参数的语言模型，并通过对大量文本进行自监督学习进行训练。

本页列出了值得注意的大型语言模型。

对于训练成本一列，1 petaFLOP-day = 1 petaFLOP/sec × 1 天 = 8.6419 FLOP。此外，仅列出最大模型的成本。

名称｜发布日期｜开发者｜参数量 (十亿)｜语料库大小｜训练成本 (petaFLOP-day)｜许可证｜注解

Attention Is All You Need｜瓦斯瓦尼等人在Google发表｜0.213｜3600万个英语-法语句子对｜0.09｜在8个NVIDIA P100 GPU上训练了30万步。训练和评估代码根据Apache 2.0许可证发布。

GPT-1｜OpenAI｜0.117｜1｜MIT｜首个GPT模型，为仅解码器transformer。在8个P600GPU上训练了30天。

BERT｜Google｜0.340｜33亿单词｜9｜Apache 2.0｜这是一个早期且有影响力的语言模型。是仅编码器模型，因此并非为提示或生成而构建。在 64个TPUv2芯片上训练耗时4天。

T5 (语言模型)｜Google｜11｜340亿 tokens｜Apache 2.0｜许多Google项目的基础模型，例如Imagen。

XLNet｜Google｜0.340｜330亿单词｜330｜Apache 2.0｜作为BERT的替代，设计为仅编码器。在512个TPU v3芯片上训练了5.5天。

GPT-2｜OpenAI｜1.5｜40 GB (~100亿 tokens)｜28｜MIT｜在32个TPU v3芯片上训练了一周。

GPT-3｜OpenAI｜175｜3000亿 tokens｜3640｜专有｜2022年，GPT-3的一个经过微调的变体，称为 GPT-3.5，通过名为ChatGPT的网络界面向公众开放。

GPT-Neo｜EleutherAI｜2.7｜825 GiB｜MIT｜The first of a series of free GPT-3 alternatives released by EleutherAI. GPT-Neo outperformed an equivalent-size GPT-3 model on some benchmarks, but was significantly worse than the largest GPT-3.

GPT-J｜EleutherAI｜6｜825 GiB｜200｜Apache 2.0｜GPT-3-style language model

Megatron-Turing NLG｜Microsoft and Nvidia｜530｜338.6 billion tokens｜38000｜Restricted web access｜Trained for 3 months on over 2000 A100 GPUs on the NVIDIA Selene Supercomputer, for over 3 million GPU-hours.

Ernie 3.0 Titan｜Baidu｜260｜4 Tb｜专有｜Chinese-language LLM. Ernie Bot is based on this model.

Claude｜Anthropic｜52｜400 billion tokens｜beta｜Fine-tuned for desirable behavior in conversations.

GLaM (Generalist Language Model)｜Google｜1200｜1.6 trillion tokens｜5600｜专有｜Sparse mixture of experts model, making it more expensive to train but cheaper to run inference compared to GPT-3.

Gopher｜DeepMind｜280｜300 billion tokens｜5833｜专有｜Later developed into the Chinchilla model.

LaMDA (Language Models for Dialog Applications)｜Google｜137｜1.56T words, 168 billion tokens｜4110｜专有｜Specialized for response generation in conversations.

GPT-NeoX｜EleutherAI｜20｜825 GiB｜740｜Apache 2.0｜based on the Megatron architecture

Chinchilla｜DeepMind｜70｜1.4 trillion tokens｜6805｜专有｜Reduced-parameter model trained on more data. Used in the Sparrow bot. Often cited for its neural scaling law.

PaLM（路径语言模型）｜Google｜540｜768 billion tokens｜29,250｜专有｜Trained for ~60 days on ~6000 TPU v4 chips. , it is the largest dense Transformer published.

OPT (Open Pretrained Transformer)｜Meta｜175｜180 billion tokens｜310｜Non-commercial research｜GPT-3 architecture with some adaptations from Megatron. Uniquely, the training logbook written by the team was published.

YaLM 100B｜Yandex｜100｜1.7TB｜Apache 2.0｜English-Russian model based on Microsoft's Megatron-LM.

Minerva｜Google｜540｜38.5B tokens from webpages filtered for mathematical content and from papers submitted to the arXiv preprint server｜专有｜For solving "mathematical and scientific questions using step-by-step reasoning". Initialized from PaLM models, then finetuned on mathematical and scientific data.

BLOOM｜Large collaboration led by Hugging Face｜175｜350 billion tokens (1.6TB)｜Responsible AI｜Essentially GPT-3 but trained on a multi-lingual corpus (30% English excluding programming languages)

Galactica｜Meta｜120｜106 billion tokens｜未知｜CC-BY-NC-4.0｜Trained on scientific text and modalities.

AlexaTM (Teacher Models)｜Amazon｜20｜1.3 trillion｜专有｜bidirectional sequence-to-sequence architecture

LLaMA (Large Language Model Meta AI)｜Meta AI｜65｜1.4 trillion｜6300｜Non-commercial research｜Corpus has 20 languages. "Overtrained" (compared to Chinchilla scaling law) for better performance with fewer parameters.

GPT-4｜OpenAI｜未知 (According to rumors: 1760)｜未知｜未知｜专有｜Available for ChatGPT Plus users and used in several products.

Chameleon｜Meta AI｜34｜4.4 trillion

Cerebras-GPT｜Cerebras｜13｜270｜Apache 2.0｜Trained with Chinchilla formula.

Falcon｜Technology Innovation Institute｜40｜1 trillion tokens, from RefinedWeb (filtered web text corpus) plus some "curated corpora".｜2800｜Apache 2.0

BloombergGPT｜Bloomberg L.P.｜50｜363 billion token dataset based on Bloomberg's data sources, plus 345 billion tokens from general purpose datasets｜专有｜Trained on financial data from proprietary sources, for financial tasks.

PanGu-Σ｜Huawei｜1085｜329 billion tokens｜专有

OpenAssistant｜LAION｜17｜1.5 trillion tokens｜Apache 2.0｜Trained on crowdsourced open data

Jurassic-2｜AI21 Labs｜未知｜未知｜专有｜Multilingual

PaLM 2（路径语言模型2）｜Google｜340｜3.6 trillion tokens｜85,000｜专有｜Was used in Bard chatbot.

Llama 2｜Meta AI｜70｜2 trillion tokens｜21,000｜Llama 2 license｜1.7 million A100-hours.

Claude 2｜Anthropic｜未知｜未知｜未知｜专有｜Used in Claude chatbot.

Granite 13b｜IBM｜未知｜未知｜未知｜专有｜Used in IBM Watsonx.

Mistral 7B｜Mistral AI｜7.3｜未知｜Apache 2.0

Claude 2.1｜Anthropic｜未知｜未知｜未知｜专有｜Used in Claude chatbot. Has a context window of 200,000 tokens, or ~500 pages.

Grok-1｜xAI｜314｜未知｜未知｜Apache 2.0｜Used in Grok chatbot. Grok-1 has a context length of 8,192 tokens and has access to X (Twitter).

Gemini 1.0｜Google DeepMind｜未知｜未知｜未知｜专有｜Multimodal model, comes in three sizes. Used in the chatbot of the same name.

Mixtral 8x7B｜Mistral AI｜46.7｜未知｜未知｜Apache 2.0｜Outperforms GPT-3.5 and Llama 2 70B on many benchmarks. Mixture of experts model, with 12.9 billion parameters activated per token.

Mixtral 8x22B｜Mistral AI｜141｜未知｜未知｜Apache 2.0

DeepSeek LLM｜DeepSeek｜67｜2T tokens｜12,000｜DeepSeek License｜Trained on English and Chinese text. 1e24 FLOPs for 67B. 1e23 FLOPs for 7B

Phi-2｜Microsoft｜2.7｜1.4T tokens｜419｜MIT｜Trained on real and synthetic "textbook-quality" data, for 14 days on 96 A100 GPUs.

Gemini 1.5｜Google DeepMind｜未知｜未知｜未知｜专有｜Multimodal model, based on a Mixture-of-Experts (MoE) architecture. Context window above 1 million tokens.

Gemini Ultra｜Google DeepMind｜未知｜未知｜未知

Gemma｜Google DeepMind｜7｜6T tokens｜未知｜Gemma Terms of Use

Claude 3｜Anthropic｜未知｜未知｜未知｜专有｜Includes three models, Haiku, Sonnet, and Opus.

Nova｜ Rubik's AI｜未知｜未知｜未知｜专有｜Includes three models, Nova-Instant, Nova-Air, and Nova-Pro.

DBRX｜Databricks与Mosaic ML｜136｜12T Tokens｜Databricks Open Model License｜Training cost 10 million USD.

Fugaku-LLM｜富士通与东京工业大学等｜13｜380B Tokens｜The largest model ever trained on CPU-only, on the Fugaku.

Phi-3｜Microsoft｜14｜4.8T Tokens｜MIT｜Microsoft markets them as "small language model".

Granite Code Models｜IBM｜未知｜未知｜未知｜Apache 2.0

Qwen2｜阿里云｜72｜3T Tokens｜未知｜Qwen License｜Multiple sizes, the smallest being 0.5B.

DeepSeek V2｜DeepSeek｜236｜8.1T tokens｜28,000｜DeepSeek License｜1.4M hours on H800.

Nemotron-4｜Nvidia｜340｜9T Tokens｜200,000｜NVIDIA Open Model License｜Trained for 1 epoch. Trained on 6144 H100 GPUs between December 2023 and May 2024.

Llama 3.1｜Meta AI｜405｜15.6T tokens｜440,000｜Llama 3 license｜405B version took 31 million hours on H100-80GB, at 3.8E25 FLOPs.

DeepSeek V3｜DeepSeek｜671｜14.8T tokens｜56,000｜DeepSeek License｜在H800 GPU上训练278.8万小时。

Amazon Nova｜Amazon｜未知｜未知｜未知｜专有｜Includes three models, Nova Micro, Nova Lite, and Nova Pro

DeepSeek R1｜DeepSeek｜671｜未知｜未知｜MIT｜无预训练，基于V3-Base强化学习。

Qwen2.5｜Alibaba｜72｜18T tokens｜未知｜Qwen License

MiniMax-Text-01｜January 2025｜Minimax｜456｜4.7T tokens｜未知｜Minimax Model license

Gemini 2.0｜Google DeepMind｜未知｜未知｜未知｜专有｜Three models released: Flash, Flash-Lite and Pro

Mistral Large｜Mistral AI｜123｜未知｜未知｜Mistral Research License｜Upgraded over time. The latest version is 24.11.

Pixtral｜Mistral AI｜123｜未知｜未知｜Mistral Research License｜多模态模型。此外还有一个采用 Apache 2.0 协议开源的 12B 版本。

Grok 3｜xAI｜未知｜未知｜未知,estimated 5,800,000.｜专有｜训练成本据称达到了此前最先进模型计算量的 10 倍。”

Llama 4｜Meta AI｜400｜40T tokens｜Llama 4 license

Qwen3｜阿里云｜235｜36T tokens｜未知｜Apache 2.0｜多种尺寸可选，最小的版本为 0.6B（6亿参数）。

GPT-OSS｜OpenAI｜117｜未知｜未知｜Apache 2.0｜有20B和120B两种模型大小发布。

Claude 4.1｜Anthropic｜未知｜未知｜未知｜专有｜Includes one model, Opus.

GPT-5｜OpenAI｜未知｜未知｜未知｜专有｜包括三个模型GPT-5，GPT-5 mini，和GPT-5 nano。GPT-5可在ChatGPT及其API中使用，包含思考能力。

DeepSeek-V3.1｜August 21, 2025｜DeepSeek｜671｜15.639T｜MIT｜训练大小：14.8T tokens, of DeepSeek V3 plus 839B tokens from the extension phases (630B + 209B)这是一个可在思考和非思考模式间切换的混合模型。

Apertus｜ETH Zurich and EPF Lausanne｜70｜15 trillion｜未知｜Apache 2.0｜据称这是首个符合欧盟《人工智能法案》的LLM。

Claude 4.5｜Anthropic｜未知｜未知｜未知｜专有

DeepSeek-V3.2-Exp｜DeepSeek｜685｜MIT｜该实验性模型基于v3.1-Terminus构建，使用名为 DeepSeek Sparse Attention (DSA) 的自定义高效机制。

GLM-4.6｜智谱｜357｜Apache 2.0

Kimi K2 Thinking｜Moonshot AI｜1000｜MIT

GPT-5.1｜OpenAI｜专有

Grok 4.1｜xAI｜专有

Gemini 3｜Google DeepMind｜专有

Claude Opus 4.5｜Anthropic｜专有

DeepSeek-V3.2｜DeepSeek｜685｜MIT｜平衡推理能力与输出长度，适合日常使用场景如问答和通用Agent任务

DeepSeek-V3.2-Speciale｜DeepSeek｜685｜MIT｜将开源模型的推理能力推向极致，探索模型能力边界；但是仅供研究使用，不支持工具调用

Ministral-3｜Mistral｜8｜Apache 2.0

Mistral-Large-3｜Mistral｜675｜Apache 2.0