大型语言模型列表

大型语言模型（LLM）是一种机器学习模型，专为语言生成等自然语言处理任务而设计。LLM 是具有许多参数的语言模型，并通过对大量文本进行自监督学习进行训练。

本页列出了值得注意的大型语言模型。

对于训练成本一列，1 petaFLOP-day = 1 petaFLOP/sec × 1 天 = 8.64 FLOP。此外，仅列出最大模型的成本。

 名称  发布日期  开发者  参数量 (十亿)   语料库大小训练成本 (petaFLOP-day) 许可证  注解 Attention Is All You Need  瓦斯瓦尼等人在Google發表 0.213 3600萬個英語-法語句子對 0.09  在8個NVIDIA P100 GPU上訓練了30万步。訓練和評估代碼根據Apache 2.0許可證發布。 GPT-1    OpenAI    1  首个GPT模型，为仅解码器transformer。 在8个P600GPU上训练了30天。 BERT    Google    单词  这是一个早期且有影响力的语言模型。是仅编码器模型，因此并非为提示或生成而构建。 在 64个TPUv2芯片上训练耗时4天。Google340亿 tokens 许多Google项目的基础模型，例如Imagen。     Google   单词 330  作为BERT的替代，设计为仅编码器 。在512个TPU v3芯片上训练了5.5天。 GPT-2    OpenAI    40 GB (~ tokens) 28  在32个TPU v3芯片上训练了一周。 GPT-3    OpenAI     tokens3640Table D.1 in   2022年，GPT-3的一个经过微调的变体，称为 GPT-3.5，通过名为ChatGPT的网络界面向公众开放。 GPT-Neo        825 GiB   The first of a series of free GPT-3 alternatives released by EleutherAI. GPT-Neo outperformed an equivalent-size GPT-3 model on some benchmarks, but was significantly worse than the largest GPT-3.         825 GiB200  GPT-3-style language model Megatron-Turing NLG     Microsoft and Nvidia     tokens 38000  Trained for 3 months on over 2000 A100 GPUs on the NVIDIA Selene Supercomputer, for over 3 million GPU-hours. Ernie 3.0 Titan    Baidu    4 Tb   Chinese-language LLM. Ernie Bot is based on this model. Claude    Anthropic     tokens   Fine-tuned for desirable behavior in conversations. GLaM (Generalist Language Model)    Google     tokens 5600  Sparse mixture of experts model, making it more expensive to train but cheaper to run inference compared to GPT-3. Gopher    DeepMind     tokens5833Table 20 and page 66 of PaLM: Scaling Language Modeling with Pathways   Later developed into the Chinchilla model. LaMDA (Language Models for Dialog Applications)    Google    1.56T words,  tokens4110  Specialized for response generation in conversations. GPT-NeoX        825 GiB740  based on the Megatron architecture Chinchilla    DeepMind     tokens6805  Reduced-parameter model trained on more data. Used in the Sparrow bot. Often cited for its neural scaling law. PaLM（路徑語言模型）    Google     tokens  Trained for ~60 days on ~6000 TPU v4 chips. , it is the largest dense Transformer published. OPT (Open Pretrained Transformer)    Meta     tokens310  GPT-3 architecture with some adaptations from Megatron. Uniquely, the training logbook written by the team was published.YaLM 100B    Yandex   1.7TB     English-Russian model based on Microsoft's Megatron-LM. Minerva    Google    38.5B tokens from webpages filtered for mathematical content and from papers submitted to the arXiv preprint server   For solving "mathematical and scientific questions using step-by-step reasoning". Initialized from PaLM models, then finetuned on mathematical and scientific data. BLOOM    Large collaboration led by Hugging Face     tokens (1.6TB)   Essentially GPT-3 but trained on a multi-lingual corpus (30% English excluding programming languages) Galactica    Meta    tokens未知  Trained on scientific text and modalities. AlexaTM (Teacher Models)    Amazon       bidirectional sequence-to-sequence architecture LLaMA (Large Language Model Meta AI)    Meta AI   6300  Corpus has 20 languages. "Overtrained" (compared to Chinchilla scaling law) for better performance with fewer parameters. GPT-4    OpenAI  未知 (According to rumors: 1760) 未知 未知   Available for ChatGPT Plus users and used in several products. Chameleon    Meta AI    Cerebras-GPT270  Trained with Chinchilla formula. Falcon        1 trillion tokens, from RefinedWeb (filtered web text corpus) plus some "curated corpora".2800 UAE's Falcon 40B, World's Top-Ranked AI Model from Technology Innovation Institute, is Now Royalty-Free , 31 May 2023 BloombergGPT    Bloomberg L.P.   363 billion token dataset based on Bloomberg's data sources, plus 345 billion tokens from general purpose datasets   Trained on financial data from proprietary sources, for financial tasks. PanGu-Σ    Huawei   329 billion tokens   OpenAssistant       1.5 trillion tokens   Trained on crowdsourced open dataJurassic-2AI21 Labs未知未知  Multilingual PaLM 2（路徑語言模型2）    Google     tokens  Was used in Bard chatbot. Llama 2    Meta AI     tokens   1.7 million A100-hours.Claude 2Anthropic 未知未知未知  Used in Claude chatbot.Granite 13bIBM 未知未知未知  Used in IBM Watsonx. Mistral 7B    Mistral AI    未知  Claude 2.1Anthropic 未知未知未知  Used in Claude chatbot. Has a context window of 200,000 tokens, or ~500 pages.Grok-1xAI 314未知未知  Used in Grok chatbot. Grok-1 has a context length of 8,192 tokens and has access to X (Twitter).Gemini 1.0Google DeepMind 未知未知未知  Multimodal model, comes in three sizes. Used in the chatbot of the same name.Mixtral 8x7BMistral AI 46.7未知未知  Outperforms GPT-3.5 and Llama 2 70B on many benchmarks. Mixture of experts model, with 12.9 billion parameters activated per token.Mixtral 8x22BMistral AI 141未知未知  DeepSeek LLMDeepSeek672T tokensTrained on English and Chinese text. 1e24 FLOPs for 67B. 1e23 FLOPs for 7BPhi-2Microsoft 2.71.4T tokens419  Trained on real and synthetic "textbook-quality" data, for 14 days on 96 A100 GPUs.Gemini 1.5Google DeepMind 未知未知未知  Multimodal model, based on a Mixture-of-Experts (MoE) architecture. Context window above 1 million tokens.Gemini UltraGoogle DeepMind 未知未知未知 Gemma    Google DeepMind  7  6T tokens  未知   Claude 3Anthropic未知未知未知Includes three models, Haiku, Sonnet, and Opus.Nova Rubik's AI 未知未知 未知Includes three models, Nova-Instant, Nova-Air, and Nova-Pro.DBRX與Mosaic ML12T TokensTraining cost 10 million USD.Fugaku-LLM富士通與東京工業大學等380B TokensThe largest model ever trained on CPU-only, on the Fugaku.Phi-3Microsoft144.8T TokensMicrosoft markets them as "small language model". Granite Code ModelsIBM 未知未知未知 Qwen2阿里雲723T Tokens未知Multiple sizes, the smallest being 0.5B.DeepSeek V2DeepSeek2368.1T tokens1.4M hours on H800.Nemotron-4Nvidia9T Tokens Trained for 1 epoch. Trained on 6144 H100 GPUs between December 2023 and May 2024.Llama 3.1Meta AI40515.6T tokens 405B version took 31 million hours on H100-80GB, at 3.8E25 FLOPs.DeepSeek V3DeepSeek67114.8T tokens在H800 GPU上训练278.8万小时。Amazon NovaAmazon未知 未知 未知Includes three models, Nova Micro, Nova Lite, and Nova ProDeepSeek R1DeepSeek671未知未知无预训练，基于V3-Base强化学习。Qwen2.5Alibaba7218T tokens未知MiniMax-Text-01January 2025Minimax4564.7T tokens未知Gemini 2.0Google DeepMind 未知未知未知  Three models released: Flash, Flash-Lite and ProMistral LargeMistral AI 123  Upgraded over time. The latest version is 24.11.PixtralMistral AI 123  Multimodal. There is also a 12B version which is under Apache 2 license.Grok 3xAI   ,estimated 5,800,000.Training cost claimed "10x the compute of previous state-of-the-art models".Llama 4Meta AI  Qwen3阿里雲235 Multiple sizes, the smallest being 0.6B.GPT-OSSOpenAI117有20B和120B兩種模型大小發布。Claude 4.1Anthropic   Includes one model, Opus. GPT-5  OpenAI     包括三个模型GPT-5，GPT-5 mini，和GPT-5 nano。GPT-5可在ChatGPT及其API中使用，包含思考能力。DeepSeek-V3.1August 21, 2025DeepSeek67115.639T 训练大小：14.8T tokens, of DeepSeek V3 plus 839B tokens from the extension phases (630B + 209B)这是一个可在思考和非思考模式间切换的混合模型。ApertusETH Zurich and EPF Lausanne 70    据称这是首个符合欧盟《人工智能法案》的LLM。Claude 4.5Anthropic   DeepSeek-V3.2-ExpDeepSeek685  该实验性模型基于v3.1-Terminus构建，使用名为 DeepSeek Sparse Attention (DSA) 的自定义高效机制。GLM-4.6智谱357  Kimi K2 ThinkingMoonshot AI1000  GPT-5.1OpenAI  Grok 4.1xAI  Gemini 3Google DeepMind  Claude Opus 4.5Anthropic  DeepSeek-V3.2DeepSeek685  平衡推理能力与输出长度，适合日常使用场景如问答和通用Agent任务DeepSeek-V3.2-SpecialeDeepSeek685  将开源模型的推理能力推向极致，探索模型能力边界；但是仅供研究使用，不支持工具调用

 参见 
 聊天机器人列表

 注释 

 参考资料 

Category:大型语言模型