大型語言模型列表

大型語言模型（LLM）是一種機器學習模型，專為語言生成等自然語言處理任務而設計。LLM 是具有許多參數的語言模型，並通過對大量文本進行自監督學習進行訓練。

本頁列出了值得注意的大型語言模型。

對於訓練成本一列，1 petaFLOP-day = 1 petaFLOP/sec × 1 天 = 8.6419 FLOP。此外，僅列出最大模型的成本。

名稱｜發布日期｜開發者｜參數量 (十億)｜語料庫大小｜訓練成本 (petaFLOP-day)｜許可證｜註解

Attention Is All You Need｜瓦斯瓦尼等人在Google發表｜0.213｜3600萬個英語-法語句子對｜0.09｜在8個NVIDIA P100 GPU上訓練了30萬步。訓練和評估代碼根據Apache 2.0許可證發布。

GPT-1｜OpenAI｜0.117｜1｜MIT｜首個GPT模型，為僅解碼器transformer。在8個P600GPU上訓練了30天。

BERT｜Google｜0.340｜33億單詞｜9｜Apache 2.0｜這是一個早期且有影響力的語言模型。是僅編碼器模型，因此並非為提示或生成而構建。在 64個TPUv2晶片上訓練耗時4天。

T5 (語言模型)｜Google｜11｜340億 tokens｜Apache 2.0｜許多Google項目的基礎模型，例如Imagen。

XLNet｜Google｜0.340｜330億單詞｜330｜Apache 2.0｜作為BERT的替代，設計為僅編碼器。在512個TPU v3晶片上訓練了5.5天。

GPT-2｜OpenAI｜1.5｜40 GB (~100億 tokens)｜28｜MIT｜在32個TPU v3晶片上訓練了一周。

GPT-3｜OpenAI｜175｜3000億 tokens｜3640｜專有｜2022年，GPT-3的一個經過微調的變體，稱為 GPT-3.5，通過名為ChatGPT的網路界面向公眾開放。

GPT-Neo｜EleutherAI｜2.7｜825 GiB｜MIT｜The first of a series of free GPT-3 alternatives released by EleutherAI. GPT-Neo outperformed an equivalent-size GPT-3 model on some benchmarks, but was significantly worse than the largest GPT-3.

GPT-J｜EleutherAI｜6｜825 GiB｜200｜Apache 2.0｜GPT-3-style language model

Megatron-Turing NLG｜Microsoft and Nvidia｜530｜338.6 billion tokens｜38000｜Restricted web access｜Trained for 3 months on over 2000 A100 GPUs on the NVIDIA Selene Supercomputer, for over 3 million GPU-hours.

Ernie 3.0 Titan｜Baidu｜260｜4 Tb｜專有｜Chinese-language LLM. Ernie Bot is based on this model.

Claude｜Anthropic｜52｜400 billion tokens｜beta｜Fine-tuned for desirable behavior in conversations.

GLaM (Generalist Language Model)｜Google｜1200｜1.6 trillion tokens｜5600｜專有｜Sparse mixture of experts model, making it more expensive to train but cheaper to run inference compared to GPT-3.

Gopher｜DeepMind｜280｜300 billion tokens｜5833｜專有｜Later developed into the Chinchilla model.

LaMDA (Language Models for Dialog Applications)｜Google｜137｜1.56T words, 168 billion tokens｜4110｜專有｜Specialized for response generation in conversations.

GPT-NeoX｜EleutherAI｜20｜825 GiB｜740｜Apache 2.0｜based on the Megatron architecture

Chinchilla｜DeepMind｜70｜1.4 trillion tokens｜6805｜專有｜Reduced-parameter model trained on more data. Used in the Sparrow bot. Often cited for its neural scaling law.

PaLM（路徑語言模型）｜Google｜540｜768 billion tokens｜29,250｜專有｜Trained for ~60 days on ~6000 TPU v4 chips. , it is the largest dense Transformer published.

OPT (Open Pretrained Transformer)｜Meta｜175｜180 billion tokens｜310｜Non-commercial research｜GPT-3 architecture with some adaptations from Megatron. Uniquely, the training logbook written by the team was published.

YaLM 100B｜Yandex｜100｜1.7TB｜Apache 2.0｜English-Russian model based on Microsoft's Megatron-LM.

Minerva｜Google｜540｜38.5B tokens from webpages filtered for mathematical content and from papers submitted to the arXiv preprint server｜專有｜For solving "mathematical and scientific questions using step-by-step reasoning". Initialized from PaLM models, then finetuned on mathematical and scientific data.

BLOOM｜Large collaboration led by Hugging Face｜175｜350 billion tokens (1.6TB)｜Responsible AI｜Essentially GPT-3 but trained on a multi-lingual corpus (30% English excluding programming languages)

Galactica｜Meta｜120｜106 billion tokens｜未知｜CC-BY-NC-4.0｜Trained on scientific text and modalities.

AlexaTM (Teacher Models)｜Amazon｜20｜1.3 trillion｜專有｜bidirectional sequence-to-sequence architecture

LLaMA (Large Language Model Meta AI)｜Meta AI｜65｜1.4 trillion｜6300｜Non-commercial research｜Corpus has 20 languages. "Overtrained" (compared to Chinchilla scaling law) for better performance with fewer parameters.

GPT-4｜OpenAI｜未知 (According to rumors: 1760)｜未知｜未知｜專有｜Available for ChatGPT Plus users and used in several products.

Chameleon｜Meta AI｜34｜4.4 trillion

Cerebras-GPT｜Cerebras｜13｜270｜Apache 2.0｜Trained with Chinchilla formula.

Falcon｜Technology Innovation Institute｜40｜1 trillion tokens, from RefinedWeb (filtered web text corpus) plus some "curated corpora".｜2800｜Apache 2.0

BloombergGPT｜Bloomberg L.P.｜50｜363 billion token dataset based on Bloomberg's data sources, plus 345 billion tokens from general purpose datasets｜專有｜Trained on financial data from proprietary sources, for financial tasks.

PanGu-Σ｜Huawei｜1085｜329 billion tokens｜專有

OpenAssistant｜LAION｜17｜1.5 trillion tokens｜Apache 2.0｜Trained on crowdsourced open data

Jurassic-2｜AI21 Labs｜未知｜未知｜專有｜Multilingual

PaLM 2（路徑語言模型2）｜Google｜340｜3.6 trillion tokens｜85,000｜專有｜Was used in Bard chatbot.

Llama 2｜Meta AI｜70｜2 trillion tokens｜21,000｜Llama 2 license｜1.7 million A100-hours.

Claude 2｜Anthropic｜未知｜未知｜未知｜專有｜Used in Claude chatbot.

Granite 13b｜IBM｜未知｜未知｜未知｜專有｜Used in IBM Watsonx.

Mistral 7B｜Mistral AI｜7.3｜未知｜Apache 2.0

Claude 2.1｜Anthropic｜未知｜未知｜未知｜專有｜Used in Claude chatbot. Has a context window of 200,000 tokens, or ~500 pages.

Grok-1｜xAI｜314｜未知｜未知｜Apache 2.0｜Used in Grok chatbot. Grok-1 has a context length of 8,192 tokens and has access to X (Twitter).

Gemini 1.0｜Google DeepMind｜未知｜未知｜未知｜專有｜Multimodal model, comes in three sizes. Used in the chatbot of the same name.

Mixtral 8x7B｜Mistral AI｜46.7｜未知｜未知｜Apache 2.0｜Outperforms GPT-3.5 and Llama 2 70B on many benchmarks. Mixture of experts model, with 12.9 billion parameters activated per token.

Mixtral 8x22B｜Mistral AI｜141｜未知｜未知｜Apache 2.0

DeepSeek LLM｜DeepSeek｜67｜2T tokens｜12,000｜DeepSeek License｜Trained on English and Chinese text. 1e24 FLOPs for 67B. 1e23 FLOPs for 7B

Phi-2｜Microsoft｜2.7｜1.4T tokens｜419｜MIT｜Trained on real and synthetic "textbook-quality" data, for 14 days on 96 A100 GPUs.

Gemini 1.5｜Google DeepMind｜未知｜未知｜未知｜專有｜Multimodal model, based on a Mixture-of-Experts (MoE) architecture. Context window above 1 million tokens.

Gemini Ultra｜Google DeepMind｜未知｜未知｜未知

Gemma｜Google DeepMind｜7｜6T tokens｜未知｜Gemma Terms of Use

Claude 3｜Anthropic｜未知｜未知｜未知｜專有｜Includes three models, Haiku, Sonnet, and Opus.

Nova｜ Rubik's AI｜未知｜未知｜未知｜專有｜Includes three models, Nova-Instant, Nova-Air, and Nova-Pro.

DBRX｜Databricks與Mosaic ML｜136｜12T Tokens｜Databricks Open Model License｜Training cost 10 million USD.

Fugaku-LLM｜富士通與東京工業大學等｜13｜380B Tokens｜The largest model ever trained on CPU-only, on the Fugaku.

Phi-3｜Microsoft｜14｜4.8T Tokens｜MIT｜Microsoft markets them as "small language model".

Granite Code Models｜IBM｜未知｜未知｜未知｜Apache 2.0

Qwen2｜阿里雲｜72｜3T Tokens｜未知｜Qwen License｜Multiple sizes, the smallest being 0.5B.

DeepSeek V2｜DeepSeek｜236｜8.1T tokens｜28,000｜DeepSeek License｜1.4M hours on H800.

Nemotron-4｜Nvidia｜340｜9T Tokens｜200,000｜NVIDIA Open Model License｜Trained for 1 epoch. Trained on 6144 H100 GPUs between December 2023 and May 2024.

Llama 3.1｜Meta AI｜405｜15.6T tokens｜440,000｜Llama 3 license｜405B version took 31 million hours on H100-80GB, at 3.8E25 FLOPs.

DeepSeek V3｜DeepSeek｜671｜14.8T tokens｜56,000｜DeepSeek License｜在H800 GPU上訓練278.8萬小時。

Amazon Nova｜Amazon｜未知｜未知｜未知｜專有｜Includes three models, Nova Micro, Nova Lite, and Nova Pro

DeepSeek R1｜DeepSeek｜671｜未知｜未知｜MIT｜無預訓練，基於V3-Base強化學習。

Qwen2.5｜Alibaba｜72｜18T tokens｜未知｜Qwen License

MiniMax-Text-01｜January 2025｜Minimax｜456｜4.7T tokens｜未知｜Minimax Model license

Gemini 2.0｜Google DeepMind｜未知｜未知｜未知｜專有｜Three models released: Flash, Flash-Lite and Pro

Mistral Large｜Mistral AI｜123｜未知｜未知｜Mistral Research License｜Upgraded over time. The latest version is 24.11.

Pixtral｜Mistral AI｜123｜未知｜未知｜Mistral Research License｜多模態模型。此外還有一個採用 Apache 2.0 協議開源的 12B 版本。

Grok 3｜xAI｜未知｜未知｜未知,estimated 5,800,000.｜專有｜訓練成本據稱達到了此前最先進模型計算量的 10 倍。」

Llama 4｜Meta AI｜400｜40T tokens｜Llama 4 license

Qwen3｜阿里雲｜235｜36T tokens｜未知｜Apache 2.0｜多種尺寸可選，最小的版本為 0.6B（6億參數）。

GPT-OSS｜OpenAI｜117｜未知｜未知｜Apache 2.0｜有20B和120B兩種模型大小發布。

Claude 4.1｜Anthropic｜未知｜未知｜未知｜專有｜Includes one model, Opus.

GPT-5｜OpenAI｜未知｜未知｜未知｜專有｜包括三個模型GPT-5，GPT-5 mini，和GPT-5 nano。GPT-5可在ChatGPT及其API中使用，包含思考能力。

DeepSeek-V3.1｜August 21, 2025｜DeepSeek｜671｜15.639T｜MIT｜訓練大小：14.8T tokens, of DeepSeek V3 plus 839B tokens from the extension phases (630B + 209B)這是一個可在思考和非思考模式間切換的混合模型。

Apertus｜ETH Zurich and EPF Lausanne｜70｜15 trillion｜未知｜Apache 2.0｜據稱這是首個符合歐盟《人工智慧法案》的LLM。

Claude 4.5｜Anthropic｜未知｜未知｜未知｜專有

DeepSeek-V3.2-Exp｜DeepSeek｜685｜MIT｜該實驗性模型基於v3.1-Terminus構建，使用名為 DeepSeek Sparse Attention (DSA) 的自定義高效機制。

GLM-4.6｜智譜｜357｜Apache 2.0

Kimi K2 Thinking｜Moonshot AI｜1000｜MIT

GPT-5.1｜OpenAI｜專有

Grok 4.1｜xAI｜專有

Gemini 3｜Google DeepMind｜專有

Claude Opus 4.5｜Anthropic｜專有

DeepSeek-V3.2｜DeepSeek｜685｜MIT｜平衡推理能力與輸出長度，適合日常使用場景如問答和通用Agent任務

DeepSeek-V3.2-Speciale｜DeepSeek｜685｜MIT｜將開源模型的推理能力推向極致，探索模型能力邊界；但是僅供研究使用，不支持工具調用

Ministral-3｜Mistral｜8｜Apache 2.0

Mistral-Large-3｜Mistral｜675｜Apache 2.0