Open LLM 一覧

Open LLM をまとめました。


2023年7月10日時点の情報です。ライセンスなどは改めて確認してください。
Release Dateに並べ替える。


| Language Model   | Developer                  | Release Date | Checkpoints                                             | Language                                                             | Params (B) | Tokens  | Context Length | Licence      | Hugging Face                                                |
|------------------|----------------------------|--------------|---------------------------------------------------------|----------------------------------------------------------------------|------------|---------|----------------|--------------|-------------------------------------------------------------|
| Xgen             | Salesforce                 | 2023/06      | XGen-7B-4K-Base, XGen-7B-8K-Base, XGen-7B-8K-Inst (for research only) | 22 languages: Bulgarian, Catalan, Czech, Danish, German, English, Spanish, French, Croatian, Hungarian, Italian, Dutch, Polish, Romanian, Russian, Slovenian, Serbian, Swedish, Ukrainian, Japanese, Chinese | 7          | 1.5T    | 4K to 8K       | Apache 2.0   | [Salesforce](https://huggingface.co/Salesforce)               |
| DLite            | AI Squared                 | 2023/05      | dlite-v2-1_5b                                           | English                                                              | 0.124 to 1.5 |         | 1024           | Apache 2.0   | [AI Squared](https://huggingface.co/aisquared)               |
| h2oGPT           | H2O LLM Studio             | 2023/05      | h2oGPT                                                  | English                                                              | 12 to 20   |         | 256 - 2048     | Apache 2.0   | [H2O LLM Studio](https://huggingface.co/h2oai)               |
| RedPajama-INCITE | Together and leaders        | 2023/05      | RedPajama-INCITE                                        | English                                                              | 3 to 7     |         | 2048           | Apache 2.0   | [Together and leaders](https://huggingface.co/togethercomputer) |
| OpenLLaMA        | OpenLM Research            | 2023/05      | open_llama_7b_700bt_preview, open_llama_3b_600bt_preview | English                                                              | 3, 7, 13   | 1T      | 2048           | Apache 2.0   | [OpenLM Research](https://huggingface.co/openlm-research)      |
| Falcon           | The Technology Innovation Institute (TII) | 2023/05 | Falcon-40B, Falcon-7B                                   | 11 languages, English, German, Spanish, French (and limited capabilities in Italian, Portuguese, Polish, Dutch, Romanian, Czech, Swedish) | 7, 40      | 1T      | 2048           | Apache 2.0   | [TII](https://huggingface.co/tiiuae)                          |
| MPT              | MosaicML                   | 2023/05      | MPT-7B, MPT-7B-Instruct, MPT-30B, MPT-30B-instruct       | English                                                              | 7, 30      | 1T      | 8192 to 65k    | Apache 2.0, CC BY-SA-3.0 | [MosaicML](https://huggingface.co/mosaicml)                    |
| Pythia           | EleutherAI                 | 2023/04      | 70M, 160M, 410M, 1B, 1.4B, 2.8B, 6.9B, 12B              | English                                                              | 0.07 to 12 |         | 2048           | Apache 2.0   | [EleutherAI](https://huggingface.co/EleutherAI)                |
| Dolly            | Databricks                 | 2023/04      | dolly-v2-3b, dolly-v2-7b, dolly-v2-12b                    | English                                                              | 3, 7, 12   |         | 2048           | MIT          | [Databricks](https://huggingface.co/databricks)               |
| StableLM-Alpha   | StableLM                   | 2023/04      | StableLM-Alpha                                          | English                                                              | 3 to 7     |         | 4096           | CC BY-SA-4.0 | [StableLM](https://huggingface.co/stabilityai)                |
| FastChat-T5      | lmsys                      | 2023/04      | fastchat-t5-3b-v1.0                                     | English                                                              | 3          |         | 512            | Apache 2.0   | [lmsys](https://huggingface.co/lmsys)                          |
| Cerebras-GPT     | Cerebras                   | 2023/03      | 111M, 256M, 590M, 1.3B, 2.7B, 6.7B, and 13B              | English                                                              | 0.111 to 13 |         | 2048           | Apache 2.0   | [Cerebras](https://huggingface.co/cerebras)                    |
| Open Assistant (Pythia family) | Open-Assistant    | 2023/03 | OA-Pythia-12B-SFT-8, OA-Pythia-12B-SFT-4, OA-Pythia-12B-SFT-1 | English                                                           | 12         |         | 2048           | Apache 2.0   | [Open-Assistant](https://huggingface.co/OpenAssistant)         |
| Bloom            | BigScience                 | 2022/11      | Bloom                                                   | 46 languages (No Japanese)                                           | 176        | 366B    | 2048           | OpenRAIL-M v1 | [BigScience](https://huggingface.co/bigscience)                |
| GPT-NeoX-20B     | EleutherAI                 | 2022/04      | GPT-NEOX-20B                                            | English                                                              | 20         |         | 2048           | Apache 2.0   | [EleutherAI](https://huggingface.co/EleutherAI)                |
| RWKV             | BlinkDL                    | 2021/08      | RWKV, ChatRWKV                                          | 100+ languages                                                       | 0.1 to 14  |         | infinity (RNN) | Apache 2.0   | [BlinkDL](https://huggingface.co/BlinkDL)                      |
| GPT-J-6B         | Eleuther AI               | 2021/06      | GPT-J-6B, GPT4All-J                                     | English                                                              | 6          |         | 2048           | Apache 2.0   | [EleutherAI](https://huggingface.co/EleutherAI)                |
| mT5              | Google                     | 2020/11      | mT5-Small, mT5-Base, mT5-Large, mT5-XL, mT5-XXL          | 101 (with Japanese)                                                 | 0.3 to 13  |         | 512            | Apache 2.0   | [Google](https://huggingface.co/google)                        |
| T5               | Google                     | 2019/10      | T5 & Flan-T5, Flan-T5-xxl (HF)                             | English                                                              | 0.06 to 11 |         | 512            | Apache 2.0   | [Google](https://huggingface.co/google)                        |