{{위키데이터 속성 추적}}

'''대형 언어 모델'''(大型言語 - , {{llang|en|large language model, '''LLM'''}}) 또는 '''거대 언어 모델'''(巨大言語 - )은 수많은 파라미터(보통 수십억 웨이트 이상)를 보유한 [[인공 신경망]]으로 구성되는 [[언어 모델]]이다. [[자기 지도 학습]]이나 반자기지도학습을 사용하여 레이블링되지 않은 상당한 양의 텍스트로 훈련된다.<ref>{{웹 인용|url=https://analyticsindiamag.com/self-supervised-learning-vs-semi-supervised-learning-how-they-differ/|title=Self-Supervised Learning Vs Semi-Supervised Learning: How They Differ|first=Shraddha|last=Goled|date=May 7, 2021|website=Analytics India Magazine}}</ref> LLM은 2018년 즈음에 모습을 드러냈으며 다양한 작업을 위해 수행된다. 이전의 특정 작업의 특수한 [[지도 학습]] 모델의 훈련 패러다임에서 벗어나 [[자연어 처리]] 연구로 초점이 옮겨졌다.

대규모 언어 모델(LLM) 은 [[인공지능|AI 챗봇]] 기술을 가능하게 하는 요소이며 많은 화제를 불러일으키고 있는 주제 중 하나다. 대규모 언어 모델(LLM)의 작동 방식은 크게 3가지로 나뉘고 있다. 토큰화, 트랜스포머 모델, 프롬프트 등. '''토큰화'''는 [[자연어 처리]]의 일부로 일반 인간 언어를 저수준 기계 시스템([[LMMS|LLMS]])이 이해할 수 있는 시퀀스로 변환하는 작업을 말하며 여기에는 섹션에 숫자 값을 할당하고 빠른 분석을 위해 인코딩하는 작업이 수반된다. 이는 음성학의 AI 버전과 같으며 토큰화의 목적은 인공지능이 문장의 구조를 예측하기 위한 학습 가이드 또는 공식과 같은 컨텍스트 백터를 생성하는 것이 목적. 언어를 더 많이 연구하고 문장이 어떻게 구성되는지 이해할수록 특정 유형의 문장에서 다음 언어에 대한 예측이 더 정확해진다. 이로 인해 온라인에서 사람들이 사용하는 다양한 커뮤니케이션 스타일을 재현하는 모델을 개발할 수 있다.

'''트랜스포머 모델'''은 순차적 데이터를 검사하여 어떤 단어가 서로 뒤따를 가능성이 높은지 관련 패턴을 식별하는 신경망의 일종으로 각각 다른 분석을 수행하여 어떤 단어가 호환되는지 결정하는 계층으로 구성된다. 이러한 모델은 언어를 학습하지 않고 알고리즘에 의존하여 사람이 쓴 단어를 이해하고 예를 들어, 힙스터 커피 블로그를 제공함으로써 커피에 대한 표준 글을 작성하도록 학습시킨다.

'''프롬프트'''는 개발자가 정보를 분석하고 토큰화하기 위해 대규모 언어 모델 LLM에 제공하는 정보로 프롬프트는 기본적으로 다양한 사용 사례에서 LLM에 도움이 되는 학습 데이터이다. 더 정확한 프롬프트를 받을수록 LLM은 다음 단어를 더 잘 예측하고 정확한 문장을 구성할 수 있다. 따라서 [[딥 러닝|딥러닝]] AI의 적절한 학습을 위해서는 적절한 프롬프트를 선택하는 것이 중요하다.

== 평가 ==
=== 퍼플렉시티 ===
[[퍼플렉시티]](perplexity)

=== 작업별 데이터 세트 및 벤치마크 ===
또한 더 구체적인 다운스트림 작업에서 언어 모델의 역량을 평가하기 위해 많은 수의 테스트 데이터 세트와 [[벤치마크 (컴퓨팅)|벤치마크]]가 개발되었다. 테스트는 일반 지식, [[상식적 추론]], 수학적 문제 해결을 포함한 다양한 역량을 평가하도록 설계될 수 있다.

== 대형 언어 모델 목록 ==
{| class="wikitable sortable"
|+ 대형 언어 모델 목록
|-
! 이름 !! 출시일{{efn|This is the date that documentation describing the model's architecture was first released.}} !! 개발 !! 파라미터 수{{efn|In many cases, researchers release or report on multiple versions of a model having different sizes. In these cases, the size of the largest model is listed here.}} !! 코퍼스 크기 !! 라이선스{{efn|This is the license of the pre-trained model weights. In almost all cases the training code itself is open-source or can be easily replicated.}}
|-
| [[BERT (언어 모델)|BERT]] || 2018년 || [[구글]] || {{sort|340000000|340 million}}<ref name="bert-paper">{{ArXiv 인용|last1=Devlin |first1=Jacob |last2=Chang |first2=Ming-Wei |last3=Lee |first3=Kenton |last4=Toutanova |first4=Kristina |title=BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding |date=11 October 2018 |eprint=1810.04805v2|class=cs.CL }}</ref> || {{sort|3300000000|3.3 billion}} words<ref name=bert-paper/> || {{yes|Apache 2.0}}<ref name=bert-web>{{웹 인용|url=https://github.com/google-research/bert|title=BERT|date=March 13, 2023|via=GitHub}}</ref>
|-
| XLNet || 2019년 || [[Google]] || {{sort|340000000|~340 million}}<ref>{{웹 인용|url=https://www.kdnuggets.com/bert-roberta-distilbert-xlnet-which-one-to-use.html|title=BERT, RoBERTa, DistilBERT, XLNet: Which one to use?}}{{깨진 링크|url=https://www.kdnuggets.com/bert-roberta-distilbert-xlnet-which-one-to-use.html }}</ref> || {{sort|3300000000|33 billion}} words ||
|-
| [[GPT-2]] || 2019년 || [[OpenAI]] || {{sort|1500000000|1.5 billion}}<ref name="15Brelease">{{웹 인용|url          = https://openai.com/blog/gpt-2-1-5b-release/ |title        = GPT-2: 1.5B Release |date         = 2019-11-05 |website      = OpenAI |language     = en |access-date  = 2019-11-14 |archive-date = 2019-11-14 |archive-url  = https://web.archive.org/web/20191114074358/https://openai.com/blog/gpt-2-1-5b-release/ |url-status   = live}}</ref> || 40GB<ref>{{웹 인용|title=Better language models and their implications |url=https://openai.com/research/better-language-models |website=openai.com}}</ref> (~{{sort|10000000000|10 billion}} tokens)<ref name="LambdaLabs">{{웹 인용|title=OpenAI's GPT-3 Language Model: A Technical Overview |url=https://lambdalabs.com/blog/demystifying-gpt-3 |website=lambdalabs.com |language=en}}</ref> || {{yes|MIT}}<ref>{{웹 인용|work=GitHub|title=gpt-2|url=https://github.com/openai/gpt-2|access-date=13 March 2023}}</ref>
|-
| [[GPT-3]] || 2020년 || OpenAI || {{sort|175000000000|175 billion}}<ref name="Wiggers">{{웹 인용|work=TechCrunch|last=Wiggers|first=Kyle|date=28 April 2022|title=The emerging types of language models and why they matter|url=https://techcrunch.com/2022/04/28/the-emerging-types-of-language-models-and-why-they-matter/}}</ref> || {{sort|300000000000|300 billion}} tokens<ref name="LambdaLabs"/> || {{partial success|공개 웹 API}}
|-
| GPT-Neo || 2021년 3월 || [[EleutherAI]] || {{sort|2700000000|2.7 billion}}<ref name="gpt-neo">{{웹 인용|url=https://github.com/EleutherAI/gpt-neo|title=GPT Neo|date=March 15, 2023|via=GitHub}}</ref> || 825 GiB<ref name="Pile">{{ArXiv 인용|last1=Gao |first1=Leo |last2=Biderman |first2=Stella |last3=Black |first3=Sid |last4=Golding |first4=Laurence |last5=Hoppe |first5=Travis |last6=Foster |first6=Charles |last7=Phang |first7=Jason |last8=He |first8=Horace |last9=Thite |first9=Anish |last10=Nabeshima |first10=Noa |last11=Presser |first11=Shawn |last12=Leahy |first12=Connor |title=The Pile: An 800GB Dataset of Diverse Text for Language Modeling |eprint=2101.00027|date=31 December 2020 |class=cs.CL }}</ref> || {{yes|MIT}}<ref name="vb-gpt-neo">{{웹 인용|work=VentureBeat|last=Iyer|first=Abhishek|title=GPT-3's free alternative GPT-Neo is something to be excited about|date=15 May 2021|url=https://venturebeat.com/ai/gpt-3s-free-alternative-gpt-neo-is-something-to-be-excited-about/}}</ref>
|-
| [[GPT-J]] || 2021년 6월 || [[EleutherAI]] || {{sort|6000000000|6 billion}}<ref>{{웹 인용 |title=GPT-J-6B: An Introduction to the Largest Open Source GPT Model {{!}} Forefront |url=https://www.forefront.ai/blog-posts/gpt-j-6b-an-introduction-to-the-largest-open-sourced-gpt-model |access-date=2023-02-28 |website=www.forefront.ai |language=en |archive-date=2023-03-09 |archive-url=https://web.archive.org/web/20230309205439/https://www.forefront.ai/blog-posts/gpt-j-6b-an-introduction-to-the-largest-open-sourced-gpt-model |url-status= }}</ref> || 825 GiB<ref name="Pile"/> || {{yes|Apache 2.0}}
|-
| Megatron-Turing NLG || 2021년 10월<ref>{{웹 인용|last1=Alvi |first1=Ali |last2=Kharya |first2=Paresh |title=Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, the World's Largest and Most Powerful Generative Language Model |url=https://www.microsoft.com/en-us/research/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/ |website=Microsoft Research |date=11 October 2021}}</ref> || [[마이크로소프트]] and [[Nvidia]] || {{sort|530000000000|530 billion}}<ref name="mtnlg-preprint">{{Cite preprint |last1=Smith |first1=Shaden |last2=Patwary |first2=Mostofa |last3=Norick |first3=Brandon |last4=LeGresley |first4=Patrick |last5=Rajbhandari |first5=Samyam |last6=Casper |first6=Jared |last7=Liu |first7=Zhun |last8=Prabhumoye |first8=Shrimai |last9=Zerveas |first9=George |last10=Korthikanti |first10=Vijay |last11=Zhang |first11=Elton |last12=Child |first12=Rewon |last13=Aminabadi |first13=Reza Yazdani |last14=Bernauer |first14=Julie |last15=Song |first15=Xia |date=2022-02-04 |title=Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model |arxiv=2201.11990 }}</ref> || {{sort|338600000000|338.6 billion}} tokens<ref name=mtnlg-preprint/> || {{no|제한된 웹 접근}}
|-
| Ernie 3.0 Titan || 2021년 12월 || [[Baidu]] || {{sort|260000000000|260 billion}}<ref>{{저널 인용|title=ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation|first1=Shuohuan|last1=Wang|first2=Yu|last2=Sun|first3=Yang|last3=Xiang|first4=Zhihua|last4=Wu|first5=Siyu|last5=Ding|first6=Weibao|last6=Gong|first7=Shikun|last7=Feng|first8=Junyuan|last8=Shang|first9=Yanbin|last9=Zhao|first10=Chao|last10=Pang|first11=Jiaxiang|last11=Liu|first12=Xuyi|last12=Chen|first13=Yuxiang|last13=Lu|first14=Weixin|last14=Liu|first15=Xi|last15=Wang|first16=Yangfan|last16=Bai|first17=Qiuliang|last17=Chen|first18=Li|last18=Zhao|first19=Shiyong|last19=Li|first20=Peng|last20=Sun|first21=Dianhai|last21=Yu|first22=Yanjun|last22=Ma|first23=Hao|last23=Tian|first24=Hua|last24=Wu|first25=Tian|last25=Wu|first26=Wei|last26=Zeng|first27=Ge|last27=Li|first28=Wen|last28=Gao|first29=Haifeng|last29=Wang|date=December 23, 2021|arxiv=2112.12731}}</ref> || 4 Tb || {{no|사유(Proprietary)}}
|-
| Claude<ref>{{웹 인용|title=Product |url=https://www.anthropic.com/product |website=Anthropic |access-date=14 March 2023 |language=en}}</ref> || 2021년 12월 || [[Anthropic]] || {{sort|52000000000|52 billion}}<ref name="AnthroArch">{{ArXiv 인용|last1=Askell |first1=Amanda |last2=Bai |first2=Yuntao |last3=Chen |first3=Anna |last4=Drain |first4=Dawn |last5=Ganguli |first5=Deep |last6=Henighan |first6=Tom |last7=Jones |first7=Andy |last8=Joseph |first8=Nicholas |last9=Mann |first9=Ben |last10=DasSarma |first10=Nova |last11=Elhage |first11=Nelson |last12=Hatfield-Dodds |first12=Zac |last13=Hernandez |first13=Danny |last14=Kernion |first14=Jackson |last15=Ndousse |first15=Kamal |last16=Olsson |first16=Catherine |last17=Amodei |first17=Dario |last18=Brown |first18=Tom |last19=Clark |first19=Jack |last20=McCandlish |first20=Sam |last21=Olah |first21=Chris |last22=Kaplan |first22=Jared |display-authors=3 |title=A General Language Assistant as a Laboratory for Alignment |eprint=2112.00861 |date=9 December 2021 |class=cs.CL }}</ref> || {{sort|400000000000|400 billion}} tokens<ref name="AnthroArch"/> || {{partial success|클로즈드 베타}}
|-
| GLaM (Generalist Language Model) || 2021년 12월 || Google || {{sort|1200000000000|1.2 trillion}}<ref name="glam-blog">{{웹 인용|last1=Dai |first1=Andrew M |last2=Du |first2=Nan |date=December 9, 2021 |title=More Efficient In-Context Learning with GLaM |url=https://ai.googleblog.com/2021/12/more-efficient-in-context-learning-with.html |access-date=2023-03-09 |website=ai.googleblog.com |language=en}}</ref> || {{sort|1600000000000|1.6 trillion}} tokens<ref name=glam-blog/> || {{no|사유(Proprietary)}}
|-
| Gopher || 2021년 12월 || [[DeepMind]] || {{sort|280000000000|280 billion}}<ref>{{웹 인용|title=Language modelling at scale: Gopher, ethical considerations, and retrieval |url=https://www.deepmind.com/blog/language-modelling-at-scale-gopher-ethical-considerations-and-retrieval |website=www.deepmind.com |access-date=20 March 2023 |language=en}}</ref> || {{sort|300000000000|300 billion}} tokens<ref name=hoffman/> || {{no|사유(Proprietary)}}
|-
| [[LaMDA]] (Language Models for Dialog Applications) || 2022년 1월 || Google || {{sort|137000000000|137 billion}}<ref name="lamda-blog">{{웹 인용|last1=Cheng |first1=Heng-Tze |last2=Thoppilan |first2=Romal |date=January 21, 2022 |title=LaMDA: Towards Safe, Grounded, and High-Quality Dialog Models for Everything |url=https://ai.googleblog.com/2022/01/lamda-towards-safe-grounded-and-high.html |access-date=2023-03-09 |website=ai.googleblog.com |language=en}}</ref> ||  1.56T words,<ref name=lamda-blog/> {{sort|168000000000|168 billion}} tokens<ref name=hoffman/> || {{no|사유(Proprietary)}}
|-
| GPT-NeoX || 2022년 2월 || [[EleutherAI]] || {{sort|20000000000|20 billion}}<ref name="gpt-neox-20b">{{콘퍼런스 인용|title=GPT-NeoX-20B: An Open-Source Autoregressive Language Model |conference=Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models |date=2022-05-01 |last1=Black |first1=Sidney |last2=Biderman |first2=Stella |last3=Hallahan |first3=Eric |display-authors=etal |volume=Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models |pages=95–136 |url=https://aclanthology.org/2022.bigscience-1.9/ |accessdate=2022-12-19 }}</ref> || 825 GiB<ref name="Pile"/> || {{yes|Apache 2.0}}
|-
| [[Chinchilla AI|Chinchilla]] || 2022년 3월 || [[DeepMind]] || {{sort|70000000000|70 billion}}<ref name="chinchilla-blog">{{웹 인용|work=Deepmind Blog|title=An empirical analysis of compute-optimal large language model training|first1=Jordan|last1=Hoffmann|first2=Sebastian|last2=Borgeaud|first3=Arthur|last3=Mensch|first4=Laurent|last4=Sifre|date=12 April 2022|url=https://www.deepmind.com/blog/an-empirical-analysis-of-compute-optimal-large-language-model-training}}</ref> || {{sort|1400000000000|1.4 trillion}} tokens<ref name=chinchilla-blog/><ref name=hoffman>{{ArXiv 인용|last1=Hoffmann |first1=Jordan |last2=Borgeaud |first2=Sebastian |last3=Mensch |first3=Arthur |last4=Buchatskaya |first4=Elena |last5=Cai |first5=Trevor |last6=Rutherford |first6=Eliza |last7=Casas |first7=Diego de Las |last8=Hendricks |first8=Lisa Anne |last9=Welbl |first9=Johannes |last10=Clark |first10=Aidan |last11=Hennigan |first11=Tom |last12=Noland |first12=Eric |last13=Millican |first13=Katie |last14=Driessche |first14=George van den |last15=Damoc |first15=Bogdan |last16=Guy |first16=Aurelia |last17=Osindero |first17=Simon |last18=Simonyan |first18=Karen |last19=Elsen |first19=Erich |last20=Rae |first20=Jack W. |last21=Vinyals |first21=Oriol |last22=Sifre |first22=Laurent |title=Training Compute-Optimal Large Language Models |eprint=2203.15556 |date=29 March 2022 |class=cs.CL |display-authors=3}}</ref> || {{no|사유(Proprietary)}}
|-
| [[PaLM]] (Pathways Language Model) || 2022년 4월 || Google || {{sort|540000000000|540 billion}}<ref name="palm-blog">{{웹 인용|last1=Narang |first1=Sharan |last2=Chowdhery |first2=Aakanksha |date=April 4, 2022 |title=Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrough Performance |url=https://ai.googleblog.com/2022/04/pathways-language-model-palm-scaling-to.html |access-date=2023-03-09 |website=ai.googleblog.com |language=en}}</ref> || {{sort|768000000000|768 billion}} tokens<ref name=chinchilla-blog/> || {{no|사유(Proprietary)}}
|-
| OPT (Open Pretrained Transformer) || 2022년 5월 || [[Meta Platforms|Meta]] || {{sort|175000000000|175 billion}}<ref>{{웹 인용|title=Democratizing access to large-scale language models with OPT-175B |url=https://ai.facebook.com/blog/democratizing-access-to-large-scale-language-models-with-opt-175b/ |website=ai.facebook.com |language=en}}</ref> || {{sort|180000000000|180 billion}} tokens<ref>{{ArXiv 인용|last1=Zhang |first1=Susan |last2=Roller |first2=Stephen |last3=Goyal |first3=Naman |last4=Artetxe |first4=Mikel |last5=Chen |first5=Moya |last6=Chen |first6=Shuohui |last7=Dewan |first7=Christopher |last8=Diab |first8=Mona |last9=Li |first9=Xian |last10=Lin |first10=Xi Victoria |last11=Mihaylov |first11=Todor |last12=Ott |first12=Myle |last13=Shleifer |first13=Sam |last14=Shuster |first14=Kurt |last15=Simig |first15=Daniel |last16=Koura |first16=Punit Singh |last17=Sridhar |first17=Anjali |last18=Wang |first18=Tianlu |last19=Zettlemoyer |first19=Luke |title=OPT: Open Pre-trained Transformer Language Models |eprint=2205.01068 |date=21 June 2022|class=cs.CL }}</ref> || {{partial success|비상업적 연구}}{{efn|The smaller models including 66B are publicly available, while the 175B model is available on request.}}
|-
|YaLM 100B
|2022년 6월
|[[Yandex]]
|{{sort|100000000000|100 billion}}<ref name="yalm-repo">{{인용|last1=Khrushchev |first1=Mikhail |title=YaLM 100B |date=2022-06-22 |url=https://github.com/yandex/YaLM-100B |access-date=2023-03-18 |last2=Vasilev |first2=Ruslan |last3=Petrov |first3=Alexey |last4=Zinov |first4=Nikolay}}</ref>
|1.7TB<ref name="yalm-repo" />
|{{Yes|Apache 2.0}}
|-
| Minerva || 2022년 6월 || Google || {{sort|540000000000|540 billion}}<ref name=minerva-paper/> || 38.5B tokens from webpages filtered for mathematical content and from papers submitted to the arXiv preprint server<ref name=minerva-paper>{{ArXiv 인용|last1=Lewkowycz |first1=Aitor |last2=Andreassen |first2=Anders |last3=Dohan |first3=David |last4=Dyer |first4=Ethan |last5=Michalewski |first5=Henryk |last6=Ramasesh |first6=Vinay |last7=Slone |first7=Ambrose |last8=Anil |first8=Cem |last9=Schlag |first9=Imanol |last10=Gutman-Solo |first10=Theo |last11=Wu |first11=Yuhuai |last12=Neyshabur |first12=Behnam |last13=Gur-Ari |first13=Guy |last14=Misra |first14=Vedant |title=Solving Quantitative Reasoning Problems with Language Models |date=30 June 2022 |class=cs.CL |eprint=2206.14858 }}</ref>  || {{no|사유(Proprietary)}}
|-
| [[BLOOM (language model)|BLOOM]] || 2022년 7월 || Large collaboration led by [[Hugging Face]] || {{sort|175000000000|175 billion}}<ref name="bigger-better">{{웹 인용|work=Nature|last=Ananthaswamy|first=Anil|title=In AI, is bigger always better?|date=8 March 2023|url=https://www.nature.com/articles/d41586-023-00641-w}}</ref> || {{sort|350000000000|350 billion}} tokens (1.6TB)<ref>{{웹 인용|title=bigscience/bloom · Hugging Face |url=https://huggingface.co/bigscience/bloom |website=huggingface.co}}</ref> || {{yes|Responsible AI}}
|-
| Galactica || 2022년 11월 || [[Meta Platforms|Meta]] || {{sort|120000000000|120 billion}} || {{sort|350000000000|106 billion}} tokens<ref>{{ArXiv 인용|last1=Taylor |first1=Ross |last2=Kardas |first2=Marcin |last3=Cucurull |first3=Guillem |last4=Scialom |first4=Thomas |last5=Hartshorn |first5=Anthony |last6=Saravia |first6=Elvis |last7=Poulton |first7=Andrew |last8=Kerkez |first8=Viktor |last9=Stojnic |first9=Robert |title=Galactica: A Large Language Model for Science |date=16 November 2022 |class=cs.CL |eprint=2211.09085 }}</ref> || {{partial success|CC-BY-NC-4.0}}
|-
| AlexaTM (Teacher Models) || 2022년 11월 || [[아마존 (기업)|Amazon]] || {{sort|20000000000|20 billion}}<ref>{{웹 인용|title=20B-parameter Alexa model sets new marks in few-shot learning |url=https://www.amazon.science/blog/20b-parameter-alexa-model-sets-new-marks-in-few-shot-learning |website=Amazon Science |language=en |date=2 August 2022}}</ref> || {{sort|1300000000000|1.3 trillion}}<ref>{{ArXiv 인용|last1=Soltan |first1=Saleh |last2=Ananthakrishnan |first2=Shankar |last3=FitzGerald |first3=Jack |last4=Gupta |first4=Rahul |last5=Hamza |first5=Wael |last6=Khan |first6=Haidar |last7=Peris |first7=Charith |last8=Rawls |first8=Stephen |last9=Rosenbaum |first9=Andy |last10=Rumshisky |first10=Anna |last11=Prakash |first11=Chandana Satya |last12=Sridhar |first12=Mukund |last13=Triefenbach |first13=Fabian |last14=Verma |first14=Apurv |last15=Tur |first15=Gokhan |last16=Natarajan |first16=Prem |display-authors=3|title=AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model |eprint=2208.01448 |date=3 August 2022|class=cs.CL }}</ref> || {{partial success|공개 웹 API}}<ref>{{웹 인용|title=AlexaTM 20B is now available in Amazon SageMaker JumpStart {{!}} AWS Machine Learning Blog |url=https://aws.amazon.com/blogs/machine-learning/alexatm-20b-is-now-available-in-amazon-sagemaker-jumpstart/ |website=aws.amazon.com |access-date=13 March 2023 |date=17 November 2022}}</ref>
|-
| [[LLaMA]] (Large Language Model Meta AI) || 2023년 2월 || [[Meta Platforms|Meta]] || {{sort|65000000000|65 billion}}<ref name="llama-blog">{{웹 인용|work=Meta AI|title=Introducing LLaMA: A foundational, 65-billion-parameter large language model|date=24 February 2023|url=https://ai.facebook.com/blog/large-language-model-llama-meta-ai/}}</ref> || {{sort|1400000000000|1.4 trillion}}<ref name=llama-blog/> || {{partial success|비상업적 연구}}{{efn|Facebook's license and distribution scheme restricted access to approved researchers, but the model weights were leaked and became widely available.}}
|-
| [[GPT-4]] || 2023년 3월 || OpenAI || 정확한 수치 알 수 없음. 대략 {{sort|1000000000000|1 trillion}} {{efn|As stated in Technical report: "Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method ..."<ref name="GPT4Tech">{{웹 인용|date=2023 |title=GPT-4 Technical Report |url=https://cdn.openai.com/papers/gpt-4.pdf |website=[[OpenAI]] |access-date=March 14, 2023 |archive-date=March 14, 2023 |archive-url=https://web.archive.org/web/20230314190904/https://cdn.openai.com/papers/gpt-4.pdf |url-status=live }}</ref> Approximate number in the comparison chart that compares the relative storage, from the same report.}} || 알 수 없음 || {{partial success|공개 웹 API}}
|-
|Cerebras-GPT
|2023년 3월
|Cerebras
|{{sort|13000000000|13 billion}}<ref>{{웹 인용|url=https://www.cerebras.net/blog/cerebras-gpt-a-family-of-open-compute-efficient-large-language-models/|title=Cerebras-GPT: A Family of Open, Compute-efficient, Large Language Models|first=Nolan|last=Dey|date=March 28, 2023|website=Cerebras}}</ref>
| || {{yes|Apache 2.0}}
|-
| Falcon || 2023년 3월 || [[Technology Innovation Institute]] || {{sort|40000000000|40 billion}}<ref name='falcon'>{{웹 인용|title=Abu Dhabi-based TII launches its own version of ChatGPT |url=https://fastcompanyme.com/news/abu-dhabi-based-tii-launches-its-own-version-of-chatgpt/ |website=tii.ae}}</ref> || {{sort|1000000000000|1 Trillion}} tokens (1TB)<ref name='falcon'/> || {{no|사유(Proprietary)}}
|-
| BloombergGPT || 2023년 3월 || [[Bloomberg L.P.]] || {{sort|50000000000|50 billion}} || 363 billion token dataset based on Bloomberg's data sources, plus 345 billion tokens from general purpose datasets<ref>{{저널 인용|title=BloombergGPT: A Large Language Model for Finance|first1=Shijie|last1=Wu|first2=Ozan|last2=Irsoy|first3=Steven|last3=Lu|first4=Vadim|last4=Dabravolski|first5=Mark|last5=Dredze|first6=Sebastian|last6=Gehrmann|first7=Prabhanjan|last7=Kambadur|first8=David|last8=Rosenberg|first9=Gideon|last9=Mann|date=March 30, 2023|arxiv=2303.17564}}</ref> || {{no|사유(Proprietary)}}
|-
| PanGu-Σ || 2023년 3월 || [[Huawei]] || {{sort|1085000000000|1.085 trillion}} || 329 billion tokens<ref>{{저널 인용|title=PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing|first1=Xiaozhe|last1=Ren|first2=Pingyi|last2=Zhou|first3=Xinfan|last3=Meng|first4=Xinjing|last4=Huang|first5=Yadao|last5=Wang|first6=Weichao|last6=Wang|first7=Pengfei|last7=Li|first8=Xiaoda|last8=Zhang|first9=Alexander|last9=Podolskiy|first10=Grigory|last10=Arshinov|first11=Andrey|last11=Bout|first12=Irina|last12=Piontkovskaya|first13=Jiansheng|last13=Wei|first14=Xin|last14=Jiang|first15=Teng|last15=Su|first16=Qun|last16=Liu|first17=Jun|last17=Yao|date=March 19, 2023|arxiv=2303.10845}}</ref> || {{no|사유(Proprietary)}}
|-
| OpenAssistant<ref>{{저널 인용|last=Köpf |first=Andreas |last2=Kilcher |first2=Yannic |last3=von Rütte |first3=Dimitri |last4=Anagnostidis |first4=Sotiris |last5=Tam |first5=Zhi-Rui |last6=Stevens |first6=Keith |last7=Barhoum |first7=Abdullah |last8=Duc |first8=Nguyen Minh |last9=Stanley |first9=Oliver |last10=Nagyfi |first10=Richárd |last11=ES |first11=Shahul |last12=Suri |first12=Sameer |last13=Glushkov |first13=David |last14=Dantuluri |first14=Arnav |last15=Maguire |first15=Andrew |date=2023-04-14 |title=OpenAssistant Conversations -- Democratizing Large Language Model Alignment |url=http://arxiv.org/abs/2304.07327 |journal=arXiv:2304.07327 [cs]}}</ref> || 2023년 3월 || [[LAION]] || {{sort|17000000000|17 billion}} || 1.5 trillion tokens || {{yes|Apache 2.0}}
|-
| [[PaLM|PaLM 2]] (Pathways Language Model 2) || 2023년 5월 || Google || {{sort|340000000000|340 billion}}<ref name="cnbc-20230516">{{웹 인용|last=Elias |first=Jennifer |url=https://www.cnbc.com/2023/05/16/googles-palm-2-uses-nearly-five-times-more-text-data-than-predecessor.html |title=Google's newest A.I. model uses nearly five times more text data for training than its predecessor |work=[[CNBC]] |date=16 May 2023 |access-date=18 May 2023}}</ref> || {{sort|3600000000000|3.6 trillion}} tokens<ref name="cnbc-20230516" /> || {{no|사유(Proprietary)}}
|}

<references group="lower-alpha" />

== 같이 보기 ==
* [[인공 초지능]]

== 각주 ==
{{각주}}

== 같이 보기 ==
* [[멀티모덜 학습]]
* [[검색증강생성]]

{{자연어 처리}}

{{전거 통제}}

[[분류:대형 언어 모델| ]]
[[분류:딥 러닝]]
[[분류:자연어 처리]]