nicholasKluge
commited on
Commit
•
cc44bcc
1
Parent(s):
0f2cb9f
Update README.md
Browse files
README.md
CHANGED
@@ -41,6 +41,8 @@ co2_eq_emissions:
|
|
41 |
|
42 |
Large language models (LLMs) have significantly advanced natural language processing, but their progress has yet to be equal across languages. While most LLMs are trained in high-resource languages like English, multilingual models generally underperform monolingual ones. Additionally, aspects of their multilingual foundation sometimes restrict the byproducts they produce, like computational demands and licensing regimes. Hence, we developed the _TeenyTinyLlama_ pair: two compact models for Brazilian Portuguese text generation.
|
43 |
|
|
|
|
|
44 |
## Details
|
45 |
|
46 |
- **Architecture:** a Transformer-based model pre-trained via causal language modeling
|
@@ -205,7 +207,7 @@ All the shown results are the higher accuracy scores achieved on the respective
|
|
205 |
```latex
|
206 |
|
207 |
@misc{correa24ttllama,
|
208 |
-
title = {TeenyTinyLlama:
|
209 |
author = {Corr{\^e}a, Nicholas Kluge and Falk, Sophia and Fatimah, Shiza and Sen, Aniket and De Oliveira, Nythamar},
|
210 |
journal={arXiv},
|
211 |
year = {2024},
|
|
|
41 |
|
42 |
Large language models (LLMs) have significantly advanced natural language processing, but their progress has yet to be equal across languages. While most LLMs are trained in high-resource languages like English, multilingual models generally underperform monolingual ones. Additionally, aspects of their multilingual foundation sometimes restrict the byproducts they produce, like computational demands and licensing regimes. Hence, we developed the _TeenyTinyLlama_ pair: two compact models for Brazilian Portuguese text generation.
|
43 |
|
44 |
+
Read our preprint on [ArXiv](xxx).
|
45 |
+
|
46 |
## Details
|
47 |
|
48 |
- **Architecture:** a Transformer-based model pre-trained via causal language modeling
|
|
|
207 |
```latex
|
208 |
|
209 |
@misc{correa24ttllama,
|
210 |
+
title = {TeenyTinyLlama: open-source tiny language models trained in Brazilian Portuguese},
|
211 |
author = {Corr{\^e}a, Nicholas Kluge and Falk, Sophia and Fatimah, Shiza and Sen, Aniket and De Oliveira, Nythamar},
|
212 |
journal={arXiv},
|
213 |
year = {2024},
|