File size: 3,788 Bytes
d4041e4 18b5bb2 739b1b5 18b5bb2 d4041e4 d8949ca d4041e4 2e32fb2 d4041e4 c43a651 d4041e4 18b5bb2 d4041e4 a8bdb1e d4041e4 739b1b5 d4041e4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
### Model Sources
- **Paper**: LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages
- **Link**: https://arxiv.org/pdf/2407.05975
- **Repository**: https://github.com/CONE-MT/LLaMAX/
### Model Description
LLaMAX3-8B is a multilingual language base model, developed through continued pre-training on Llama3, and supports over 100 languages.
LLaMAX3-8B can serve as a base model to support downstream multilingual tasks but without instruct-following capability.
We further fine-tuned LLaMAX3-8B on Alpaca dataset to enhance its instruct-following capabilities. The model is available at https://huggingface.co/LLaMAX/LLaMAX3-8B-Alpaca.
### 🔥 Effective Base Model for Multilingual Task
LLaMAX preserves its efficacy in general tasks and improves the performance on multilingual tasks.
We fine-tuned LLaMAX using only the English training set of downstream task, which also shows significant improvements in non-English. We provide fine-tuning LLaMAX models for the following three tasks:
- **Math Reasoning**: https://huggingface.co/LLaMAX/LLaMAX2-7B-MetaMath
- **Commonsense Reasoning**: https://huggingface.co/LLaMAX/LLaMAX2-7B-X-CSQA
- **Natural Language Inference**: https://huggingface.co/LLaMAX/LLaMAX2-7B-XNLI
### Supported Languages
Akrikaans (af), Amharic (am), Arabic (ar), Armenian (hy), Assamese (as), Asturian (ast), Azerbaijani (az), Belarusian (be), Bengali (bn), Bosnian (bs), Bulgarian (bg), Burmese (my), Catalan (ca), Cebuano (ceb), Chinese Simpl (zho), Chinese Trad (zho), Croatian (hr), Czech (cs), Danish (da), Dutch (nl), English (en), Estonian (et), Filipino (tl), Finnish (fi), French (fr), Fulah (ff), Galician (gl), Ganda (lg), Georgian (ka), German (de), Greek (el), Gujarati (gu), Hausa (ha), Hebrew (he), Hindi (hi), Hungarian (hu), Icelandic (is), Igbo (ig), Indonesian (id), Irish (ga), Italian (it), Japanese (ja), Javanese (jv), Kabuverdianu (kea), Kamba (kam), Kannada (kn), Kazakh (kk), Khmer (km), Korean (ko), Kyrgyz (ky), Lao (lo), Latvian (lv), Lingala (ln), Lithuanian (lt), Luo (luo), Luxembourgish (lb), Macedonian (mk), Malay (ms), Malayalam (ml), Maltese (mt), Maori (mi), Marathi (mr), Mongolian (mn), Nepali (ne), Northern Sotho (ns), Norwegian (no), Nyanja (ny), Occitan (oc), Oriya (or), Oromo (om), Pashto (ps), Persian (fa), Polish (pl), Portuguese (pt), Punjabi (pa), Romanian (ro), Russian (ru), Serbian (sr), Shona (sn), Sindhi (sd), Slovak (sk), Slovenian (sl), Somali (so), Sorani Kurdish (ku), Spanish (es), Swahili (sw), Swedish (sv), Tajik (tg), Tamil (ta), Telugu (te), Thai (th), Turkish (tr), Ukrainian (uk), Umbundu (umb), Urdu (ur), Uzbek (uz), Vietnamese (vi), Welsh (cy), Wolof (wo), Xhosa (xh), Yoruba (yo), Zulu (zu)
### Model Index
| Model | LLaMAX | LLaMAX-Alpaca |
|---------|----------------------------------------------------------|-----------------------------------------------------------------|
| Llama-2 | [Link](https://huggingface.co/LLaMAX/LLaMAX2-7B) | [Link](https://huggingface.co/LLaMAX/LLaMAX2-7B-Alpaca) |
| Llama-3 | [Link](https://huggingface.co/LLaMAX/LLaMAX3-8B) | [Link](https://huggingface.co/LLaMAX/LLaMAX3-8B-Alpaca) |
### Citation
If our model helps your work, please cite this paper:
```
@misc{lu2024llamaxscalinglinguistichorizons,
title={LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages},
author={Yinquan Lu and Wenhao Zhu and Lei Li and Yu Qiao and Fei Yuan},
year={2024},
eprint={2407.05975},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2407.05975},
}
```
|