|
--- |
|
language: |
|
- et |
|
- en |
|
pipeline_tag: text-generation |
|
base_model: |
|
- meta-llama/Llama-2-7b-hf |
|
--- |
|
# LLammas-base 🐑 |
|
|
|
Llama-2-7B with continued pre-training of 5B tokens of CulturaX (75% Estonian, 25% English documents). |
|
|
|
This model is also instruction-tuned resulting in [Llammas](https://huggingface.co/tartuNLP/Llammas). |
|
|
|
More details in our [paper](https://arxiv.org/abs/2404.04042). |
|
|
|
### Citation |
|
``` |
|
@misc{kuulmets2024teaching, |
|
title={Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer}, |
|
author={Hele-Andra Kuulmets and Taido Purason and Agnes Luhtaru and Mark Fishel}, |
|
year={2024}, |
|
eprint={2404.04042}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL} |
|
} |
|
``` |
