Multilingual-Transfer
Collection
Pretraining models to find what allows multilingual transfer • 21 items • Updated • 2
llama3-7b-en-translated-ru
LLaMA-3 7B pretrained on English and translated Russian data, 134k steps.
Validation Results
Final checkpoint: step 133,600
| Validation set | Cross-entropy loss (nats) | Perplexity |
|---|---|---|
| English (en) | 2.2272 | 9.27 |
RU (ru) |
1.8976 | 6.67 |
Note: The training config had a copy-paste bug: the Russian validation dataloader (
fineweb2-hq-ru) was keyed"ar"instead of"ru", causing the log to showar validate. The validation data and loss value are correct Russian validation.
Evaluation Results
EEE-format evaluation results are stored under eval_results/eee/ in this repository.
Tasks: Global MMLU (EN/AR/RU), PIQA, ECLeKTic, Fictive Entity (2-rate mix).
Citation
Part of the The-CoLab multilingual-transfer collection.
- Downloads last month
- 57
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support