llama3-7b-en-ru-v3

LLaMA-3 7B pretrained on English-Russian bilingual data, 133,600 total steps (v3 run). Uses a shared 65k English-Russian tokenizer (65k_en1.0_ru1.0).

Validation Results

Last logged validation: step 132,264 (weights from step-133,500; step-133,600 checkpoint was empty due to async write not completing before job exit)

Validation set	Cross-entropy loss (nats)	Perplexity
English (en)	2.244	9.43
Russian (ru)	2.1955	8.98

Training Curves

Training and validation loss curves are in training_curves/:

• train_en_ru_v3.csv — step, train_loss (13,361 entries, steps 1–133,600)
• val_en_ru_v3.csv — step, lang, val_loss (100 checkpoints × 2 languages, steps 1–132,264)

Evaluation Results

EEE-format evaluation results are stored under eval_results/eee/ in this repository. Tasks: Global MMLU (EN), Global MMLU (RU), PIQA, ECLeKTic.

Related Models

• The-CoLab/llama3-7b-en-ru-v2 — v2 run (injection fix)

Citation

Part of the The-CoLab multilingual-transfer collection.