End of training

README.md

@@ -0,0 +1,65 @@
+---
+base_model: igorktech/nllb-pruned-6L-512d-finetuned-v1
+tags:
+- peft
+- lora
+- generated_from_trainer
+metrics:
+- bleu
+model-index:
+- name: nllb-200-tiny-tuned
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/igorktech01/nllb-finetuning/runs/n78ud9nz)
+# nllb-200-tiny-tuned
+
+This model is a fine-tuned version of [igorktech/nllb-pruned-6L-512d-finetuned-v1](https://huggingface.co/igorktech/nllb-pruned-6L-512d-finetuned-v1) on the your_dataset_name dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0980
+- Bleu: 52.9983
+- Chrf++: 73.2746
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0001
+- train_batch_size: 16
+- eval_batch_size: 4
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 2.0
+
+### Training results
+
+| Training Loss | Epoch  | Step  | Validation Loss | Bleu    | Chrf++  |
+|:-------------:|:------:|:-----:|:---------------:|:-------:|:-------:|
+| 0.2285        | 0.6563 | 5000  | 0.1744          | 37.0851 | 62.3627 |
+| 0.1872        | 1.3125 | 10000 | 0.1214          | 47.5689 | 69.8186 |
+| 0.1089        | 1.9688 | 15000 | 0.0980          | 52.9983 | 73.2746 |
+
+
+### Framework versions
+
+- Transformers 4.42.3
+- Pytorch 2.3.1+cu121
+- Datasets 2.20.0
+- Tokenizers 0.19.1

generation_config.json

@@ -0,0 +1,8 @@
+{
+  "bos_token_id": 0,
+  "decoder_start_token_id": 2,
+  "eos_token_id": 2,
+  "max_length": 200,
+  "pad_token_id": 1,
+  "transformers_version": "4.42.3"
+}

tokenizer.json

@@ -2,14 +2,12 @@
   "version": "1.0",
   "truncation": {
     "direction": "Right",
-    "max_length": 384,
+    "max_length": 512,
     "strategy": "LongestFirst",
     "stride": 0
   },
   "padding": {
-    "strategy": {
-      "Fixed": 384
-    },
+    "strategy": "BatchLongest",
     "direction": "Right",
     "pad_to_multiple_of": null,
     "pad_id": 1,
@@ -1931,7 +1929,7 @@
     "single": [
       {
         "SpecialToken": {
-          "id": "arn_Latn",
+          "id": "spa_Latn",
           "type_id": 0
         }
       },
@@ -1951,7 +1949,7 @@
     "pair": [
       {
         "SpecialToken": {
-          "id": "arn_Latn",
+          "id": "spa_Latn",
           "type_id": 0
         }
       },
@@ -1984,13 +1982,13 @@
           "</s>"
         ]
       },
-      "arn_Latn": {
-        "id": "arn_Latn",
+      "spa_Latn": {
+        "id": "spa_Latn",
         "ids": [
-          50014
+          50163
         ],
         "tokens": [
-          "arn_Latn"
+          "spa_Latn"
         ]
       }
     }

tokenizer_config.json

@@ -1889,8 +1889,8 @@
   "pad_token": "<pad>",
   "sep_token": "</s>",
   "sp_model_kwargs": {},
-  "src_lang": "arn_Latn",
-  "tgt_lang": null,
+  "src_lang": "spa_Latn",
+  "tgt_lang": "arn_Latn",
   "tokenizer_class": "NllbTokenizer",
   "unk_token": "<unk>"
 }