End of training

README.md

@@ -1,6 +1,6 @@
 ---
 license: apache-2.0
-base_model: t5-small
+base_model: t5-base
 tags:
 - generated_from_trainer
 metrics:
@@ -15,11 +15,11 @@ should probably proofread and complete it, then remove this comment. -->
 
 # indi-translate
 
-This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on an unknown dataset.
+This model is a fine-tuned version of [t5-base](https://huggingface.co/t5-base) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.5069
+- Loss: 0.4235
 - Bleu: 0.0
-- Gen Len: 18.9521
+- Gen Len: 14.018
 
 ## Model description
 
@@ -50,8 +50,8 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len |
 |:-------------:|:-----:|:----:|:---------------:|:----:|:-------:|
-| No log        | 1.0   | 126  | 0.5299          | 0.0  | 18.9162 |
-| No log        | 2.0   | 252  | 0.5069          | 0.0  | 18.9521 |
+| No log        | 1.0   | 126  | 0.4373          | 0.0  | 14.0619 |
+| No log        | 2.0   | 252  | 0.4235          | 0.0  | 14.018  |
 
 
 ### Framework versions

config.json

@@ -1,12 +1,12 @@
 {
-  "_name_or_path": "t5-small",
+  "_name_or_path": "t5-base",
   "architectures": [
     "T5ForConditionalGeneration"
   ],
   "classifier_dropout": 0.0,
-  "d_ff": 2048,
+  "d_ff": 3072,
   "d_kv": 64,
-  "d_model": 512,
+  "d_model": 768,
   "decoder_start_token_id": 0,
   "dense_act_fn": "relu",
   "dropout_rate": 0.1,
@@ -18,9 +18,9 @@
   "layer_norm_epsilon": 1e-06,
   "model_type": "t5",
   "n_positions": 512,
-  "num_decoder_layers": 6,
-  "num_heads": 8,
-  "num_layers": 6,
+  "num_decoder_layers": 12,
+  "num_heads": 12,
+  "num_layers": 12,
   "output_past": true,
   "pad_token_id": 0,
   "relative_attention_max_distance": 128,

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ef667498544731d47368e458d37b8c94f077b0bdb253df413a348840d90a6b8a
-size 242071641
+oid sha256:4248eb5b7dce75cedfea3c5c3474de02068a6686f7aea4c0eb7a3afcc39d6cf6
+size 891702929

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:92b9caaf621c42a5579a9b703e9859f17f644ecca41bd0a39cfb52d055aa23b9
+oid sha256:eb15aaf9fb5a21bf9631362774c3ac8d4450c0b9565b70af2d1bc5b161a559b6
 size 4219