End of training

README.md

@@ -1,5 +1,4 @@
 ---
-base_model: google/pegasus-xsum
 tags:
 - generated_from_trainer
 metrics:
@@ -14,14 +13,14 @@ should probably proofread and complete it, then remove this comment. -->
 
 # LLM_Teached_Pegasus
 
-This model is a fine-tuned version of [google/pegasus-xsum](https://huggingface.co/google/pegasus-xsum) on an unknown dataset.
+This model was trained from scratch on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.7905
-- Rouge1: 0.4388
-- Rouge2: 0.1916
-- Rougel: 0.3479
-- Rougelsum: 0.3476
-- Gen Len: 28.7182
+- Loss: 1.6452
+- Rouge1: 0.4595
+- Rouge2: 0.2033
+- Rougel: 0.3629
+- Rougelsum: 0.3628
+- Gen Len: 30.8536
 
 ## Model description
 
@@ -41,9 +40,11 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 2e-05
-- train_batch_size: 16
+- train_batch_size: 8
 - eval_batch_size: 8
 - seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 32
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - num_epochs: 2
@@ -53,8 +54,8 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
 |:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|
-| 2.0092        | 1.0   | 1250 | 1.8228          | 0.4351 | 0.188  | 0.3414 | 0.3411    | 28.7045 |
-| 1.8992        | 2.0   | 2500 | 1.7905          | 0.4388 | 0.1916 | 0.3479 | 0.3476    | 28.7182 |
+| 1.7637        | 1.0   | 625  | 1.6549          | 0.4591 | 0.205  | 0.3628 | 0.3628    | 30.8636 |
+| 1.7226        | 2.0   | 1250 | 1.6452          | 0.4595 | 0.2033 | 0.3629 | 0.3628    | 30.8536 |
 
 
 ### Framework versions

generation_config.json

@@ -3,8 +3,8 @@
   "decoder_start_token_id": 0,
   "eos_token_id": 1,
   "forced_eos_token_id": 1,
-  "length_penalty": 0.6,
-  "max_length": 64,
+  "length_penalty": 0.8,
+  "max_length": 256,
   "num_beams": 8,
   "pad_token_id": 0,
   "transformers_version": "4.36.0"

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1609b09f05b64a32b53d0ad81dd41f022a9854f7f32b8cd32a0ea25e24042cea
+oid sha256:ab52b4b7f1b050bd880fa12529d3c148fdd68fce219a518cfcb8e630a158c091
 size 2283652852

runs/Dec28_11-12-24_n4bcoectr1703727001286-fmclw/events.out.tfevents.1703733148.n4bcoectr1703727001286-fmclw.64933.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a65f50becec57e756077bf6804400342a4c6426d8b155c0eba55c0325aef9fe5
-size 7836
+oid sha256:2149872a8dbcce347ede2e31bda798bf8f519bb64566837cdbe635dbd35b028b
+size 8715