Hugging Face's logo

GlycerinLOL
/
LLM_Teached_Pegasus_From_Scratch

Text2Text Generation
Transformers
TensorBoard
Safetensors
pegasus
generated_from_trainer
Inference Endpoints
Model card Files Metrics Community
GlycerinLOL commited on
Commit
5dfc59b
•
1 Parent(s): ca874d1

Model save

Browse files
Files changed (4) hide show
  1. README.md +87 -0
  2. generation_config.json +11 -0
  3. model.safetensors +1 -1
  4. runs/Mar05_09-51-13_oi5vv8ctr1709312124223-tkfr5/events.out.tfevents.1709603487.oi5vv8ctr1709312124223-tkfr5.11611.0 +2 -2
README.md ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: google/pegasus-large
3
+ tags:
4
+ - generated_from_trainer
5
+ metrics:
6
+ - rouge
7
+ - precision
8
+ - recall
9
+ - f1
10
+ model-index:
11
+ - name: LLM_Teached_Pegasus_From_Scratch
12
+ results: []
13
+ ---
14
+
15
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
16
+ should probably proofread and complete it, then remove this comment. -->
17
+
18
+ # LLM_Teached_Pegasus_From_Scratch
19
+
20
+ This model is a fine-tuned version of [google/pegasus-large](https://huggingface.co/google/pegasus-large) on an unknown dataset.
21
+ It achieves the following results on the evaluation set:
22
+ - Loss: 1.5146
23
+ - Rouge1: 0.4863
24
+ - Rouge2: 0.2348
25
+ - Rougel: 0.4011
26
+ - Rougelsum: 0.4012
27
+ - Gen Len: 27.5716
28
+ - Precision: 0.9118
29
+ - Recall: 0.9131
30
+ - F1: 0.9122
31
+
32
+ ## Model description
33
+
34
+ More information needed
35
+
36
+ ## Intended uses & limitations
37
+
38
+ More information needed
39
+
40
+ ## Training and evaluation data
41
+
42
+ More information needed
43
+
44
+ ## Training procedure
45
+
46
+ ### Training hyperparameters
47
+
48
+ The following hyperparameters were used during training:
49
+ - learning_rate: 2e-05
50
+ - train_batch_size: 24
51
+ - eval_batch_size: 16
52
+ - seed: 42
53
+ - gradient_accumulation_steps: 4
54
+ - total_train_batch_size: 96
55
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
56
+ - lr_scheduler_type: linear
57
+ - num_epochs: 16
58
+ - mixed_precision_training: Native AMP
59
+
60
+ ### Training results
61
+
62
+ | Training Loss | Epoch | Step | F1 | Gen Len | Validation Loss | Precision | Recall | Rouge1 | Rouge2 | Rougel | Rougelsum |
63
+ |:-------------:|:-----:|:----:|:------:|:-------:|:---------------:|:---------:|:------:|:------:|:------:|:------:|:---------:|
64
+ | 2.0443 | 1.0 | 521 | 0.9049 | 28.3633 | 1.7046 | 0.9041 | 0.9061 | 0.4488 | 0.203 | 0.3633 | 0.3633 |
65
+ | 1.7826 | 2.0 | 1042 | 0.9072 | 28.1949 | 1.6347 | 0.9062 | 0.9085 | 0.4616 | 0.2133 | 0.3761 | 0.3758 |
66
+ | 1.7134 | 3.0 | 1563 | 0.9084 | 28.5218 | 1.5991 | 0.9072 | 0.91 | 0.4683 | 0.2186 | 0.3824 | 0.3822 |
67
+ | 1.6664 | 4.0 | 2084 | 0.9096 | 28.2498 | 1.5767 | 0.9087 | 0.9109 | 0.4738 | 0.2233 | 0.3878 | 0.3876 |
68
+ | 1.6296 | 5.0 | 2605 | 0.9103 | 28.2396 | 1.5595 | 0.9093 | 0.9117 | 0.4775 | 0.2265 | 0.3911 | 0.391 |
69
+ | 1.5984 | 6.0 | 3126 | 0.9109 | 28.28 | 1.5468 | 0.9098 | 0.9124 | 0.4805 | 0.2284 | 0.3941 | 0.3938 |
70
+ | 1.5738 | 7.0 | 3647 | 1.5370 | 0.4807 | 0.2296 | 0.3945 | 0.3946 | 27.8378| 0.9105 | 0.9124 | 0.9113 |
71
+ | 1.5476 | 8.0 | 4168 | 1.5308 | 0.4823 | 0.2315 | 0.3963 | 0.3965 | 27.7364| 0.9108 | 0.9125 | 0.9114 |
72
+ | 1.535 | 9.0 | 4689 | 1.5261 | 0.4829 | 0.2309 | 0.3974 | 0.3974 | 27.6535| 0.911 | 0.9125 | 0.9116 |
73
+ | 1.52 | 10.0 | 5210 | 1.5231 | 0.4847 | 0.2332 | 0.3992 | 0.3993 | 27.816 | 0.911 | 0.9128 | 0.9117 |
74
+ | 1.5145 | 11.0 | 5731 | 1.5200 | 0.4851 | 0.2339 | 0.4004 | 0.4006 | 27.3604| 0.9119 | 0.9127 | 0.9121 |
75
+ | 1.5028 | 12.0 | 6252 | 1.5178 | 0.4858 | 0.2345 | 0.4001 | 0.4002 | 27.4625| 0.9118 | 0.9129 | 0.9122 |
76
+ | 1.4946 | 13.0 | 6773 | 1.5164 | 0.4859 | 0.2341 | 0.4004 | 0.4005 | 27.6789| 0.9115 | 0.9131 | 0.9121 |
77
+ | 1.4877 | 14.0 | 7294 | 1.5151 | 0.4868 | 0.235 | 0.4013 | 0.4013 | 27.5804| 0.9119 | 0.9131 | 0.9123 |
78
+ | 1.4855 | 15.0 | 7815 | 1.5146 | 0.4863 | 0.2349 | 0.4014 | 0.4016 | 27.5844| 0.9117 | 0.9131 | 0.9122 |
79
+ | 1.4782 | 16.0 | 8336 | 1.5146 | 0.4863 | 0.2348 | 0.4011 | 0.4012 | 27.5716| 0.9118 | 0.9131 | 0.9122 |
80
+
81
+
82
+ ### Framework versions
83
+
84
+ - Transformers 4.36.0
85
+ - Pytorch 2.0.1+cu117
86
+ - Datasets 2.14.5
87
+ - Tokenizers 0.15.0
generation_config.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 0,
3
+ "decoder_start_token_id": 0,
4
+ "eos_token_id": 1,
5
+ "forced_eos_token_id": 1,
6
+ "length_penalty": 0.8,
7
+ "max_length": 256,
8
+ "num_beams": 8,
9
+ "pad_token_id": 0,
10
+ "transformers_version": "4.36.0"
11
+ }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c9ad71b479c99492713448de0adc8768e069d59357d8b5c51ce622607d3577c9
3
  size 2283652852
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b49db019f36e1bffdeef6532ad2e2632c8c1f038357109277a309c597e3a6648
3
  size 2283652852
runs/Mar05_09-51-13_oi5vv8ctr1709312124223-tkfr5/events.out.tfevents.1709603487.oi5vv8ctr1709312124223-tkfr5.11611.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:88983b803440e6c0f4e9c1c44b09161935356dbb4dbab6d60975954307f7edef
3
- size 14476
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff5eb4d4f8737589407fad7b6790a4b4387edb87cf26f0bd013aa4092ec3e997
3
+ size 15504