Hugging Face's logo

UNIST-Eunchan
/
Pegasus-x-base-govreport-12288-1024-numepoch-5

Text2Text Generation
Transformers
PyTorch
pegasus_x
Generated from Trainer
Inference Endpoints
Model card Files Community
1
UNIST-Eunchan commited on
Commit
c0e1178
·
1 Parent(s): 2213291

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +126 -0
README.md ADDED
@@ -0,0 +1,126 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ datasets:
5
+ - govreport-summarization
6
+ model-index:
7
+ - name: Pegasus-x-base-govreport-12288-1024-numepoch-5
8
+ results: []
9
+ ---
10
+
11
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
+ should probably proofread and complete it, then remove this comment. -->
13
+
14
+ # Pegasus-x-base-govreport-12288-1024-numepoch-5
15
+
16
+ This model is a fine-tuned version of [google/pegasus-x-base](https://huggingface.co/google/pegasus-x-base) on the govreport-summarization dataset.
17
+ It achieves the following results on the evaluation set:
18
+ - Loss: 1.6740
19
+
20
+ ## Model description
21
+
22
+ More information needed
23
+
24
+ ## Intended uses & limitations
25
+
26
+ More information needed
27
+
28
+ ## Training and evaluation data
29
+
30
+ More information needed
31
+
32
+ ## Training procedure
33
+
34
+ ### Training hyperparameters
35
+
36
+ The following hyperparameters were used during training:
37
+ - learning_rate: 0.0001
38
+ - train_batch_size: 1
39
+ - eval_batch_size: 2
40
+ - seed: 42
41
+ - gradient_accumulation_steps: 64
42
+ - total_train_batch_size: 64
43
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
+ - lr_scheduler_type: linear
45
+ - num_epochs: 5
46
+
47
+ ### Training results
48
+
49
+ | Training Loss | Epoch | Step | Validation Loss |
50
+ |:-------------:|:-----:|:----:|:---------------:|
51
+ | 3.0173 | 0.07 | 20 | 2.6677 |
52
+ | 2.5674 | 0.15 | 40 | 2.2993 |
53
+ | 2.3013 | 0.22 | 60 | 2.1024 |
54
+ | 2.2145 | 0.29 | 80 | 1.9833 |
55
+ | 2.1191 | 0.37 | 100 | 1.9383 |
56
+ | 2.0709 | 0.44 | 120 | 1.8815 |
57
+ | 2.0287 | 0.51 | 140 | 1.8623 |
58
+ | 2.003 | 0.58 | 160 | 1.8467 |
59
+ | 1.9842 | 0.66 | 180 | 1.8314 |
60
+ | 1.9603 | 0.73 | 200 | 1.8307 |
61
+ | 1.9493 | 0.8 | 220 | 1.8157 |
62
+ | 1.9631 | 0.88 | 240 | 1.7919 |
63
+ | 1.9332 | 0.95 | 260 | 1.7919 |
64
+ | 1.9123 | 1.02 | 280 | 1.7836 |
65
+ | 1.887 | 1.1 | 300 | 1.7672 |
66
+ | 1.8743 | 1.17 | 320 | 1.7629 |
67
+ | 1.8412 | 1.24 | 340 | 1.7566 |
68
+ | 1.8508 | 1.32 | 360 | 1.7410 |
69
+ | 1.8564 | 1.39 | 380 | 1.7403 |
70
+ | 1.8686 | 1.46 | 400 | 1.7393 |
71
+ | 1.8881 | 1.53 | 420 | 1.7420 |
72
+ | 1.8629 | 1.61 | 440 | 1.7367 |
73
+ | 1.8683 | 1.68 | 460 | 1.7288 |
74
+ | 1.833 | 1.75 | 480 | 1.7300 |
75
+ | 1.8621 | 1.83 | 500 | 1.7208 |
76
+ | 1.8622 | 1.9 | 520 | 1.7211 |
77
+ | 1.8147 | 1.97 | 540 | 1.7158 |
78
+ | 1.8161 | 2.05 | 560 | 1.7117 |
79
+ | 1.8239 | 2.12 | 580 | 1.7090 |
80
+ | 1.8185 | 2.19 | 600 | 1.7100 |
81
+ | 1.8605 | 2.27 | 620 | 1.7057 |
82
+ | 1.7919 | 2.34 | 640 | 1.6996 |
83
+ | 1.8026 | 2.41 | 660 | 1.7012 |
84
+ | 1.7785 | 2.48 | 680 | 1.6980 |
85
+ | 1.8296 | 2.56 | 700 | 1.6941 |
86
+ | 1.802 | 2.63 | 720 | 1.6944 |
87
+ | 1.7783 | 2.7 | 740 | 1.6927 |
88
+ | 1.7998 | 2.78 | 760 | 1.6922 |
89
+ | 1.8128 | 2.85 | 780 | 1.6890 |
90
+ | 1.7762 | 2.92 | 800 | 1.6909 |
91
+ | 1.7631 | 3.0 | 820 | 1.6959 |
92
+ | 1.8191 | 3.07 | 840 | 1.6823 |
93
+ | 1.795 | 3.14 | 860 | 1.6873 |
94
+ | 1.7587 | 3.22 | 880 | 1.6850 |
95
+ | 1.8091 | 3.29 | 900 | 1.6828 |
96
+ | 1.7617 | 3.36 | 920 | 1.6860 |
97
+ | 1.7933 | 3.43 | 940 | 1.6796 |
98
+ | 1.8041 | 3.51 | 960 | 1.6805 |
99
+ | 1.7596 | 3.58 | 980 | 1.6855 |
100
+ | 1.7518 | 3.65 | 1000 | 1.6791 |
101
+ | 1.7384 | 3.73 | 1020 | 1.6795 |
102
+ | 1.7855 | 3.8 | 1040 | 1.6784 |
103
+ | 1.7938 | 3.87 | 1060 | 1.6780 |
104
+ | 1.7637 | 3.95 | 1080 | 1.6809 |
105
+ | 1.7914 | 4.02 | 1100 | 1.6779 |
106
+ | 1.7903 | 4.09 | 1120 | 1.6753 |
107
+ | 1.7874 | 4.17 | 1140 | 1.6745 |
108
+ | 1.7982 | 4.24 | 1160 | 1.6728 |
109
+ | 1.7709 | 4.31 | 1180 | 1.6761 |
110
+ | 1.7583 | 4.38 | 1200 | 1.6754 |
111
+ | 1.778 | 4.46 | 1220 | 1.6739 |
112
+ | 1.7526 | 4.53 | 1240 | 1.6746 |
113
+ | 1.7713 | 4.6 | 1260 | 1.6723 |
114
+ | 1.734 | 4.68 | 1280 | 1.6742 |
115
+ | 1.7498 | 4.75 | 1300 | 1.6737 |
116
+ | 1.751 | 4.82 | 1320 | 1.6730 |
117
+ | 1.7562 | 4.9 | 1340 | 1.6739 |
118
+ | 1.7549 | 4.97 | 1360 | 1.6740 |
119
+
120
+
121
+ ### Framework versions
122
+
123
+ - Transformers 4.30.2
124
+ - Pytorch 2.0.1+cu117
125
+ - Datasets 2.13.1
126
+ - Tokenizers 0.13.3