Hugging Face's logo

AlekseyKorshuk
/
1.3b-all-2-epoch-v1-after-book

Text Generation
Transformers
PyTorch
opt
Generated from Trainer
Eval Results
Inference Endpoints
text-generation-inference
Model card Files Community
AlekseyKorshuk commited on
Commit
8b08045
•
1 Parent(s): 708dd92

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +91 -0
README.md ADDED
@@ -0,0 +1,91 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ metrics:
5
+ - accuracy
6
+ model-index:
7
+ - name: 1.3b-all-2-epoch-v1-after-book
8
+ results: []
9
+ ---
10
+
11
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
+ should probably proofread and complete it, then remove this comment. -->
13
+
14
+ # 1.3b-all-2-epoch-v1-after-book
15
+
16
+ This model was trained from scratch on an unknown dataset.
17
+ It achieves the following results on the evaluation set:
18
+ - Loss: 1.9482
19
+ - Accuracy: 0.0640
20
+
21
+ ## Model description
22
+
23
+ More information needed
24
+
25
+ ## Intended uses & limitations
26
+
27
+ More information needed
28
+
29
+ ## Training and evaluation data
30
+
31
+ More information needed
32
+
33
+ ## Training procedure
34
+
35
+ ### Training hyperparameters
36
+
37
+ The following hyperparameters were used during training:
38
+ - learning_rate: 3e-05
39
+ - train_batch_size: 4
40
+ - eval_batch_size: 4
41
+ - seed: 42
42
+ - distributed_type: multi-GPU
43
+ - num_devices: 8
44
+ - total_train_batch_size: 32
45
+ - total_eval_batch_size: 32
46
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
47
+ - lr_scheduler_type: cosine
48
+ - num_epochs: 2.0
49
+
50
+ ### Training results
51
+
52
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy |
53
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|
54
+ | 2.17 | 0.07 | 1 | 2.0547 | 0.0621 |
55
+ | 2.1814 | 0.13 | 2 | 2.0547 | 0.0621 |
56
+ | 2.0963 | 0.2 | 3 | 2.0234 | 0.0625 |
57
+ | 2.1383 | 0.27 | 4 | 2.0195 | 0.0625 |
58
+ | 2.1625 | 0.33 | 5 | 2.0195 | 0.0625 |
59
+ | 2.1808 | 0.4 | 6 | 2.0156 | 0.0624 |
60
+ | 2.1587 | 0.47 | 7 | 2.0176 | 0.0626 |
61
+ | 2.0847 | 0.53 | 8 | 2.0137 | 0.0627 |
62
+ | 2.0336 | 0.6 | 9 | 2.0137 | 0.0627 |
63
+ | 2.1777 | 0.67 | 10 | 2.0059 | 0.0629 |
64
+ | 2.2034 | 0.73 | 11 | 2.0 | 0.0630 |
65
+ | 2.1665 | 0.8 | 12 | 1.9941 | 0.0628 |
66
+ | 2.0352 | 0.87 | 13 | 1.9883 | 0.0629 |
67
+ | 2.1263 | 0.93 | 14 | 1.9834 | 0.0628 |
68
+ | 2.1282 | 1.0 | 15 | 1.9785 | 0.0632 |
69
+ | 1.7159 | 1.07 | 16 | 1.9766 | 0.0633 |
70
+ | 1.8346 | 1.13 | 17 | 1.9775 | 0.0635 |
71
+ | 1.7183 | 1.2 | 18 | 1.9824 | 0.0634 |
72
+ | 1.6086 | 1.27 | 19 | 1.9883 | 0.0635 |
73
+ | 1.6497 | 1.33 | 20 | 1.9893 | 0.0634 |
74
+ | 1.6267 | 1.4 | 21 | 1.9854 | 0.0637 |
75
+ | 1.5962 | 1.47 | 22 | 1.9766 | 0.0637 |
76
+ | 1.5168 | 1.53 | 23 | 1.9697 | 0.0637 |
77
+ | 1.6213 | 1.6 | 24 | 1.9619 | 0.0637 |
78
+ | 1.4789 | 1.67 | 25 | 1.9580 | 0.0638 |
79
+ | 1.6796 | 1.73 | 26 | 1.9551 | 0.0638 |
80
+ | 1.5964 | 1.8 | 27 | 1.9531 | 0.0638 |
81
+ | 1.787 | 1.87 | 28 | 1.9512 | 0.0639 |
82
+ | 1.6536 | 1.93 | 29 | 1.9492 | 0.0640 |
83
+ | 1.7178 | 2.0 | 30 | 1.9482 | 0.0640 |
84
+
85
+
86
+ ### Framework versions
87
+
88
+ - Transformers 4.25.0.dev0
89
+ - Pytorch 1.12.1+cu113
90
+ - Datasets 2.3.2
91
+ - Tokenizers 0.12.1