AlekseyKorshuk commited on
Commit
7285628
1 Parent(s): a4f4e56

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +78 -0
README.md ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ tags:
4
+ - generated_from_trainer
5
+ metrics:
6
+ - accuracy
7
+ model-index:
8
+ - name: 6.7b-ri-reproduce-combined-4-gpu-20-val
9
+ results: []
10
+ ---
11
+
12
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
+ should probably proofread and complete it, then remove this comment. -->
14
+
15
+ # 6.7b-ri-reproduce-combined-4-gpu-20-val
16
+
17
+ This model is a fine-tuned version of [facebook/opt-6.7b](https://huggingface.co/facebook/opt-6.7b) on the None dataset.
18
+ It achieves the following results on the evaluation set:
19
+ - Loss: 3.9434
20
+ - Accuracy: 0.0329
21
+ - Perplexity: 51.5916
22
+
23
+ ## Model description
24
+
25
+ More information needed
26
+
27
+ ## Intended uses & limitations
28
+
29
+ More information needed
30
+
31
+ ## Training and evaluation data
32
+
33
+ More information needed
34
+
35
+ ## Training procedure
36
+
37
+ ### Training hyperparameters
38
+
39
+ The following hyperparameters were used during training:
40
+ - learning_rate: 9e-07
41
+ - train_batch_size: 1
42
+ - eval_batch_size: 8
43
+ - seed: 100
44
+ - distributed_type: multi-GPU
45
+ - num_devices: 4
46
+ - total_train_batch_size: 4
47
+ - total_eval_batch_size: 32
48
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
49
+ - lr_scheduler_type: constant
50
+ - num_epochs: 15.0
51
+
52
+ ### Training results
53
+
54
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy | Perplexity |
55
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|:----------:|
56
+ | 2.5731 | 1.0 | 79 | 2.6113 | 0.0317 | 13.6171 |
57
+ | 2.206 | 2.0 | 158 | 2.4805 | 0.0328 | 11.9469 |
58
+ | 1.9105 | 3.0 | 237 | 2.4512 | 0.0333 | 11.6019 |
59
+ | 1.6301 | 4.0 | 316 | 2.5078 | 0.0345 | 12.2780 |
60
+ | 1.3733 | 5.0 | 395 | 2.6816 | 0.0342 | 14.6090 |
61
+ | 1.1337 | 6.0 | 474 | 3.0078 | 0.0330 | 20.2431 |
62
+ | 0.9619 | 7.0 | 553 | 3.1777 | 0.0330 | 23.9923 |
63
+ | 0.798 | 8.0 | 632 | 3.2559 | 0.0330 | 25.9419 |
64
+ | 0.6653 | 9.0 | 711 | 3.4277 | 0.0331 | 30.8068 |
65
+ | 0.552 | 10.0 | 790 | 3.5566 | 0.0333 | 35.0453 |
66
+ | 0.4568 | 11.0 | 869 | 3.7324 | 0.0324 | 41.7802 |
67
+ | 0.3756 | 12.0 | 948 | 3.8184 | 0.0328 | 45.5295 |
68
+ | 0.3119 | 13.0 | 1027 | 3.8477 | 0.0331 | 46.8831 |
69
+ | 0.2448 | 14.0 | 1106 | 3.9062 | 0.0329 | 49.7122 |
70
+ | 0.1986 | 15.0 | 1185 | 3.9434 | 0.0329 | 51.5916 |
71
+
72
+
73
+ ### Framework versions
74
+
75
+ - Transformers 4.25.0.dev0
76
+ - Pytorch 1.12.1+cu113
77
+ - Datasets 2.3.2
78
+ - Tokenizers 0.12.1