Hugging Face's logo

Vishnou
/
TinyBERT_SST2

Text Classification
Transformers
Safetensors
bert
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
Vishnou commited on
Commit
c94f474
•
1 Parent(s): a55da62

Vishnou/TinyBERT_SST2

Browse files
Files changed (2) hide show
  1. README.md +38 -37
  2. model.safetensors +1 -1
README.md CHANGED
@@ -1,4 +1,5 @@
1
  ---
 
2
  tags:
3
  - generated_from_trainer
4
  datasets:
@@ -20,7 +21,7 @@ model-index:
20
  metrics:
21
  - name: Accuracy
22
  type: accuracy
23
- value: 0.8428899082568807
24
  ---
25
 
26
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -28,10 +29,10 @@ should probably proofread and complete it, then remove this comment. -->
28
 
29
  # TinyBERT_SST2
30
 
31
- This model was trained from scratch on the sst2 dataset.
32
  It achieves the following results on the evaluation set:
33
- - Loss: 1.1843
34
- - Accuracy: 0.8429
35
 
36
  ## Model description
37
 
@@ -62,39 +63,39 @@ The following hyperparameters were used during training:
62
 
63
  | Training Loss | Epoch | Step | Validation Loss | Accuracy |
64
  |:-------------:|:-----:|:-----:|:---------------:|:--------:|
65
- | 0.012 | 0.06 | 500 | 1.4602 | 0.8475 |
66
- | 0.0111 | 0.12 | 1000 | 1.4848 | 0.8475 |
67
- | 0.0277 | 0.18 | 1500 | 1.5532 | 0.8452 |
68
- | 0.0291 | 0.24 | 2000 | 1.4006 | 0.8440 |
69
- | 0.0283 | 0.3 | 2500 | 1.4589 | 0.8406 |
70
- | 0.0361 | 0.36 | 3000 | 1.2831 | 0.8429 |
71
- | 0.0261 | 0.42 | 3500 | 1.3951 | 0.8417 |
72
- | 0.04 | 0.48 | 4000 | 1.3990 | 0.8245 |
73
- | 0.0333 | 0.53 | 4500 | 1.1859 | 0.8463 |
74
- | 0.0475 | 0.59 | 5000 | 1.1699 | 0.8486 |
75
- | 0.0304 | 0.65 | 5500 | 1.2672 | 0.8394 |
76
- | 0.0323 | 0.71 | 6000 | 1.3541 | 0.8440 |
77
- | 0.0482 | 0.77 | 6500 | 1.2858 | 0.8417 |
78
- | 0.0393 | 0.83 | 7000 | 1.2595 | 0.8463 |
79
- | 0.0371 | 0.89 | 7500 | 1.2028 | 0.8314 |
80
- | 0.0444 | 0.95 | 8000 | 1.1606 | 0.8440 |
81
- | 0.0407 | 1.01 | 8500 | 1.2363 | 0.8406 |
82
- | 0.0238 | 1.07 | 9000 | 1.2556 | 0.8475 |
83
- | 0.0253 | 1.13 | 9500 | 1.2557 | 0.8475 |
84
- | 0.0234 | 1.19 | 10000 | 1.2927 | 0.8521 |
85
- | 0.0293 | 1.25 | 10500 | 1.3345 | 0.8383 |
86
- | 0.0235 | 1.31 | 11000 | 1.3742 | 0.8349 |
87
- | 0.026 | 1.37 | 11500 | 1.3648 | 0.8337 |
88
- | 0.0359 | 1.43 | 12000 | 1.3063 | 0.8337 |
89
- | 0.0225 | 1.48 | 12500 | 1.3475 | 0.8360 |
90
- | 0.0274 | 1.54 | 13000 | 1.3568 | 0.8337 |
91
- | 0.0304 | 1.6 | 13500 | 1.3533 | 0.8372 |
92
- | 0.0534 | 1.66 | 14000 | 1.2560 | 0.8417 |
93
- | 0.0379 | 1.72 | 14500 | 1.2770 | 0.8417 |
94
- | 0.0678 | 1.78 | 15000 | 1.1950 | 0.8429 |
95
- | 0.0488 | 1.84 | 15500 | 1.1796 | 0.8440 |
96
- | 0.0598 | 1.9 | 16000 | 1.1650 | 0.8452 |
97
- | 0.0565 | 1.96 | 16500 | 1.1843 | 0.8429 |
98
 
99
 
100
  ### Framework versions
 
1
  ---
2
+ base_model: huawei-noah/TinyBERT_General_4L_312D
3
  tags:
4
  - generated_from_trainer
5
  datasets:
 
21
  metrics:
22
  - name: Accuracy
23
  type: accuracy
24
+ value: 0.875
25
  ---
26
 
27
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
29
 
30
  # TinyBERT_SST2
31
 
32
+ This model is a fine-tuned version of [huawei-noah/TinyBERT_General_4L_312D](https://huggingface.co/huawei-noah/TinyBERT_General_4L_312D) on the sst2 dataset.
33
  It achieves the following results on the evaluation set:
34
+ - Loss: 0.5560
35
+ - Accuracy: 0.875
36
 
37
  ## Model description
38
 
 
63
 
64
  | Training Loss | Epoch | Step | Validation Loss | Accuracy |
65
  |:-------------:|:-----:|:-----:|:---------------:|:--------:|
66
+ | 0.4661 | 0.06 | 500 | 0.3888 | 0.8337 |
67
+ | 0.3684 | 0.12 | 1000 | 0.3557 | 0.8544 |
68
+ | 0.3857 | 0.18 | 1500 | 0.3839 | 0.8544 |
69
+ | 0.3616 | 0.24 | 2000 | 0.3700 | 0.8670 |
70
+ | 0.3559 | 0.3 | 2500 | 0.3586 | 0.8544 |
71
+ | 0.3501 | 0.36 | 3000 | 0.3886 | 0.8498 |
72
+ | 0.3232 | 0.42 | 3500 | 0.4819 | 0.8624 |
73
+ | 0.3178 | 0.48 | 4000 | 0.5071 | 0.8452 |
74
+ | 0.2969 | 0.53 | 4500 | 0.4325 | 0.8578 |
75
+ | 0.3162 | 0.59 | 5000 | 0.4296 | 0.8635 |
76
+ | 0.2995 | 0.65 | 5500 | 0.5547 | 0.8463 |
77
+ | 0.3016 | 0.71 | 6000 | 0.4364 | 0.8670 |
78
+ | 0.2973 | 0.77 | 6500 | 0.4595 | 0.8555 |
79
+ | 0.3068 | 0.83 | 7000 | 0.4519 | 0.8670 |
80
+ | 0.2917 | 0.89 | 7500 | 0.4175 | 0.8716 |
81
+ | 0.2819 | 0.95 | 8000 | 0.4741 | 0.8739 |
82
+ | 0.2711 | 1.01 | 8500 | 0.5015 | 0.8842 |
83
+ | 0.2173 | 1.07 | 9000 | 0.4956 | 0.8830 |
84
+ | 0.2259 | 1.13 | 9500 | 0.6080 | 0.8761 |
85
+ | 0.2655 | 1.19 | 10000 | 0.5456 | 0.8807 |
86
+ | 0.2499 | 1.25 | 10500 | 0.5349 | 0.8796 |
87
+ | 0.2291 | 1.31 | 11000 | 0.5214 | 0.8784 |
88
+ | 0.2207 | 1.37 | 11500 | 0.5743 | 0.8853 |
89
+ | 0.2463 | 1.43 | 12000 | 0.5499 | 0.8761 |
90
+ | 0.2214 | 1.48 | 12500 | 0.5270 | 0.8819 |
91
+ | 0.2114 | 1.54 | 13000 | 0.5762 | 0.8727 |
92
+ | 0.2087 | 1.6 | 13500 | 0.5400 | 0.8819 |
93
+ | 0.2123 | 1.66 | 14000 | 0.5719 | 0.8796 |
94
+ | 0.2112 | 1.72 | 14500 | 0.5236 | 0.8819 |
95
+ | 0.2042 | 1.78 | 15000 | 0.5373 | 0.8807 |
96
+ | 0.2176 | 1.84 | 15500 | 0.5504 | 0.8853 |
97
+ | 0.2032 | 1.9 | 16000 | 0.5701 | 0.8761 |
98
+ | 0.213 | 1.96 | 16500 | 0.5560 | 0.875 |
99
 
100
 
101
  ### Framework versions
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c67a1a46b102966699da487ee8b0f96ee5b2d6af0dfe7371dab41ef5872b55eb
3
  size 57411808
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0cda1109d7145e88bd53032a1945a62fedcafd8aa50a53b4404c15acd36b5fa5
3
  size 57411808