Hugging Face's logo

habedi
/
deberta-v3-small-kaggle-mlm

Fill-Mask
Transformers
Safetensors
deberta-v2
Generated from Trainer
Inference Endpoints
Model card Files Community
habedi commited on
Commit
197bb38
•
1 Parent(s): e1c3f5b

Model save

Browse files
Files changed (2) hide show
  1. README.md +36 -7
  2. model.safetensors +1 -1
README.md CHANGED
@@ -15,7 +15,7 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  This model is a fine-tuned version of [microsoft/deberta-v3-small](https://huggingface.co/microsoft/deberta-v3-small) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 3.5180
19
 
20
  ## Model description
21
 
@@ -35,19 +35,48 @@ More information needed
35
 
36
  The following hyperparameters were used during training:
37
  - learning_rate: 1e-05
38
- - train_batch_size: 16
39
- - eval_batch_size: 16
40
  - seed: 42
41
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
  - lr_scheduler_type: linear
43
- - num_epochs: 1
44
  - mixed_precision_training: Native AMP
45
 
46
  ### Training results
47
 
48
- | Training Loss | Epoch | Step | Validation Loss |
49
- |:-------------:|:-----:|:----:|:---------------:|
50
- | 3.7367 | 1.0 | 3761 | 3.4957 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
51
 
52
 
53
  ### Framework versions
 
15
 
16
  This model is a fine-tuned version of [microsoft/deberta-v3-small](https://huggingface.co/microsoft/deberta-v3-small) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 1.6169
19
 
20
  ## Model description
21
 
 
35
 
36
  The following hyperparameters were used during training:
37
  - learning_rate: 1e-05
38
+ - train_batch_size: 8
39
+ - eval_batch_size: 8
40
  - seed: 42
41
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
  - lr_scheduler_type: linear
43
+ - num_epochs: 30
44
  - mixed_precision_training: Native AMP
45
 
46
  ### Training results
47
 
48
+ | Training Loss | Epoch | Step | Validation Loss |
49
+ |:-------------:|:-----:|:------:|:---------------:|
50
+ | 3.0931 | 1.0 | 6848 | 2.8467 |
51
+ | 2.6186 | 2.0 | 13696 | 2.4089 |
52
+ | 2.3498 | 3.0 | 20544 | 2.2224 |
53
+ | 2.2399 | 4.0 | 27392 | 2.1105 |
54
+ | 2.1226 | 5.0 | 34240 | 2.0204 |
55
+ | 2.0768 | 6.0 | 41088 | 1.9402 |
56
+ | 2.0251 | 7.0 | 47936 | 1.8767 |
57
+ | 1.9587 | 8.0 | 54784 | 1.8527 |
58
+ | 1.9209 | 9.0 | 61632 | 1.8108 |
59
+ | 1.8829 | 10.0 | 68480 | 1.8113 |
60
+ | 1.8454 | 11.0 | 75328 | 1.7698 |
61
+ | 1.8077 | 12.0 | 82176 | 1.7504 |
62
+ | 1.7991 | 13.0 | 89024 | 1.7390 |
63
+ | 1.7896 | 14.0 | 95872 | 1.7138 |
64
+ | 1.7608 | 15.0 | 102720 | 1.6847 |
65
+ | 1.7636 | 16.0 | 109568 | 1.6863 |
66
+ | 1.7416 | 17.0 | 116416 | 1.6816 |
67
+ | 1.7363 | 18.0 | 123264 | 1.6651 |
68
+ | 1.7013 | 19.0 | 130112 | 1.6465 |
69
+ | 1.6828 | 20.0 | 136960 | 1.6528 |
70
+ | 1.6889 | 21.0 | 143808 | 1.6406 |
71
+ | 1.6882 | 22.0 | 150656 | 1.6358 |
72
+ | 1.6742 | 23.0 | 157504 | 1.6338 |
73
+ | 1.6657 | 24.0 | 164352 | 1.6062 |
74
+ | 1.6685 | 25.0 | 171200 | 1.6086 |
75
+ | 1.6701 | 26.0 | 178048 | 1.6256 |
76
+ | 1.6755 | 27.0 | 184896 | 1.6186 |
77
+ | 1.6505 | 28.0 | 191744 | 1.6013 |
78
+ | 1.6573 | 29.0 | 198592 | 1.6108 |
79
+ | 1.6497 | 30.0 | 205440 | 1.6009 |
80
 
81
 
82
  ### Framework versions
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b3505dbb354eeaccb2efcf110daad8be3120a66a5b5b8b6dbe38c7171bc81e52
3
  size 568111128
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e9b8080018b4a0674f6777c826f4762d7452537cf24de83920bce081485b1e21
3
  size 568111128