bigmorning's picture
add model
d5e12e0
metadata
license: apache-2.0
tags:
  - generated_from_keras_callback
model-index:
  - name: distilbert_oscarth_0080
    results: []

distilbert_oscarth_0080

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

  • Train Loss: 1.1236
  • Validation Loss: 1.0821
  • Epoch: 79

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
  • training_precision: float32

Training results

Train Loss Validation Loss Epoch
4.1327 2.9983 0
2.7813 2.4562 1
2.4194 2.2066 2
2.2231 2.0562 3
2.0894 1.9450 4
1.9905 1.8621 5
1.9148 1.7941 6
1.8508 1.7363 7
1.7976 1.6909 8
1.7509 1.6488 9
1.7126 1.6124 10
1.6764 1.5835 11
1.6450 1.5521 12
1.6175 1.5282 13
1.5919 1.5045 14
1.5679 1.4833 15
1.5476 1.4627 16
1.5271 1.4498 17
1.5098 1.4270 18
1.4909 1.4161 19
1.4760 1.3995 20
1.4609 1.3864 21
1.4475 1.3717 22
1.4333 1.3590 23
1.4203 1.3478 24
1.4093 1.3403 25
1.3980 1.3296 26
1.3875 1.3176 27
1.3773 1.3094 28
1.3674 1.3011 29
1.3579 1.2920 30
1.3497 1.2826 31
1.3400 1.2764 32
1.3326 1.2694 33
1.3236 1.2635 34
1.3169 1.2536 35
1.3096 1.2477 36
1.3024 1.2408 37
1.2957 1.2364 38
1.2890 1.2296 39
1.2818 1.2236 40
1.2751 1.2168 41
1.2691 1.2126 42
1.2644 1.2044 43
1.2583 1.2008 44
1.2529 1.1962 45
1.2473 1.1919 46
1.2416 1.1857 47
1.2365 1.1812 48
1.2318 1.1765 49
1.2273 1.1738 50
1.2224 1.1672 51
1.2177 1.1673 52
1.2132 1.1595 53
1.2084 1.1564 54
1.2033 1.1518 55
1.1993 1.1481 56
1.1966 1.1445 57
1.1924 1.1412 58
1.1876 1.1378 59
1.1834 1.1340 60
1.1806 1.1329 61
1.1783 1.1289 62
1.1739 1.1251 63
1.1705 1.1223 64
1.1669 1.1192 65
1.1628 1.1172 66
1.1599 1.1140 67
1.1570 1.1084 68
1.1526 1.1081 69
1.1496 1.1043 70
1.1463 1.0999 71
1.1438 1.1006 72
1.1397 1.0964 73
1.1378 1.0918 74
1.1347 1.0917 75
1.1319 1.0889 76
1.1296 1.0855 77
1.1271 1.0848 78
1.1236 1.0821 79

Framework versions

  • Transformers 4.20.1
  • TensorFlow 2.8.2
  • Datasets 2.3.2
  • Tokenizers 0.12.1