Hugging Face's logo

Zemulax
/
masked-lm-tpu

Fill-Mask
Transformers
TensorFlow
roberta
generated_from_keras_callback
Inference Endpoints
Model card Files Community
masked-lm-tpu / README.md
Zemulax's picture
Zemulax
Training in progress epoch 98
4f2f573
preview code
|
raw
history blame contribute delete
9.62 kB
metadata
license: mit
tags:
  - generated_from_keras_callback
model-index:
  - name: Zemulax/masked-lm-tpu
    results: []

Zemulax/masked-lm-tpu

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Train Loss: 7.7770
  • Train Accuracy: 0.0241
  • Validation Loss: 7.7589
  • Validation Accuracy: 0.0230
  • Epoch: 98

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'WarmUp', 'config': {'initial_learning_rate': 0.0001, 'decay_schedule_fn': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 0.0001, 'decay_steps': 223250, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'passive_serialization': True}, 'warmup_steps': 11750, 'power': 1.0, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.001}
  • training_precision: float32

Training results

Train Loss Train Accuracy Validation Loss Validation Accuracy Epoch
10.2868 0.0 10.2891 0.0 0
10.2817 0.0000 10.2764 0.0 1
10.2772 0.0000 10.2667 0.0000 2
10.2604 0.0000 10.2521 0.0 3
10.2421 0.0000 10.2282 0.0000 4
10.2219 0.0 10.2010 0.0 5
10.1957 0.0 10.1669 0.0 6
10.1667 0.0000 10.1388 0.0000 7
10.1278 0.0000 10.0908 0.0000 8
10.0848 0.0000 10.0405 0.0001 9
10.0496 0.0002 9.9921 0.0007 10
9.9940 0.0010 9.9422 0.0039 11
9.9424 0.0035 9.8765 0.0110 12
9.8826 0.0092 9.8156 0.0182 13
9.8225 0.0155 9.7461 0.0209 14
9.7670 0.0201 9.6768 0.0222 15
9.7065 0.0219 9.6127 0.0222 16
9.6352 0.0227 9.5445 0.0220 17
9.5757 0.0226 9.4795 0.0219 18
9.4894 0.0232 9.3985 0.0222 19
9.4277 0.0234 9.3386 0.0222 20
9.3676 0.0229 9.2753 0.0220 21
9.2980 0.0229 9.2170 0.0219 22
9.2361 0.0233 9.1518 0.0219 23
9.1515 0.0236 9.0827 0.0223 24
9.1171 0.0228 9.0406 0.0218 25
9.0447 0.0234 8.9867 0.0218 26
9.0119 0.0229 8.9307 0.0221 27
8.9625 0.0229 8.8969 0.0221 28
8.9098 0.0230 8.8341 0.0223 29
8.8726 0.0227 8.8118 0.0220 30
8.8574 0.0223 8.7910 0.0219 31
8.7798 0.0231 8.7506 0.0221 32
8.7535 0.0231 8.7055 0.0222 33
8.7333 0.0228 8.6801 0.0223 34
8.6985 0.0231 8.6837 0.0220 35
8.6816 0.0229 8.6243 0.0223 36
8.6356 0.0228 8.6323 0.0217 37
8.6392 0.0225 8.5603 0.0225 38
8.5802 0.0233 8.5722 0.0219 39
8.5825 0.0228 8.5548 0.0220 40
8.5625 0.0228 8.5272 0.0220 41
8.5415 0.0228 8.5200 0.0222 42
8.5124 0.0230 8.4787 0.0222 43
8.4999 0.0229 8.4819 0.0218 44
8.4561 0.0235 8.4453 0.0221 45
8.4854 0.0223 8.4378 0.0220 46
8.4367 0.0229 8.4212 0.0222 47
8.4096 0.0232 8.4033 0.0221 48
8.4162 0.0228 8.3869 0.0221 49
8.4005 0.0229 8.3768 0.0218 50
8.3583 0.0235 8.3470 0.0224 51
8.3428 0.0235 8.3540 0.0221 52
8.3491 0.0231 8.3201 0.0225 53
8.3551 0.0231 8.3382 0.0221 54
8.3186 0.0231 8.3136 0.0219 55
8.3139 0.0226 8.2844 0.0222 56
8.3170 0.0229 8.2740 0.0221 57
8.2886 0.0231 8.2485 0.0223 58
8.2648 0.0233 8.2336 0.0223 59
8.2714 0.0225 8.2321 0.0221 60
8.2446 0.0233 8.2135 0.0223 61
8.2303 0.0230 8.1980 0.0223 62
8.2022 0.0237 8.1996 0.0222 63
8.2222 0.0227 8.1822 0.0222 64
8.1690 0.0236 8.2005 0.0220 65
8.1741 0.0233 8.1446 0.0226 66
8.1990 0.0224 8.1586 0.0219 67
8.1395 0.0236 8.1243 0.0225 68
8.1675 0.0229 8.1275 0.0222 69
8.1432 0.0229 8.1374 0.0217 70
8.1197 0.0234 8.1078 0.0221 71
8.1046 0.0232 8.0991 0.0221 72
8.1013 0.0231 8.0794 0.0222 73
8.0887 0.0228 8.0720 0.0221 74
8.0661 0.0233 8.0573 0.0222 75
8.0548 0.0231 8.0313 0.0226 76
8.0307 0.0235 8.0278 0.0222 77
8.0626 0.0226 8.0084 0.0224 78
8.0276 0.0229 8.0099 0.0221 79
8.0213 0.0231 7.9930 0.0222 80
7.9798 0.0237 7.9742 0.0224 81
8.0135 0.0226 7.9857 0.0218 82
7.9500 0.0235 7.9505 0.0223 83
7.9519 0.0234 7.9711 0.0217 84
7.9616 0.0228 7.9288 0.0223 85
7.9803 0.0225 7.8997 0.0226 86
7.9369 0.0227 7.9015 0.0225 87
7.9309 0.0229 7.9010 0.0224 88
7.9367 0.0226 7.8988 0.0220 89
7.8840 0.0230 7.8774 0.0216 90
7.8785 0.0233 7.8527 0.0225 91
7.8998 0.0226 7.8509 0.0219 92
7.8451 0.0232 7.8488 0.0221 93
7.8596 0.0231 7.8310 0.0222 94
7.8434 0.0231 7.8168 0.0229 95
7.7929 0.0238 7.7815 0.0233 96
7.8174 0.0236 7.7857 0.0232 97
7.7770 0.0241 7.7589 0.0230 98

Framework versions

  • Transformers 4.30.1
  • TensorFlow 2.12.0
  • Tokenizers 0.13.3