Regression_bert_10

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

  • Train Loss: 0.0535
  • Train Mae: 0.2673
  • Train Mse: 0.1031
  • Train R2-score: 0.6896
  • Validation Loss: 0.1142
  • Validation Mae: 0.3549
  • Validation Mse: 0.1957
  • Validation R2-score: 0.9230
  • Epoch: 9

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 1e-04, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
  • training_precision: float32

Training results

Train Loss Train Mae Train Mse Train R2-score Validation Loss Validation Mae Validation Mse Validation R2-score Epoch
0.2988 0.4759 0.3361 0.6079 0.1967 0.3939 0.2542 0.9026 0
0.1715 0.4010 0.2357 0.6812 0.1680 0.4014 0.2478 0.9049 1
0.0903 0.3374 0.1532 0.8384 0.1354 0.3432 0.1971 0.9210 2
0.0636 0.3139 0.1272 0.4117 0.1538 0.4066 0.2304 0.9034 3
0.0746 0.3142 0.1294 0.9220 0.1184 0.3589 0.2015 0.9224 4
0.0604 0.2837 0.1119 0.9439 0.1268 0.3450 0.1994 0.9209 5
0.0556 0.2660 0.1049 0.6002 0.1193 0.3037 0.1704 0.9265 6
0.0541 0.2581 0.1007 0.8081 0.1125 0.3350 0.1743 0.9229 7
0.0532 0.2679 0.1044 0.8917 0.1109 0.3131 0.1757 0.9311 8
0.0535 0.2673 0.1031 0.6896 0.1142 0.3549 0.1957 0.9230 9

Framework versions

  • Transformers 4.27.4
  • TensorFlow 2.12.0
  • Datasets 2.11.0
  • Tokenizers 0.13.2
Downloads last month
62
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.