distilbert-base-uncased-finetuned

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 7.2813

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 30
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
8.6309 1.0 76 7.4774
7.0806 2.0 152 6.9937
6.6842 3.0 228 6.9314
6.4592 4.0 304 6.9088
6.2936 5.0 380 6.9135
6.1301 6.0 456 6.9018
5.9878 7.0 532 6.8865
5.8071 8.0 608 6.8926
5.6372 9.0 684 6.8750
5.4791 10.0 760 6.9394
5.3365 11.0 836 6.9594
5.2117 12.0 912 6.9962
5.0887 13.0 988 7.0570
4.9288 14.0 1064 7.0549
4.8169 15.0 1140 7.0971
4.7008 16.0 1216 7.1439
4.6149 17.0 1292 7.1320
4.487 18.0 1368 7.1577
4.364 19.0 1444 7.1712
4.3208 20.0 1520 7.1959
4.2492 21.0 1596 7.2136
4.1423 22.0 1672 7.2304
4.0873 23.0 1748 7.2526
4.0261 24.0 1824 7.2681
3.9598 25.0 1900 7.2715
3.9562 26.0 1976 7.2648
3.8951 27.0 2052 7.2665
3.8772 28.0 2128 7.2781
3.8403 29.0 2204 7.2801
3.8275 30.0 2280 7.2813

Framework versions

  • Transformers 4.21.3
  • Pytorch 1.12.1
  • Datasets 2.4.0
  • Tokenizers 0.12.1
Downloads last month
25
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.