Edit model card

5e06_output_dir_clean_df_10-100_noX_100_50_epoch_cluster

This model is a fine-tuned version of nferruz/ProtGPT2 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 4.4642
  • Accuracy: 0.3637

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-06
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 50.0

Training results

Training Loss Epoch Step Validation Loss Accuracy
No log 1.0 148 6.0108 0.2185
No log 2.0 296 5.8304 0.2297
No log 3.0 444 5.6868 0.2394
5.9726 4.0 592 5.5732 0.2492
5.9726 5.0 740 5.4748 0.2587
5.9726 6.0 888 5.3871 0.2673
5.5397 7.0 1036 5.3085 0.2756
5.5397 8.0 1184 5.2401 0.2827
5.5397 9.0 1332 5.1811 0.2888
5.5397 10.0 1480 5.1277 0.2933
5.2883 11.0 1628 5.0796 0.2983
5.2883 12.0 1776 5.0358 0.3030
5.2883 13.0 1924 4.9951 0.3067
5.1076 14.0 2072 4.9572 0.3103
5.1076 15.0 2220 4.9200 0.3139
5.1076 16.0 2368 4.8877 0.3172
4.9674 17.0 2516 4.8551 0.3203
4.9674 18.0 2664 4.8258 0.3232
4.9674 19.0 2812 4.8008 0.3265
4.9674 20.0 2960 4.7743 0.3289
4.858 21.0 3108 4.7497 0.3317
4.858 22.0 3256 4.7271 0.3338
4.858 23.0 3404 4.7058 0.3363
4.76 24.0 3552 4.6866 0.3384
4.76 25.0 3700 4.6684 0.3404
4.76 26.0 3848 4.6486 0.3425
4.76 27.0 3996 4.6323 0.3443
4.6863 28.0 4144 4.6155 0.3459
4.6863 29.0 4292 4.6016 0.3476
4.6863 30.0 4440 4.5874 0.3490
4.6168 31.0 4588 4.5742 0.3505
4.6168 32.0 4736 4.5628 0.3518
4.6168 33.0 4884 4.5507 0.3534
4.5684 34.0 5032 4.5412 0.3543
4.5684 35.0 5180 4.5316 0.3558
4.5684 36.0 5328 4.5207 0.3570
4.5684 37.0 5476 4.5132 0.3580
4.5277 38.0 5624 4.5054 0.3588
4.5277 39.0 5772 4.4993 0.3597
4.5277 40.0 5920 4.4931 0.3604
4.4886 41.0 6068 4.4879 0.3611
4.4886 42.0 6216 4.4821 0.3617
4.4886 43.0 6364 4.4778 0.3622
4.4727 44.0 6512 4.4741 0.3626
4.4727 45.0 6660 4.4710 0.3630
4.4727 46.0 6808 4.4691 0.3633
4.4727 47.0 6956 4.4664 0.3634
4.4542 48.0 7104 4.4652 0.3636
4.4542 49.0 7252 4.4644 0.3637
4.4542 50.0 7400 4.4642 0.3637

Framework versions

  • Transformers 4.38.0.dev0
  • Pytorch 2.2.0+cu121
  • Datasets 2.16.1
  • Tokenizers 0.15.0
Downloads last month
2
Safetensors
Model size
774M params
Tensor type
F32
·
Inference API
This model can be loaded on Inference API (serverless).

Finetuned from