Edit model card

1e05_output_dir_clean_df_10-100_noX_100_50_epoch_cluster

This model is a fine-tuned version of nferruz/ProtGPT2 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 3.9890
  • Accuracy: 0.4213

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 50.0

Training results

Training Loss Epoch Step Validation Loss Accuracy
No log 1.0 148 5.8378 0.2287
No log 2.0 296 5.5732 0.2491
No log 3.0 444 5.3800 0.2680
5.761 4.0 592 5.2259 0.2842
5.761 5.0 740 5.1006 0.2963
5.761 6.0 888 4.9997 0.3067
5.1725 7.0 1036 4.9103 0.3153
5.1725 8.0 1184 4.8287 0.3230
5.1725 9.0 1332 4.7578 0.3301
5.1725 10.0 1480 4.6942 0.3376
4.8482 11.0 1628 4.6364 0.3439
4.8482 12.0 1776 4.5813 0.3497
4.8482 13.0 1924 4.5328 0.3554
4.609 14.0 2072 4.4897 0.3610
4.609 15.0 2220 4.4454 0.3657
4.609 16.0 2368 4.4132 0.3702
4.4241 17.0 2516 4.3742 0.3738
4.4241 18.0 2664 4.3438 0.3782
4.4241 19.0 2812 4.3164 0.3817
4.4241 20.0 2960 4.2879 0.3848
4.283 21.0 3108 4.2602 0.3878
4.283 22.0 3256 4.2373 0.3902
4.283 23.0 3404 4.2160 0.3932
4.1606 24.0 3552 4.1964 0.3954
4.1606 25.0 3700 4.1816 0.3976
4.1606 26.0 3848 4.1605 0.3995
4.1606 27.0 3996 4.1443 0.4016
4.0685 28.0 4144 4.1290 0.4034
4.0685 29.0 4292 4.1146 0.4052
4.0685 30.0 4440 4.1037 0.4067
3.9859 31.0 4588 4.0874 0.4082
3.9859 32.0 4736 4.0784 0.4098
3.9859 33.0 4884 4.0669 0.4115
3.9275 34.0 5032 4.0581 0.4125
3.9275 35.0 5180 4.0479 0.4136
3.9275 36.0 5328 4.0384 0.4148
3.9275 37.0 5476 4.0330 0.4159
3.8799 38.0 5624 4.0262 0.4166
3.8799 39.0 5772 4.0212 0.4174
3.8799 40.0 5920 4.0136 0.4180
3.8348 41.0 6068 4.0111 0.4186
3.8348 42.0 6216 4.0048 0.4193
3.8348 43.0 6364 4.0004 0.4195
3.8151 44.0 6512 3.9978 0.4199
3.8151 45.0 6660 3.9951 0.4203
3.8151 46.0 6808 3.9934 0.4207
3.8151 47.0 6956 3.9904 0.4209
3.7948 48.0 7104 3.9911 0.4211
3.7948 49.0 7252 3.9890 0.4213
3.7948 50.0 7400 3.9890 0.4213

Framework versions

  • Transformers 4.38.0.dev0
  • Pytorch 2.2.0+cu121
  • Datasets 2.16.1
  • Tokenizers 0.15.0
Downloads last month
0
Safetensors
Model size
774M params
Tensor type
F32
·

Finetuned from