shafin's picture
update model card README.md
99a69c7
metadata
license: mit
tags:
  - generated_from_trainer
model-index:
  - name: gbert-large-finetuned-cust18
    results: []

gbert-large-finetuned-cust18

This model is a fine-tuned version of deepset/gbert-large on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.1232

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
0.604 1.0 391 0.3560
0.3497 2.0 782 0.2838
0.2812 3.0 1173 0.2484
0.2452 4.0 1564 0.2232
0.2253 5.0 1955 0.2240
0.2202 6.0 2346 0.1993
0.1922 7.0 2737 0.1747
0.182 8.0 3128 0.1631
0.1609 9.0 3519 0.1555
0.1553 10.0 3910 0.1434
0.147 11.0 4301 0.1399
0.144 12.0 4692 0.1340
0.1307 13.0 5083 0.1319
0.128 14.0 5474 0.1490
0.1304 15.0 5865 0.1338
0.1165 16.0 6256 0.1233
0.1456 17.0 6647 0.1673
0.1419 18.0 7038 0.1591
0.1447 19.0 7429 0.1360
0.1317 20.0 7820 0.1232

Framework versions

  • Transformers 4.28.1
  • Pytorch 2.0.0+cu118
  • Datasets 2.11.0
  • Tokenizers 0.13.3