Edit model card

medical_jargons_simplifier2

This model is a fine-tuned version of luqh/ClinicalT5-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4641

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 5
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss
10.6338 0.3378 50 5.9582
3.6156 0.6757 100 1.0741
1.3304 1.0135 150 0.8368
1.0096 1.3514 200 0.7519
0.933 1.6892 250 0.7019
0.8178 2.0270 300 0.6586
0.7714 2.3649 350 0.6188
0.7077 2.7027 400 0.5924
0.7406 3.0405 450 0.5673
0.6601 3.3784 500 0.5531
0.6637 3.7162 550 0.5388
0.6489 4.0541 600 0.5281
0.6369 4.3919 650 0.5187
0.5996 4.7297 700 0.5109
0.5816 5.0676 750 0.5028
0.5714 5.4054 800 0.4961
0.5826 5.7432 850 0.4910
0.5646 6.0811 900 0.4855
0.5379 6.4189 950 0.4827
0.5586 6.7568 1000 0.4785
0.5408 7.0946 1050 0.4751
0.5576 7.4324 1100 0.4727
0.5241 7.7703 1150 0.4710
0.5298 8.1081 1200 0.4695
0.5424 8.4459 1250 0.4677
0.5038 8.7838 1300 0.4665
0.5545 9.1216 1350 0.4653
0.523 9.4595 1400 0.4644
0.5029 9.7973 1450 0.4641

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.1.2
  • Datasets 2.19.2
  • Tokenizers 0.19.1
Downloads last month
5
Safetensors
Model size
223M params
Tensor type
F32
·
Inference API
This model can be loaded on Inference API (serverless).

Finetuned from