Edit model card

llama3-2M-MedEV

This model is a fine-tuned version of unsloth/llama-3-8b-Instruct-bnb-4bit on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.4249
  • Bleu: 47.7973

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0002
  • train_batch_size: 32
  • eval_batch_size: 16
  • seed: 3407
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 128
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 5
  • num_epochs: 3

Training results

Training Loss Epoch Step Validation Loss
1.534 0.1200 320 1.4902
1.3171 0.2399 640 1.4705
1.29 0.3599 960 1.4644
1.2699 0.4798 1280 1.4287
1.2567 0.5998 1600 1.4576
1.2448 0.7197 1920 1.4196
1.2353 0.8397 2240 1.4249
1.2274 0.9596 2560 1.4172
1.1635 1.0796 2880 1.4180
1.1337 1.1995 3200 1.4219
1.1346 1.3195 3520 1.3954
1.131 1.4394 3840 1.3714
1.1325 1.5594 4160 1.3923
1.1269 1.6793 4480 1.4118
1.1221 1.7993 4800 1.4251
1.1226 1.9192 5120 1.3970
1.0898 2.0392 5440 1.4198
1.0372 2.1591 5760 1.4310
1.0325 2.2791 6080 1.4209
1.0334 2.3990 6400 1.4205
1.0328 2.5190 6720 1.4306
1.0303 2.6389 7040 1.4222
1.0283 2.7589 7360 1.4266
1.0273 2.8788 7680 1.4251
1.0295 2.9988 8000 1.4249

Framework versions

  • PEFT 0.10.0
  • Transformers 4.40.2
  • Pytorch 2.3.0
  • Datasets 2.19.1
  • Tokenizers 0.19.1
Downloads last month
2
Unable to determine this model’s pipeline type. Check the docs .

Adapter for