Hugging Face's logo

imdatta0
/
lora_final

PEFT
Safetensors
unsloth
Generated from Trainer
Model card Files Community
lora_final / Mistral-7B-v0.3_pct_reverse_r16 /README.md
imdatta0's picture
imdatta0
End of training
8b1b653 verified
preview code
|
raw
history blame contribute delete
3.92 kB
metadata
base_model: unsloth/mistral-7b-v0.3
library_name: peft
license: apache-2.0
tags:
  - unsloth
  - generated_from_trainer
model-index:
  - name: Mistral-7B-v0.3_pct_reverse_r16
    results: []

Mistral-7B-v0.3_pct_reverse_r16

This model is a fine-tuned version of unsloth/mistral-7b-v0.3 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 2.0162

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • gradient_accumulation_steps: 32
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.02
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
1.9648 0.0206 8 2.0392
2.0599 0.0413 16 2.0531
2.1274 0.0619 24 2.0571
2.0718 0.0825 32 2.0473
2.0646 0.1032 40 2.0420
2.0883 0.1238 48 2.0460
2.0611 0.1445 56 2.0497
2.0841 0.1651 64 2.0536
2.0695 0.1857 72 2.0688
2.0696 0.2064 80 2.0792
2.1315 0.2270 88 2.0900
2.1466 0.2476 96 2.0827
2.1575 0.2683 104 2.0826
2.0925 0.2889 112 2.0864
2.1647 0.3096 120 2.0815
2.1018 0.3302 128 2.0882
2.1062 0.3508 136 2.0904
2.1596 0.3715 144 2.0847
2.1473 0.3921 152 2.0933
2.1388 0.4127 160 2.0888
2.093 0.4334 168 2.0887
2.1704 0.4540 176 2.0933
2.0697 0.4746 184 2.0779
2.1725 0.4953 192 2.0714
2.1339 0.5159 200 2.0695
2.106 0.5366 208 2.0640
2.0857 0.5572 216 2.0792
2.0751 0.5778 224 2.0658
2.0987 0.5985 232 2.0659
2.0817 0.6191 240 2.0628
2.1341 0.6397 248 2.0564
2.0567 0.6604 256 2.0517
2.1246 0.6810 264 2.0457
2.0623 0.7017 272 2.0423
2.1106 0.7223 280 2.0369
2.1094 0.7429 288 2.0375
2.0678 0.7636 296 2.0330
2.0521 0.7842 304 2.0326
2.0594 0.8048 312 2.0241
2.051 0.8255 320 2.0208
2.0392 0.8461 328 2.0201
2.0143 0.8667 336 2.0207
2.0678 0.8874 344 2.0222
2.0473 0.9080 352 2.0187
2.0324 0.9287 360 2.0165
2.0404 0.9493 368 2.0160
2.0426 0.9699 376 2.0163
2.0635 0.9906 384 2.0162

Framework versions

  • PEFT 0.12.0
  • Transformers 4.44.0
  • Pytorch 2.4.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1