Edit model card

TindeBielik_finetunes

This model is a fine-tuned version of speakleash/Bielik-7B-Instruct-v0.1 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.5315

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 8e-05
  • train_batch_size: 4
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: constant
  • lr_scheduler_warmup_steps: 0.03
  • training_steps: 75

Training results

Training Loss Epoch Step Validation Loss
No log 0.2632 5 2.2403
2.2352 0.5263 10 1.9857
2.2352 0.7895 15 1.8578
1.8626 1.0526 20 1.7444
1.8626 1.3158 25 1.6693
1.5773 1.5789 30 1.6168
1.5773 1.8421 35 1.5745
1.4663 2.1053 40 1.5478
1.4663 2.3684 45 1.5356
1.1649 2.6316 50 1.5239
1.1649 2.8947 55 1.5185
0.9855 3.1579 60 1.5044
0.9855 3.4211 65 1.5510
0.6071 3.6842 70 1.5145
0.6071 3.9474 75 1.5315

Framework versions

  • PEFT 0.10.0
  • Transformers 4.40.1
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.0
  • Tokenizers 0.19.1
Downloads last month
3
Unable to determine this model’s pipeline type. Check the docs .

Adapter for