Training procedure

Framework versions

  • eval_loss = 0.193117
  • PEFT 0.4.0
  • learning_rate = 1e-5
  • lora_r = 16
  • lora_alpha = 64
  • lora_dropout = 0.05
  • gradient_accumulation_steps = 8
  • per_device_train_batch_size = 8
  • optim = "paged_adamw_8bit"
Downloads last month
5
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no pipeline_tag.