Visualize in Weights & Biases

paligemma_khatt_paragraph_mix_448_ep1_r128

This model is a fine-tuned version of google/paligemma-3b-mix-448 on the khatt-paragraphs dataset. It achieves the following results on the evaluation set:

  • Loss: 1.6759

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-06
  • train_batch_size: 1
  • eval_batch_size: 1
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: constant
  • lr_scheduler_warmup_steps: 2
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
2.5113 0.1014 100 2.4764
2.1232 0.2027 200 2.1948
1.8843 0.3041 300 2.0064
1.815 0.4055 400 1.8942
1.8574 0.5068 500 1.8223
1.6986 0.6082 600 1.7674
1.749 0.7096 700 1.7335
1.7122 0.8109 800 1.7025
1.61 0.9123 900 1.6759

Framework versions

  • PEFT 0.11.1
  • Transformers 4.42.3
  • Pytorch 2.1.2
  • Datasets 2.20.0
  • Tokenizers 0.19.1
Downloads last month
9
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for ahmedheakl/paligemma_khatt_paragraph_mix_448_ep1_r128

Adapter
(23)
this model