Hugging Face's logo

ynuwara
/
finetune_colpali_v1_2-olje_norsk-4bit

Transformers
Safetensors
ColPali
Generated from Trainer
Inference Endpoints
Model card Files Community
1
finetune_colpali_v1_2-olje_norsk-4bit / README.md
ynuwara's picture
ynuwara
Update README.md
027fced verified
preview code
|
raw
history blame
1.71 kB
metadata
library_name: transformers
license: gemma
base_model: vidore/colpaligemma-3b-pt-448-base
tags:
  - colpali
  - generated_from_trainer
model-index:
  - name: finetune_colpali_v1_2-olje_norsk-4bit
    results: []

finetune_colpali_v1_2-olje_norsk-4bit

This model is a fine-tuned version of vidore/colpaligemma-3b-pt-448-base on the ynuwara/norsk-olje-gass-QnA-ColPali dataset.

Model description

ColPaliGemma-3B model is fine-tuned on a specific question and answering dataset from a book about Oil and Gas in Norwegian language. Fine tuning is done on Colab Pro's NVIDIA A100 GPU.

Intended uses & limitations

This model is suited for RAG uses in oil and gas domain, specifically if the query is in Norwegian

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 16
  • optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 100
  • num_epochs: 1.5

Training results

Training Loss Epoch Step Validation Loss
No log 0.0833 1 0.0837

Framework versions

  • Transformers 4.46.3
  • Pytorch 2.5.1+cu121
  • Datasets 3.2.0
  • Tokenizers 0.20.3