Gemma 3 Lithuanian Weather Caption LoRA — Run D
This is the fourth LoRA adapter fine-tuned for Lithuanian weather-focused image captioning.
This version uses a stronger LoRA configuration with a higher rank and alpha value.
The goal of Run D is to test whether more adapter capacity improves learning and produces better Lithuanian weather captions.
Because the dataset is small, this run also has a higher risk of overfitting compared with the lighter LoRA runs.
The model was trained to describe the weather in an image using short, simple Lithuanian sentences.
Task
Given an image, the model generates a short Lithuanian caption focused on:
- cloudiness
- sunlight
- precipitation
- visibility
- time of day
- general weather conditions
Base model
unsloth/gemma-3-4b-it
Dataset
Dataset: Matas5/GMM_team_task
Training examples used: 103
The dataset contains images with Lithuanian weather captions. Captions were standardized to focus mainly on weather conditions rather than unrelated objects.
Training setup
Training method: LoRA fine-tuning with Unsloth
Model loading: 4-bit quantized base model
Fine-tuning type: PEFT / LoRA adapter
Run version: D
Purpose of this run: stronger LoRA adapter capacity
Number of epochs: 3
Total training steps: approximately 78
Per-device batch size: 1
Gradient accumulation steps: 2
Number of GPUs: 2 Tesla T4 GPUs
Effective total batch size: 4
Learning rate: 2e-4
Optimizer: adamw_8bit
Gradient checkpointing: enabled
Save strategy: save every epoch
LoRA rank: 8
LoRA alpha: 16
Target modules: q_proj, v_proj
Vision layers fine-tuned: yes
Prompt used during training
Trumpai apibūdink orą šioje nuotraukoje lietuviškai. Atsakyk vienu paprastu sakiniu.
Example expected output style
Dangus giedras ir ryškiai mėlynas, debesų beveik nėra. Oras saulėtas, sausas, matomumas labai geras.
Training result summary
Run D was trained with a stronger LoRA setup than Runs A, B, and C.
Run A used LoRA rank 4 and alpha 8 with 3 epochs and learning rate 2e-4.
Run B used LoRA rank 4 and alpha 8 with 2 epochs and learning rate 2e-4.
Run C used LoRA rank 4 and alpha 8 with 3 epochs and learning rate 1e-4.
Run D uses LoRA rank 8 and alpha 16 with 3 epochs and learning rate 2e-4.
The purpose of this run is to test whether a larger LoRA adapter can learn the Lithuanian weather-captioning task better.
However, because the dataset contains only 103 training examples, the stronger adapter may also increase overfitting risk.
The result should be evaluated on unseen test images and compared with Runs A, B, and C.
Intended use
This adapter is intended for a university project demonstrating fine-tuning of a vision-language model for Lithuanian weather captioning.
It is not intended for professional meteorological forecasting.
- Downloads last month
- 22