Hugging Face's logo

dhanishetty
/
flan-t5-base_Adapters

PEFT
Safetensors
Generated from Trainer
Model card Files Community
flan-t5-base_Adapters / README.md
dhanishetty's picture
dhanishetty
End of training
5cca563 verified
preview code
|
raw
history blame
2.47 kB
metadata
base_model: google/flan-t5-base
library_name: peft
license: apache-2.0
tags:
  - generated_from_trainer
model-index:
  - name: phi-3-mini-LoRA
    results: []

Visualize in Weights & Biases

phi-3-mini-LoRA

This model is a fine-tuned version of google/flan-t5-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.0702

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.001
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: inverse_sqrt
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 3

Training results

Training Loss Epoch Step Validation Loss
1.6034 0.1688 50 1.3534
1.44 0.3376 100 1.2534
1.39 0.5063 150 1.2045
1.3075 0.6751 200 1.1710
1.2984 0.8439 250 1.1482
1.2933 1.0127 300 1.1500
1.2286 1.1814 350 1.1385
1.206 1.3502 400 1.1237
1.2097 1.5190 450 1.1112
1.1982 1.6878 500 1.1074
1.2451 1.8565 550 1.0894
1.1447 2.0253 600 1.1006
1.1324 2.1941 650 1.0787
1.137 2.3629 700 1.0798
1.14 2.5316 750 1.0739
1.1112 2.7004 800 1.0694
1.1436 2.8692 850 1.0702

Framework versions

  • PEFT 0.12.0
  • Transformers 4.43.2
  • Pytorch 2.1.0
  • Datasets 2.20.0
  • Tokenizers 0.19.1