Hugging Face's logo

yunaseo
/
emotions_google_gemma

PEFT
Safetensors
gemma
Generated from Trainer
Model card Files Community
emotions_google_gemma / README.md
yunaseo's picture
yunaseo
yunaseo/google_gemma_lora_emotion_detection
c12296c verified
preview code
|
raw
history blame
3.53 kB
metadata
license: gemma
library_name: peft
tags:
  - generated_from_trainer
base_model: google/gemma-1.1-2b-it
metrics:
  - accuracy
model-index:
  - name: emotions_google_gemma
    results: []

emotions_google_gemma

This model is a fine-tuned version of google/gemma-1.1-2b-it on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4792
  • F1 Micro: 0.6970
  • F1 Macro: 0.6089
  • Accuracy: 0.2104

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 5

Training results

Training Loss Epoch Step Validation Loss F1 Micro F1 Macro Accuracy
0.7081 0.2067 20 0.6048 0.6244 0.5113 0.1528
0.5228 0.4134 40 0.5096 0.6713 0.5815 0.1883
0.5048 0.6202 60 0.4928 0.7002 0.5865 0.2155
0.5129 0.8269 80 0.4792 0.6970 0.6089 0.2104
0.4842 1.0336 100 0.4801 0.6972 0.6023 0.2369
0.3372 1.2403 120 0.5545 0.6687 0.5877 0.1761
0.3302 1.4470 140 0.5374 0.6895 0.6020 0.2019
0.3342 1.6537 160 0.5330 0.6860 0.5993 0.2117
0.3392 1.8605 180 0.5190 0.6894 0.5913 0.2006
0.2844 2.0672 200 0.5853 0.6891 0.5819 0.2369
0.1458 2.2739 220 0.7038 0.6743 0.5749 0.2097
0.1508 2.4806 240 0.6808 0.6802 0.5834 0.1994
0.1481 2.6873 260 0.7026 0.6773 0.5721 0.2
0.1378 2.8941 280 0.7336 0.6790 0.5768 0.2162
0.0961 3.1008 300 0.8397 0.6709 0.5465 0.2272
0.0552 3.3075 320 0.8260 0.6743 0.5654 0.2168
0.0509 3.5142 340 0.8692 0.6777 0.5666 0.2233
0.0489 3.7209 360 0.8505 0.6874 0.5722 0.2388
0.0526 3.9276 380 0.8269 0.6842 0.5778 0.2233
0.0278 4.1344 400 0.9280 0.6813 0.5557 0.2414
0.0187 4.3411 420 0.9390 0.6829 0.5588 0.2382
0.0169 4.5478 440 0.9510 0.6834 0.5612 0.2485
0.0158 4.7545 460 0.9325 0.6819 0.5612 0.2427
0.0161 4.9612 480 0.9311 0.6822 0.5634 0.2440

Framework versions

  • PEFT 0.10.0
  • Transformers 4.40.2
  • Pytorch 2.2.1+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1