---
license: other
library_name: peft
tags:
- generated_from_trainer
base_model: google/gemma-7b
metrics:
- accuracy
model-index:
- name: amazon_attrprompt
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# amazon_attrprompt

This model is a fine-tuned version of [google/gemma-7b](https://huggingface.co/google/gemma-7b) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3678
- Accuracy: 0.8999
- F1 Macro: 0.8836
- F1 Micro: 0.8999

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- distributed_type: multi-GPU
- num_devices: 2
- total_train_batch_size: 32
- total_eval_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 Macro | F1 Micro |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:--------:|:--------:|
| 1.5514        | 0.13  | 50   | 1.1177          | 0.6825   | 0.5826   | 0.6825   |
| 0.5947        | 0.26  | 100  | 0.6187          | 0.8314   | 0.7860   | 0.8314   |
| 0.6672        | 0.39  | 150  | 0.5728          | 0.8445   | 0.7929   | 0.8445   |
| 0.5886        | 0.53  | 200  | 0.5686          | 0.8353   | 0.8093   | 0.8353   |
| 0.6311        | 0.66  | 250  | 0.5128          | 0.8511   | 0.8339   | 0.8511   |
| 0.4423        | 0.79  | 300  | 0.4992          | 0.8458   | 0.8258   | 0.8458   |
| 0.516         | 0.92  | 350  | 0.4209          | 0.8768   | 0.8642   | 0.8768   |
| 0.2427        | 1.05  | 400  | 0.4809          | 0.8748   | 0.8606   | 0.8748   |
| 0.2254        | 1.18  | 450  | 0.4480          | 0.8748   | 0.8608   | 0.8748   |
| 0.1819        | 1.32  | 500  | 0.4528          | 0.8742   | 0.8545   | 0.8742   |
| 0.2049        | 1.45  | 550  | 0.4906          | 0.8735   | 0.8589   | 0.8735   |
| 0.2247        | 1.58  | 600  | 0.4423          | 0.8841   | 0.8709   | 0.8841   |
| 0.2982        | 1.71  | 650  | 0.4455          | 0.8696   | 0.8603   | 0.8696   |
| 0.2299        | 1.84  | 700  | 0.3809          | 0.8933   | 0.8863   | 0.8933   |
| 0.1919        | 1.97  | 750  | 0.3678          | 0.8999   | 0.8836   | 0.8999   |
| 0.0728        | 2.11  | 800  | 0.3973          | 0.9005   | 0.8857   | 0.9005   |
| 0.0468        | 2.24  | 850  | 0.4278          | 0.8979   | 0.8824   | 0.8979   |
| 0.0576        | 2.37  | 900  | 0.4295          | 0.8972   | 0.8813   | 0.8972   |
| 0.0313        | 2.5   | 950  | 0.4330          | 0.8979   | 0.8829   | 0.8979   |
| 0.0509        | 2.63  | 1000 | 0.4032          | 0.9032   | 0.8947   | 0.9032   |
| 0.0235        | 2.76  | 1050 | 0.4134          | 0.9012   | 0.8919   | 0.9012   |
| 0.0335        | 2.89  | 1100 | 0.4066          | 0.9025   | 0.8929   | 0.9025   |


### Framework versions

- PEFT 0.9.0
- Transformers 4.39.0.dev0
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2