---
license: apache-2.0
base_model: google/flan-t5-small
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: flan_t5_small_ledgar
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# flan_t5_small_ledgar

This model is a fine-tuned version of [google/flan-t5-small](https://huggingface.co/google/flan-t5-small) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5534
- Accuracy: 0.8525
- F1 Macro: 0.7680
- F1 Micro: 0.8525

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- distributed_type: multi-GPU
- num_devices: 2
- total_train_batch_size: 64
- total_eval_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 Macro | F1 Micro |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:--------:|:--------:|
| 2.1741        | 0.11  | 100  | 1.6958          | 0.6204   | 0.3397   | 0.6204   |
| 1.2325        | 0.21  | 200  | 1.0701          | 0.7302   | 0.5340   | 0.7302   |
| 0.9558        | 0.32  | 300  | 0.8877          | 0.7713   | 0.6186   | 0.7713   |
| 0.8635        | 0.43  | 400  | 0.8029          | 0.788    | 0.6469   | 0.788    |
| 0.8667        | 0.53  | 500  | 0.7517          | 0.8035   | 0.6794   | 0.8035   |
| 0.7975        | 0.64  | 600  | 0.7280          | 0.8031   | 0.6852   | 0.8031   |
| 0.7511        | 0.75  | 700  | 0.7209          | 0.8124   | 0.6907   | 0.8124   |
| 0.7683        | 0.85  | 800  | 0.6883          | 0.811    | 0.6968   | 0.811    |
| 0.6874        | 0.96  | 900  | 0.6764          | 0.8137   | 0.7147   | 0.8137   |
| 0.6042        | 1.07  | 1000 | 0.6628          | 0.8236   | 0.7097   | 0.8236   |
| 0.6397        | 1.17  | 1100 | 0.6546          | 0.8233   | 0.7171   | 0.8233   |
| 0.6584        | 1.28  | 1200 | 0.6371          | 0.831    | 0.7400   | 0.831    |
| 0.5718        | 1.39  | 1300 | 0.6346          | 0.8295   | 0.7350   | 0.8295   |
| 0.5012        | 1.49  | 1400 | 0.6176          | 0.8343   | 0.7446   | 0.8343   |
| 0.5843        | 1.6   | 1500 | 0.6214          | 0.8331   | 0.7376   | 0.8331   |
| 0.6021        | 1.71  | 1600 | 0.6024          | 0.8395   | 0.7455   | 0.8395   |
| 0.5538        | 1.81  | 1700 | 0.5964          | 0.843    | 0.7516   | 0.843    |
| 0.5391        | 1.92  | 1800 | 0.5835          | 0.8431   | 0.7590   | 0.8431   |
| 0.4632        | 2.03  | 1900 | 0.5845          | 0.842    | 0.7432   | 0.842    |
| 0.4581        | 2.13  | 2000 | 0.5832          | 0.8451   | 0.7575   | 0.8451   |
| 0.4806        | 2.24  | 2100 | 0.5749          | 0.8444   | 0.7639   | 0.8444   |
| 0.4438        | 2.35  | 2200 | 0.5704          | 0.85     | 0.7642   | 0.85     |
| 0.4379        | 2.45  | 2300 | 0.5667          | 0.8486   | 0.7598   | 0.8486   |
| 0.4342        | 2.56  | 2400 | 0.5614          | 0.8503   | 0.7642   | 0.8503   |
| 0.4197        | 2.67  | 2500 | 0.5605          | 0.8527   | 0.7684   | 0.8527   |
| 0.4417        | 2.77  | 2600 | 0.5568          | 0.8505   | 0.7652   | 0.8505   |
| 0.4401        | 2.88  | 2700 | 0.5542          | 0.8529   | 0.7685   | 0.8529   |
| 0.4666        | 2.99  | 2800 | 0.5534          | 0.8525   | 0.7680   | 0.8525   |


### Framework versions

- Transformers 4.39.0.dev0
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2