---
license: other
base_model: HuggingFaceM4/idefics-9b
tags:
- generated_from_trainer
datasets:
- diffusiondb
model-index:
- name: idefics-9b-ft-describe-diffusion-bf16
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# idefics-9b-ft-describe-diffusion-bf16

This model is a fine-tuned version of [HuggingFaceM4/idefics-9b](https://huggingface.co/HuggingFaceM4/idefics-9b) on the diffusiondb dataset.
It achieves the following results on the evaluation set:
- Loss: 1.4081

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 32
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.03
- num_epochs: 2

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 2.0874        | 0.07  | 50   | 2.1257          |
| 2.0532        | 0.14  | 100  | 1.9973          |
| 1.9417        | 0.21  | 150  | 1.9246          |
| 1.8358        | 0.28  | 200  | 1.8735          |
| 1.8499        | 0.36  | 250  | 1.8305          |
| 1.7695        | 0.43  | 300  | 1.7770          |
| 1.7505        | 0.5   | 350  | 1.7454          |
| 1.713         | 0.57  | 400  | 1.7115          |
| 1.7352        | 0.64  | 450  | 1.6791          |
| 1.6689        | 0.71  | 500  | 1.6526          |
| 1.6183        | 0.78  | 550  | 1.6257          |
| 1.6118        | 0.85  | 600  | 1.6001          |
| 1.6095        | 0.92  | 650  | 1.5800          |
| 1.5598        | 1.0   | 700  | 1.5598          |
| 1.4785        | 1.07  | 750  | 1.5403          |
| 1.4999        | 1.14  | 800  | 1.5219          |
| 1.4589        | 1.21  | 850  | 1.5063          |
| 1.4559        | 1.28  | 900  | 1.4942          |
| 1.4332        | 1.35  | 950  | 1.4792          |
| 1.4859        | 1.42  | 1000 | 1.4658          |
| 1.3888        | 1.49  | 1050 | 1.4537          |
| 1.4032        | 1.56  | 1100 | 1.4445          |
| 1.3702        | 1.64  | 1150 | 1.4352          |
| 1.3625        | 1.71  | 1200 | 1.4276          |
| 1.4067        | 1.78  | 1250 | 1.4199          |
| 1.3829        | 1.85  | 1300 | 1.4149          |
| 1.4251        | 1.92  | 1350 | 1.4103          |
| 1.3619        | 1.99  | 1400 | 1.4081          |


### Framework versions

- Transformers 4.32.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.4
- Tokenizers 0.13.3