---
base_model: unsloth/mistral-7b-v0.3
library_name: peft
license: apache-2.0
tags:
- unsloth
- generated_from_trainer
model-index:
- name: Mistral-7B-v0.3_pct_reverse_r16
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Mistral-7B-v0.3_pct_reverse_r16

This model is a fine-tuned version of [unsloth/mistral-7b-v0.3](https://huggingface.co/unsloth/mistral-7b-v0.3) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.0162

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 32
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.02
- num_epochs: 1

### Training results

| Training Loss | Epoch  | Step | Validation Loss |
|:-------------:|:------:|:----:|:---------------:|
| 1.9648        | 0.0206 | 8    | 2.0392          |
| 2.0599        | 0.0413 | 16   | 2.0531          |
| 2.1274        | 0.0619 | 24   | 2.0571          |
| 2.0718        | 0.0825 | 32   | 2.0473          |
| 2.0646        | 0.1032 | 40   | 2.0420          |
| 2.0883        | 0.1238 | 48   | 2.0460          |
| 2.0611        | 0.1445 | 56   | 2.0497          |
| 2.0841        | 0.1651 | 64   | 2.0536          |
| 2.0695        | 0.1857 | 72   | 2.0688          |
| 2.0696        | 0.2064 | 80   | 2.0792          |
| 2.1315        | 0.2270 | 88   | 2.0900          |
| 2.1466        | 0.2476 | 96   | 2.0827          |
| 2.1575        | 0.2683 | 104  | 2.0826          |
| 2.0925        | 0.2889 | 112  | 2.0864          |
| 2.1647        | 0.3096 | 120  | 2.0815          |
| 2.1018        | 0.3302 | 128  | 2.0882          |
| 2.1062        | 0.3508 | 136  | 2.0904          |
| 2.1596        | 0.3715 | 144  | 2.0847          |
| 2.1473        | 0.3921 | 152  | 2.0933          |
| 2.1388        | 0.4127 | 160  | 2.0888          |
| 2.093         | 0.4334 | 168  | 2.0887          |
| 2.1704        | 0.4540 | 176  | 2.0933          |
| 2.0697        | 0.4746 | 184  | 2.0779          |
| 2.1725        | 0.4953 | 192  | 2.0714          |
| 2.1339        | 0.5159 | 200  | 2.0695          |
| 2.106         | 0.5366 | 208  | 2.0640          |
| 2.0857        | 0.5572 | 216  | 2.0792          |
| 2.0751        | 0.5778 | 224  | 2.0658          |
| 2.0987        | 0.5985 | 232  | 2.0659          |
| 2.0817        | 0.6191 | 240  | 2.0628          |
| 2.1341        | 0.6397 | 248  | 2.0564          |
| 2.0567        | 0.6604 | 256  | 2.0517          |
| 2.1246        | 0.6810 | 264  | 2.0457          |
| 2.0623        | 0.7017 | 272  | 2.0423          |
| 2.1106        | 0.7223 | 280  | 2.0369          |
| 2.1094        | 0.7429 | 288  | 2.0375          |
| 2.0678        | 0.7636 | 296  | 2.0330          |
| 2.0521        | 0.7842 | 304  | 2.0326          |
| 2.0594        | 0.8048 | 312  | 2.0241          |
| 2.051         | 0.8255 | 320  | 2.0208          |
| 2.0392        | 0.8461 | 328  | 2.0201          |
| 2.0143        | 0.8667 | 336  | 2.0207          |
| 2.0678        | 0.8874 | 344  | 2.0222          |
| 2.0473        | 0.9080 | 352  | 2.0187          |
| 2.0324        | 0.9287 | 360  | 2.0165          |
| 2.0404        | 0.9493 | 368  | 2.0160          |
| 2.0426        | 0.9699 | 376  | 2.0163          |
| 2.0635        | 0.9906 | 384  | 2.0162          |


### Framework versions

- PEFT 0.12.0
- Transformers 4.44.0
- Pytorch 2.4.0+cu121
- Datasets 2.20.0
- Tokenizers 0.19.1