Hugging Face's logo

krishnapal2308
/
mistral-instruct-7b-finetuned-sroie

Text Generation
PEFT
TensorBoard
Safetensors
trl
sft
conversational
Model card Files Metrics Community
mistral-instruct-7b-finetuned-sroie / README.md
krishnapal2308's picture
krishnapal2308
krishnapal2308/mistral-instruct-7b-finetuned-sroie
c1ccae4 verified
preview code
|
raw
history blame
2.06 kB
metadata
base_model: mistralai/Mistral-7B-Instruct-v0.1
datasets:
  - generator
library_name: peft
license: apache-2.0
tags:
  - trl
  - sft
  - generated_from_trainer
model-index:
  - name: mistral-instruct-7b-finetuned-sroie
    results: []

Visualize in Weights & Biases

mistral-instruct-7b-finetuned-sroie

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.1 on the generator dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7668

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0002
  • train_batch_size: 1
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: constant
  • lr_scheduler_warmup_steps: 50
  • training_steps: 120
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
1.4458 0.4848 20 1.3313
1.1796 0.9697 40 1.1143
1.0555 1.4545 60 0.9733
0.9514 1.9394 80 0.8829
0.8599 2.4242 100 0.8150
0.857 2.9091 120 0.7668

Framework versions

  • PEFT 0.12.0
  • Transformers 4.42.4
  • Pytorch 2.3.1+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1