metadata
language:
- en
license: apache-2.0
tags:
- trl
- text-generation-inference
- unsloth
- mistral
- gguf
base_model: teknium/OpenHermes-2.5-Mistral-7B
datasets:
- sayhan/strix-philosophy-qa
library_name: transformers
# OpenHermes 2.5 Stix Philosophy Mistral 7B
- Finetuned by: sayhan
- License: apache-2.0
- Finetuned from model : teknium/OpenHermes-2.5-Mistral-7B
- Dataset: sayhan/strix-philosophy-qa
LoRA rank: 8
LoRA alpha: 16
LoRA dropout: 0
Rank-stabilized LoRA: Yes
Number of epochs: 3
Learning rate: 1e-5
Batch size: 2
Gradient accumulation steps: 4
Weight decay: 0.01
Target modules:
- Query projection (`q_proj`)
- Key projection (`k_proj`)
- Value projection (`v_proj`)
- Output projection (`o_proj`)
- Gate projection (`gate_proj`)
- Up projection (`up_proj`)
- Down projection (`down_proj`)