Model Card for Model ID
MistralAI 7B model fine-tuned for 1 epoch on Dataricks instruction tuning dataset.
Model Details
Model Description
- Developed by: Andrew Chahnwoo Park
- Model type: Mistral
- Language(s) (NLP): English
- License: apache-2.0
- Finetuned from model: mistralai/Mistral-7B-v0.1
Mistral Repository
- Repository: mistralai/Mistral-7B-v0.1
Training Details
Training Data
Training Procedure
- Quantized Low-Rank Adaptation (QLoRA)
- Transformers Trainer
- DataCollatorForSeq2Seq
- Distributed Data Parallel (DDP) across two GPUs
Preprocessing
Manually created tokenized 'labels' for the dataset. Prompt template utilized basic template for instruction-tuning
Hardware
Performed fine-tuning with 2 * A100 GPUs
- Provided by Gnewsoft during work period Model and dataset are too large for free run sessions on Google Colab
- Downloads last month
- 0