Model Card for Model ID
SOLAR 10.7B model fine-tuned for 1 epoch on Dataricks instruction tuning dataset.
Model Details
Model Description
- Developed by: Andrew Chahnwoo Park
- Model type: SOLAR
- Language(s) (NLP): English
- License: apache-2.0
- Finetuned from model: upstage/SOLAR-10.7B-v1.0
Mistral Repository
Training Details
Training Data
Training Procedure
- Quantized Low-Rank Adaptation (QLoRA)
- Transformers Trainer
- DataCollatorForSeq2Seq
- Distributed Data Parallel (DDP) across two GPUs
Preprocessing
Manually created tokenized 'labels' for the dataset.
Prompt template utilized basic template for instruction-tuning
Hardware
Performed fine-tuning with 2 * A100 GPUs
- Provided by Gnewsoft during work period
Model and dataset are too large for free run sessions on Google Colab
