Model Card for Model ID
SOLAR 10.7B model fine-tuned for 1 epoch on Dataricks instruction tuning dataset.
Model Details
Model Description
- Developed by: Andrew Chahnwoo Park
- Model type: SOLAR
- Language(s) (NLP): English
- License: apache-2.0
- Finetuned from model: upstage/SOLAR-10.7B-v1.0
Mistral Repository
- Repository: upstage/SOLAR-10.7B-v1.0
Training Details
Training Data
Training Procedure
- Quantized Low-Rank Adaptation (QLoRA)
- Transformers Trainer
- DataCollatorForSeq2Seq
- Distributed Data Parallel (DDP) across two GPUs
Preprocessing
Manually created tokenized 'labels' for the dataset. Prompt template utilized basic template for instruction-tuning
Hardware
Performed fine-tuning with 2 * A100 GPUs
- Provided by Gnewsoft during work period Model and dataset are too large for free run sessions on Google Colab
- Downloads last month
- 2
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.