Alpaca-2
Collection
LLaMa 2 instruction tuned on cleaned Alpaca dataset using QLoRA. For languages other than English, Alpaca data was first translated using NLLB-1.3B
•
32 items
•
Updated
This represents the PEFT weights only. The base model is LLaMA 2. Instruction finetuning was done using 4 bit QLoRA on a single A100 GPU with the PEFT config as given below. The dataset used for this instruction finetuning process is a translated version of the cleaned alpaca dataset (translated using NLLB-1.3B).
Do note that this model might have inferior performance on some language specific tasks compared to full finetuning or a different base model trained with more language specific data.
The following bitsandbytes
quantization config was used during training:
Base model
meta-llama/Llama-2-7b-hf