Prerequisites
The vitalik-7b.csv
file contains the QA set required for the fine-tuning. The convert.py
script converts the CSV file into QAs in the llama2 chat template.
python convert.py
It generates a vitalik-7b.txt
file, which can now be used in fine-tuning.
Fine-tuning steps
Clone this repo into the llama.cpp/models/
folder.
cd llama.cpp/models
git clone https://huggingface.co/gaianet/vitalik.eth-7b
Move the Llama2-7b-chat base model to the folder.
cd vitalik-7b
mv path/to/llama-2-7b-chat.Q5_K_M.gguf .
From the llama.cpp/models/vitalik-7b
folder run the following command.
../../build/bin/finetune --model-base llama-2-7b-chat.Q5_K_M.gguf --lora-out lora.bin --train-data vitalik-7b.txt --sample-start '<SFT>' --adam-iter 1024
Wait for several days until the above process finishes. You will have a lora.bin
file, which can generate the fine-tuned model.
../../build/bin/export-lora --model-base llama-2-7b-chat.Q5_K_M.gguf --lora lora.bin --model-out vitalik.eth-7b-q5_k_m.gguf
Learn more about Llama2 model fine tuning here.
Run with Gaianet
Prompt template:
prompt template: llama-2-chat
Context size:
chat_ctx_size: 4096
Run with GaiaNet:
Quick start: https://docs.gaianet.ai/node-guide/quick-start
Customize your node: https://docs.gaianet.ai/node-guide/customize
- Downloads last month
- 289