Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
yanghaojin 
posted an update May 2, 2024
Post
2258
Full parameter fine-tuning of the LLaMA-3 8B model using a single GTX 3090 GPU with 24GB of graphics memory?

Please check out our tool for fine-tuning, inferencing, and evaluating GreenBitAI's low-bit LLMs:
https://github.com/GreenBitAI/green-bit-llm
Model Zoo:
GreenBitAI
yanghaojin

Command for reproducing this run 😉 :
CUDA_VISIBLE_DEVICES=0 WANDB_DISABLED=true python -m sft.finetune --model GreenBitAI/Llama-3-8B-layer-mix-bpw-2.2 --tune-qweight-only --galore --galore-rank 64 --optimizer adamw8bit --batch-size 1 --seqlen 96

jqodiriy

How you prepare dataset for finetuning llama3?
Could you show the structure of your dataset and how you fine-tune using that dataset?

jqodiriy

How you prepare dataset for finetuning llama3?
Could you show the structure of your dataset and how you fine-tune using that dataset?

In this post