Llama 2 7B Chat : LoRA Fine-Tuning
A learning project where I fine-tuned Meta's Llama 2 7B Chat model using the LoRA (Low-Rank Adaptation) technique.
About
The goal of this project was to gain hands-on experience with fine-tuning a Large Language Model.
- Base model: Meta Llama 2 7B Chat
- Fine-tuning method: LoRA
- Dataset: "mlabonne/guanaco-llama2-1k"
- Dataset size: 1,000 samples
- Platform: Hugging Face
What I Learned
Through this project, I gained practical experience with:
- Working with pretrained LLMs
- Tokenization and dataset preparation
- Parameter-efficient fine-tuning using LoRA
- Saving and loading fine-tuned models
- Uploading and sharing models on Hugging Face
Project Note
This is a learning and experimental project created to understand the LLM fine-tuning workflow. The model was fine-tuned on the Guanaco 1K dataset using LoRA.
Base Model & License
This model is a LoRA fine-tuned derivative of Meta's Llama 2 7B Chat model.
The base model is subject to the Llama 2 Community License and Meta's Acceptable Use Policy.
Dataset
The training dataset used is "mlabonne/guanaco-llama2-1k" from the Hugging Face Hub.
Model tree for rc11tech/llm-finetuning
Base model
meta-llama/Llama-2-7b-chat-hf