mmaaz60's picture
Update README.md
b60c625 verified
metadata
{}

CODE

LLaMA-3-V: Extending the Visual Capabilities of LLaVA with Meta-Llama-3-8B-Instruct

Repository Overview

This repository features LLaVA v1.5 trained with the Meta-Llama-3-8B-Instruct LLM. This integration aims to leverage the strengths of both models to offer advanced vision-language understanding.

Training Strategy

  • Pretraining: Only Vision-to-Language projector is trained. The rest of the model is frozen.
  • Fine-tuning: LLM is LoRA fine-tuned. Only the vision-backbone (CLIP) is kept frozen.
  • Note: The repository contains projector and LoRA weights.

Key Components

Training Data

Download It As

git lfs install
git clone https://huggingface.co/MBZUAI/LLaVA-Meta-Llama-3-8B-Instruct-lora

Contributions

Contributions are welcome! Please 🌟 our repository LLaVA++ if you find this model useful.