TGIF-LLaVA-v1.5-7B

Released checkpoint for TGIF: Text-Guided Layer Fusion Mitigates Hallucination in Multimodal LLMs.

This model is a LLaVA-v1.5 7B checkpoint with TGIF text-guided layer fusion enabled through:

{
  "mm_projector_type": "layer_selector",
  "mm_vision_tower": "openai/clip-vit-large-patch14-336"
}

Usage

Use the official TGIF code:

git clone https://github.com/Linchenchen/TGIF.git
cd TGIF
pip install -e .

Then load the checkpoint with the standard LLaVA loader:

from llava.model.builder import load_pretrained_model
from llava.mm_utils import get_model_name_from_path

model_path = "cclinn/TGIF-LLaVA-v1.5-7B"
model_name = get_model_name_from_path(model_path)

tokenizer, model, image_processor, context_len = load_pretrained_model(
    model_path=model_path,
    model_base=None,
    model_name=model_name,
)

Notes

• This repository contains model weights only. Training and evaluation code is released at https://github.com/Linchenchen/TGIF.
• The model builds on LLaVA and Vicuna/LLaMA-family components; users must comply with the licenses and terms of all base models and datasets.
• Evaluation scripts for POPE, HallusionBench, OCRBench, TextVQA, MMRel, MMBench, ScienceQA, and GQA are documented in the TGIF repository.

Citation

Please cite the TGIF paper if you use this checkpoint. BibTeX metadata will be added after the paper record is finalized.

@article{tgif2026,
  title={TGIF: Text-Guided Layer Fusion Mitigates Hallucination in Multimodal LLMs},
  year={2026}
}