Pokzy/flickr8k-finetuned

🖼️ BLIP Fine-Tuned on Flickr8k (Image Captioning)

This repository contains a Fine-Tuned BLIP (Bootstrapping Language-Image Pre-training) model for Image Captioning, trained on the Flickr8k dataset. - Base Model: Salesforce/blip-image-captioning-base - Developer: Pokzy - Interactive Web Demo: Hugging Face Space Demo

📌 Model Description

This model takes an image as input and generates descriptive English captions. It has been fine-tuned on the 8,000 daily images and human-annotated captions of the Flickr8k dataset to improve caption accuracy for everyday outdoor scenes, people, animals, and sports activities. In the associated web application, generated captions are also translated into Thai (ภาษาไทย) using the NLLB-200 translation model (facebook/nllb-200-distilled-600M).

🚀 How to Use

You can easily load and use this fine-tuned model using Hugging Face's transformers library:

import torch
from PIL import Image
from transformers import BlipProcessor, BlipForConditionalGeneration
# 1. Load Preprocessor & Fine-Tuned Model
processor = BlipProcessor.from_pretrained("Salesforce/blip-image-captioning-base")
model = BlipForConditionalGeneration.from_pretrained("Pokzy/flickr8k-finetuned")
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)
model.eval()
# 2. Load & Preprocess Image
image_path = "example.jpg"
image = Image.open(image_path).convert("RGB")
inputs = processor(images=image, return_tensors="pt").to(device)
# 3. Generate Caption
with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        do_sample=True,
        temperature=0.7,
        top_p=0.9,
        max_length=50
    )
caption = processor.decode(output_ids[0], skip_special_tokens=True)
print("Generated Caption:", caption)