WasteAssistant-BLIP-Qwen2-VL

WasteAssistant-BLIP-Qwen2-VL is a BLIP-based visual question-answering model fine-tuned for waste-related questions. In this training variant, Qwen2-VL-7B was used to generate candidate/reference answers for the training pipeline. Qwen2-VL-7B is the answer-generation model; the model served from this repository is the fine-tuned BLIP checkpoint.

• Base model: Salesforce/blip-vqa-base
• Dataset: KhushKataruka/WasteVQA
• Research paper: arXiv:2607.10610

Question types represented in the evaluation split

The supplied evaluation split contains 2,784 image-question-answer records, 2,070 unique image paths, 15 question templates, and 21 waste labels. The question coverage below is based on the actual questions in the provided test.json split.

1. Identify the waste and its materials

The model is trained/evaluated on questions such as:

• “What specific type of waste item do you see in this image?”
• “What materials is this waste object primarily made of?”
• “What visible characteristics like labels, shape, texture, color, and size help identify this item?”

2. Inspect visible contamination

• “Are there any visible signs of contamination such as food residue, liquids, stains, or dirt on this object?”

This asks about visible evidence in the image; it cannot establish hidden contamination or chemical safety.

3. Classify waste under the dataset's SWM framing

• “Which main SWM 2016 waste category does this item belong to - Dry Waste, Wet Waste, Sanitary Waste, or Domestic Hazardous Waste?”
• “How would you classify this item as biodegradable, non-biodegradable, or hazardous according to SWM Rules 2016?”
• “Does this item fall under any special sub-categories like E-waste, Composite Waste, or Inert Waste and why?”

4. Recommend disposal preparation and handling

• “What preprocessing steps like cleaning, drying, or wrapping are required before disposing of this item?”
• “Are there any special handling procedures needed for this item including separate collection methods?”

5. Discuss recycling and compostability

• “How recyclable is this item through standard municipal recycling systems in India?”
• “What is the compostability potential of this item at household level and what is your reasoning?”

6. Explain bin selection

• “What colored bin should this waste be disposed in according to SWM Rules and what is the reasoning?”

7. Discuss hazards and environmental consequences

• “On a scale of 1 to 10, what is the hazard level of this item and what hazard category does it belong to?”
• “What environmental impacts would occur if this item were disposed of improperly?”

8. Estimate decomposition time

• “How long would this item take to decompose in a typical Indian landfill and what factors affect this?”

The decomposition question has some missing-answer placeholders in the supplied split (8 records). Review these records before using them for evaluation or reporting metrics.

Waste labels in the supplied split

Cans, CardBoard, Chips Packets, CofeeCup, Construction scrap, Electrical Cables, Electronic chips, Electronics, Empty Medicine Packets, Glass, Gloves, Mask, Medicines, Metal, Paper, Plastic Bottles, Plastic Materials, Polythene, Syringes, WetWaste, Wire.

These labels describe the dataset's annotation categories. Their presence does not mean every label is equally represented or that the model performs equally well on each category.

Installation

pip install torch transformers pillow

Inference example

This example assumes that the repository contains a compatible BLIP checkpoint and the required processor files.

import torch
from PIL import Image
from transformers import BlipProcessor, BlipForQuestionAnswering

model_id = "KhushKataruka/WasteAssistant-BLIP-Qwen2-VL"

processor = BlipProcessor.from_pretrained(model_id)
model = BlipForQuestionAnswering.from_pretrained(model_id)

device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)
model.eval()

image = Image.open("waste_image.jpg").convert("RGB")
question = "What specific type of waste item do you see in this image?"

inputs = processor(image, question, return_tensors="pt").to(device)

with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=60)

print(processor.decode(output[0], skip_special_tokens=True))

Intended use

This model is intended for research and prototyping of image-grounded waste identification and waste-management question answering, with questions framed around Indian waste-management practices.

Limitations and responsible use

• The question list describes coverage in the provided evaluation split; it is not proof that the model answers every question type accurately.
• Some reference answers may be incomplete, inconsistent, or model-generated. Inspect annotations and evaluate performance before making accuracy claims.
• Bin colours, collection arrangements and legal requirements can depend on the applicable rules and local authority. Do not treat model outputs as authoritative legal guidance.
• Hazard ratings and decomposition estimates are approximate and may not be inferable from an image alone.
• Visible appearance cannot confirm hidden contamination, chemical composition, or whether an item is safe to handle.
• Do not rely on this model alone for medical waste, sharps, hazardous substances, or other safety-critical disposal decisions.

Citation

Please cite the associated research paper and dataset if you use this model. Verify the official paper title and author metadata before publication.

@article{wasteassistant2026,
  title = {WasteAssistant},
  journal = {arXiv preprint arXiv:2607.10610},
  year = {2026},
  url = {https://arxiv.org/abs/2607.10610}
}

License

This repository declares the MIT license. Before public release, confirm that the checkpoint, training code, and underlying image/annotation sources can legally be redistributed under the selected license. Source datasets may have separate terms.