Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,94 @@
+---
+base_model: Qwen/Qwen2.5-VL-3B-Instruct
+library_name: transformers
+pipeline_tag: image-text-to-text
+tags:
+- qwen2.5-vl
+- lora
+- sft
+- context-classification
+- out-of-context-detection
+- coinco
+license: cc-by-4.0
+---
+
+# COinCO Context Classification Models
+
+**Authors:** Tianze Yang\*, Tyson Jordan\*, Ruitong Sun\*, Ninghao Liu, Jin Sun
+\*Equal contribution
+**Affiliation:** University of Georgia
+
+## Overview
+
+Fine-grained context classification models for detecting **out-of-context objects** in images. Each model is a fully merged Qwen2.5-VL-3B-Instruct fine-tuned via LoRA on the [COinCO dataset](https://huggingface.co/datasets/COinCO/COinCO-dataset).
+
+The models classify whether an object (marked by a red bounding box) is **in-context** or **out-of-context** based on three criteria:
+
+| Model | Criterion | Description |
+|-------|-----------|-------------|
+| `co_occurrence/` | Co-occurrence | Whether the object can reasonably appear together with other objects in the scene |
+| `location/` | Location | Whether the object is placed in a physically and contextually reasonable position |
+| `size/` | Size | Whether the object's size is proportional and realistic relative to other objects |
+
+## How to Use
+
+```python
+from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
+import torch
+
+# Choose a model: "co_occurrence", "location", or "size"
+model_id = "COinCO/Context_Classification_Models"
+subfolder = "co_occurrence"  # or "location" or "size"
+
+model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
+    model_id,
+    subfolder=subfolder,
+    torch_dtype=torch.float16,
+    device_map="auto",
+)
+processor = AutoProcessor.from_pretrained(model_id, subfolder=subfolder)
+```
+
+## Training Details
+
+- **Base Model:** [Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct)
+- **Method:** LoRA fine-tuning (merged into base model)
+- **Dataset:** [COinCO](https://huggingface.co/datasets/COinCO/COinCO-dataset) inpainted images with multi-model consensus labels
+- **Training Data:** ~5,000 samples per criterion from the training split
+- **Epochs:** 3
+- **Learning Rate:** 2e-4
+- **LoRA Rank:** See adapter config for details
+
+## Evaluation Results
+
+### Inpainted Test Set (binary classification: In-context vs Out-of-context)
+
+| Criterion | Baseline (Qwen2.5-VL-3B) | Fine-tuned | Improvement |
+|-----------|--------------------------|------------|-------------|
+| Co-occurrence | 75.54% | **80.82%** | +5.28% |
+| Location | 74.43% | 71.05% | -3.38% |
+| Size | 50.21% | **66.01%** | +15.80% |
+
+### Real COCO Images (shortcut learning detection, higher = less shortcut reliance)
+
+| Criterion | Baseline | Fine-tuned | Improvement |
+|-----------|----------|------------|-------------|
+| Co-occurrence | 88.95% | 87.00% | -1.95% |
+| Location | 47.55% | **91.35%** | +43.80% |
+| Size | 52.55% | **83.20%** | +30.65% |
+
+## Related Resources
+
+- **Paper:** "Common Inpainted Objects In-N-Out of Context"
+- **Dataset:** [COinCO/COinCO-dataset](https://huggingface.co/datasets/COinCO/COinCO-dataset)
+- **Code:** [YangTianze009/COinCO](https://github.com/YangTianze009/COinCO)
+
+## Citation
+
+```bibtex
+@article{yang2025coinco,
+  title={Common Inpainted Objects In-N-Out of Context},
+  author={Tianze Yang and Tyson Jordan and Ruitong Sun and Ninghao Liu and Jin Sun},
+  year={2025}
+}
+```