Create README.md

README.md

@@ -0,0 +1,41 @@
+---
+tags:
+- image-to-text
+- visual-question-answering
+- image-captioning
+datasets:
+- kaist-ai/volcano-train
+license: apache-2.0
+language:
+- en
+pipeline_tag: image-to-text
+library_name: transformers
+---
+## Links for Reference
+
+- **Repository:** 
+- **Paper:** 
+  
+# Overview
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/6550c4f27bbfce1878f5f280/AnqbCNf6pRiQ_5uNX0r4d.png)
+Volcano employs a single LMM to generate initial responses, feedback, and revisions, as well as decisions to accept revisions. It follows a sequential procedure of an iterative critique-revision-decide loop.
+
+# Model details
+
+**Model type:**
+Volcano-13b is a multimodal self-feedback guided revision model that was fine-tuned by mixing the visual instruction tuning dataset used in [LLaVA-v1.5](https://llava-vl.github.io/) with multimodal feedback and revision data collected through [gpt-3.5-turbo](https://platform.openai.com/docs/models/gpt-3-5), applied to the [vicuna-13b-v1.5](https://huggingface.co/lmsys/vicuna-13b-v1.5) model.
+
+**Model date:**
+Volcano-13b was trained in October 2023.
+
+# Training dataset
+- **274K multimodal feedback and revision data**
+- 558K filtered image-text pairs from LAION/CC/SBU, captioned by BLIP.
+- 158K GPT-generated multimodal instruction-following data.
+- 450K academic-task-oriented VQA data mixture.
+- 40K ShareGPT data
+
+You can find [here](https://huggingface.co/datasets/kaist-ai/volcano-train) the dataset used to train Volcano, which includes all the aforementioned datasets.
+
+# Evaluation dataset
+A collection of three multimodal hallucination benchmarks ([MMHal-Bench](https://huggingface.co/datasets/Shengcao1006/MMHal-Bench), [Pope](https://github.com/RUCAIBox/POPE), [GAVIE](https://github.com/FuxiaoLiu/LRV-Instruction)) and two multimodal understanding benchmarks ([MM-Vet](https://github.com/yuweihao/MM-Vet), [MMBench](https://github.com/open-compass/MMBench)).