# Generative Models in Medical Visual Question Answering: A Survey
## 4 Advances in Generative MedVQA
### Automated Dataset Building and Data Augmentation  
The scarcity of annotated datasets presents a significant challenge in advancing MedVQA systems. Medical data are often difficult to obtain, requiring expert annotations that are both time-consuming and costly. To overcome this limitation, scalable data generation methods, such as synthetic dataset creation and data augmentation, have emerged as vital tools. Automated dataset building not only addresses the scarcity of labeled data but also enhances diversity and robustness in model training, ensuring that generative MedVQA systems generalize better to unseen scenarios. This subsection explores key techniques in data augmentation and automated dataset building, including synthetic dataset generation, data augmentation on existing datasets, and retrieval-based enrichment to expand training data effectively. **Synthetic Dataset Generation** leverages automated methods to create large-scale datasets tailored for pretraining and fine-tuning MedVQA models. Frameworks such as Med-Flamingo [25], LLaVA-Med [24], and PMC-VQA [14] exemplify this approach. These models generate synthetic QA pairs using LLMs, MLLMs or domain-specific algorithms. The PMC-15M dataset [60] comprises 15 million biomedical image-text pairs sourced from PubMed Central. The dataset creation involves two distinct subsets: biomedical concept alignment data and instruction-following data. The concept alignment data include basic image-caption pairs curated to provide foundational visual-textual alignment. Meanwhile, the instruction-following data are generated using GPT-4, which produces conversational instruction-answer pairs based on image captions and their surrounding textual context. For example, the pipeline prompts GPT-4 to simulate questions and answers as if it could view the images, enriching the data with high-quality multi-turn conversations. This dataset covers a variety of imaging modalities and clinical contexts, ensuring robust pretraining for a broad spectrum of biomedical tasks. Together, these datasets provide the necessary diversity and detail to train a versatile and capable multimodal assistant. The MTB (Medical Textbook Benchmark) dataset [25] was constructed to evaluate and fine-tune the performance of Med-Flamingo in medical contexts by leveraging structured knowledge from medical textbooks. This dataset consists of image-text pairs extracted and curated from authoritative medical literature, including diagnostic atlases and foundational medical textbooks. Each data point is designed to simulate real-world clinical scenarios, comprising visual inputs like medical images (e.g., X-rays, CT scans, histopathology slides) paired with text that includes captions, explanations, and contextual information. The dataset captures a wide range of medical domains, from anatomy and radiology to pathology and surgery, ensuring a comprehensive scope. To maintain quality and relevance, medical experts reviewed and annotated the dataset, focusing on clarity, accuracy, and clinical applicability. By providing high-quality multimodal data, the MTB dataset facilitates pretraining and evaluation of models like Med-Flamingo, ensuring they can generalize across diverse medical imaging tasks and deliver clinically relevant insights. The Visual USMLE dataset [25] draws inspiration from the format and structure of the United States Medical Licensing Examination (USMLE) and is curated to assess the model's ability to handle real-world, complex medical reasoning tasks, encompassing open ended questions that require advanced understanding of medical concepts. The dataset includes a diverse array of question types, such as diagnosis-based queries, treatment recommendations, and pathology explanations, each paired with supporting images like X-rays, CT scans, or MRIs. The dataset contains 618 USMLE-style questions modified from the real USMLE questions from the Amboss platform. The original multiple-choice questions are turned into open-ended questions to increase the difficulty.