# Generative Models in Medical Visual Question Answering: A Survey
## 2 The Evolution of Generative MedVQA
### Explorations of Working with MLLMs  
MLLMs provide a natural extension of LLMs into the multimodal space, allowing models to process images and text natively. Unlike LLM-based MedVQA approaches that require explicit vision-language alignment, MLLMs already incorporate cross-modal understanding as part of their pretraining. The focus in MedVQA research has shifted towards optimizing these models through instruction tuning, modular architectures, and knowledge augmentation. By refining existing MLLMs such as Flamingo, LLaVA, and IDEFICS, researchers have enhanced zero-shot generalization, reasoning capabilities, and task adaptability. The key difference between the approaches discussed here and those in the previous section is that these works leverage existing MLLMs as the foundation, modifying them for better domain alignment and interpretability. The primary strategies, however, overlaps with those in the previous subsection, involving instruction tuning, knowledge augmentation, modular reasoning frameworks, and PEFT techniques to optimize these MLLMs for medical applications. This subsection reviews how MLLMs are being leveraged for MedVQA, examining novel training methodologies and fine-tuning strategies that maximize performance across medical datasets. Med-Flamingo [25] builds on the OpenFlamingo-9B framework, adapting it specifically for the medical domain. The architecture integrates a frozen LLaMA-7B language model and CLIP ViT/L-14 vision encoder through gated cross-attention layers and perceiver layers, specializing in few-shot settings. The model consists of 1.3 billion trainable parameters focused on cross-attention and perceiver layers, while 7 billion parameters remain frozen within the decoder and vision encoder, totaling 8.3 billion parameters. Med-Flamingo undergoes continued pretraining using paired and interleaved image-text data from PMC-OA [51] and the constructed Medical Textbook Dataset (MTB), aligning visual and textual representations for improved medical reasoning. The model is trained with memory-efficient optimizations, ensuring computational efficiency and scalability. LLaVA-Med [24] extends LLaVA from the general domain to medical applications. It employs a linear projection layer to map visual features extracted by a vision encoder (e.g., CLIP-ViT) into the embedding space of an LLM, functioning similarly to visual prefix tuning. The model is trained in two stages using curriculum learning. The first stage aligns biomedical concepts by training on 600K image-caption pairs sampled from PMC-15M, updating the projector to align visual and textual embeddings. The second stage involves instruction tuning with a dataset of 60K GPT-4-generated instruction-following samples, enabling the model to handle open-ended, conversational MedVQA tasks. MC-CoT [35] introduces a Modular Collaborative Chain-of-Thought (CoT) framework designed to improve zero-shot MedVQA by leveraging LLMs for reasoning and MLLMs for image interpretation. The architecture comprises three specialized medical modules: a radiology module for imaging modality and lesion localization, an anatomy module for structural identification, and a pathology module for clinical significance assessment. The workflow begins with an LLM-guided task assignment, where the LLM receives the question and a caption of the image to provide strategic guidance. The MLLM then extracts image-specific features before the LLM synthesizes the final answer. Unlike traditional fine-tuned MedVQA models, MC-CoT does not require task-specific pretraining, making it highly adaptable for zero-shot settings. HuatuoGPT-Vision [26] is a medical MLLM built on Yi-VL-34B [52], integrating medical visual knowledge into its architecture. To enhance domain-specific knowledge alignment, it is pretrained with structured knowledge-injection approach on the constructed PubMedVision dataset containing 1.3 million medical image-text pairs refined using GPT-4V-assisted data curation.