# Generative Models in Medical Visual Question Answering: A Survey
## 5 Challenges and Future Directions in Generative MedVQA
### Evaluation Benchmarks and Metrics  
Recent benchmarks in MedVQA indicate key trends shaping the field. One major trend is the integration of more data modality, where datasets like MIMIC-CXR-VQA [56] combine chest X-ray images with structured EHR data to enhance the model's ability to perform multimodal medical reasoning. This approach allows MedVQA systems to understand both visual and textual aspects of medical records, improving their real-world applicability. As more structured clinical data become available, future benchmarks are likely to expand the range of medical knowledge that models can leverage. Another key development is the emphasis on reasoning and clinical context. Datasets such as MLe-VQA [27] introduce multi-level difficulty settings, testing models beyond simple recognition tasks and evaluating their ability to diagnose and reason through medical scenarios. Visual USMLE [25] adopts questions from USMLE that requires a higher-level of clinical understanding and reasoning ability. On the other hand, IAI-Med VQA [36] integrates user instructions to into model inference to align better with clinician's intent. This shift towards higher-order reasoning suggests that future benchmarks will place greater demands on models to interpret medical findings, integrate prior knowledge, and provide clinically meaningful responses rather than just factual retrieval. As the field matures, domain-specific benchmarks are becoming more prevalent, with datasets like O-VQA [58] focusing specifically on orthopedic medical records. Similarly, P-VQA [57] captures patient-oriented questions across multiple imaging modalities, making the dataset more relevant for real hospital settings. This specialization ensures that MedVQA models are fine-tuned for specific clinical applications, which could lead to AI systems that better support subspecialties in medicine such as cardiology, neurology, and dermatology. A notable shift is the growing use of automation in dataset creation, combined with expert validation to maintain data quality. IAI-Med VQA [36] demonstrates this approach by using AI-driven QA generation alongside manual expert validation to ensure the reliability of its 14,000 QA pairs. Similarly, MIMIC-CXR-VQA [56] leverages semi-automated template-based question generation reviewed by medical experts. This hybrid approach balances scalability and accuracy, making it a practical method for producing large-scale, clinically relevant datasets. Overall, these trends highlight the increasing complexity of MedVQA tasks and the evolving expectations for AI systems in clinical settings. Future benchmarks will likely continue expanding in scale, reasoning capability, and domain specialization, pushing AI towards more advanced clinical decision support applications. Evaluation metrics for MedVQA are evolving beyond traditional accuracy-based measures that assess the exact match between generated answers and ground truth. While BLEU remains a common choice for evaluating n-gram overlap, it fails to account for synonymy and semantic similarity, which are crucial in medical contexts. This limitation has led to the adoption of metrics such as BERTScore, which leverages contextualized embeddings from BERT to measure the similarity between generated and reference answers. Unlike BLEU, BERTScore can capture semantic equivalence and paraphrasing, making it a more suitable choice for medical applications. The reasonability of traditional NLP metrics (BLEU, ROUGE, BERTScore) that widely adopted in MedVQA can be further discussed, in terms that they fall short in capturing clinical correctness due to the following reasons. _(1) Surface-level alignment does not ensure clinical validity._ Metrics such as BLEU and ROUGE prioritize lexical or statistical overlap but ignore the contextual relevance and clinical appropriateness of the generated answers. A generated answer may achieve high BLEU scores yet contain life-threatening errors (e.g., misclassifying \"benign\" as \"malignant\").