# Generative Models in Medical Visual Question Answering: A Survey
## 2 The Evolution of Generative MedVQA
### Early Explorations of Generative MedVQA  
The early exploration of generative models in MedVQA marked a shift from traditional discriminative-based systems to fully generative systems, with an intermediate phase featuring hybrid approaches that combined both classification and generation. Some early models [30; 32] were fully generative systems that used decoders to generate answers for both open-ended and closed-ended questions. These models were pioneering efforts in exploring how generative approaches could address the complex and variable nature of medical question answering.  
VGG-Seq2 Seq [30], proposed for the ImageCLEF 2018 task [1], adopts an encoder-decoder architecture. The model combines a pretrained VGG network with an LSTM for encoding image and question inputs, followed by a simple concatenation of the visual and textual embeddings. The decoder, also LSTM-based, generates the answer sequentially.  
The VQA-Med 2020 task introduced a sequential model by [PERSON] and [PERSON]. [32]. This model uses VGG16 for image feature extraction and BERT for question encoding. It combines these features using Multimodal Factorized Bilinear Pooling (MFB) and refines them with self-attention. The decoder, based on LSTM, generates answers word-by-word, utilizing Bahdanau attention and GLOVE embeddings.  
MED-GPVS [17], a model for both MedVQA and object detection, uses ResNet50 and BERT for image and text feature extraction, respectively. The image feature is processed through DEtection TRansformer (DETR) to extract area descriptors. Vision and language features are fused in ViLBERT's co-attention layers, followed by a text decoder that generates answers for MedVQA tasks.  
In contrast, some models took a hybrid approach, using both classification and generation to handle different types of questions [6; 16; 31]. These models demonstrated the flexibility of combining classification and generation but still had to balance the strengths and weaknesses of both approaches.  
TUA1 [31] is a hybrid system that combines classification and generation models designed for the ImageCLEF VQA-Med 2019 task [39]. The model leverages Inception-ResNet-V2 for image feature extraction and BERT for question feature processing. The model concatenates the image and question embeddings and then uses classifiers for modality, plane, and organ system questions, while a sequence-to-sequence generator handles \"abnormality\" questions using LSTM with beam search.  
CGMVQA [6] is another hybrid model that performs both classification and answer generation tasks in a unified framework. It utilizes ResNet-152 for image feature extraction and simple concatenation for image-text fusion. The model employs a lightweight transformer-based architecture and is trained with a combination of cross-entropy loss for classification and mask-based loss for generative tasks. A beam search algorithm is incorporated for sequential answer generation.  
Lastly, another hybrid framework MedFuseNet [16] is a multimodal attention-based model. It uses ResNet-152 for image feature extraction and BERT for question encoding, with Multimodal Factorized Bilinear Pooling (MFB) for fusion. The answer generator employs an LSTM-based decoder with attention mechanisms and beam search for generating answers to open-ended \"abnormality\" questions in datasets like MED-VQA-2019 and PathVQA.  
These early generative models paved the way for the more advanced, large-scale models seen today. While each model introduced unique innovations, the common challenge was to balance the flexibility of generative systems with the need for structured and interpretable answers in the medical domain.