# Hierarchical Modeling for Medical Visual Question Answering with Cross-Attention Fusion
## 4 Experiments
### Baseline and SOTA  
We compare our method with the baseline Hi-VQA, which also uses the Rad-restruct dataset. The difference between our method and Hi-VQA is that Hi-VQA does not employ any prompt module. It adopts a conventional VQA architecture, with its core process consisting of two key stages: feature extraction and fusion. In the feature extraction stage, a pre-trained EfficientNet-b5 [40] image encoder is used to extract global and spatial-aware local features, while a domain-specific RadBERT text encoder processes hierarchical text inputs, including historical question-answer pairs and the current question concatenated in the format of <Question> <SEP> <Answer>. In the feature fusion stage, the image encoding, RadBERT-encoded historical text, and current question text are concatenated in the order of <image> <history> <question>, and injected with hybrid position encodings (2D sinusoidal encoding for the image part to retain spatial coordinates, and 1D absolute position encoding for the text part) and four types of token type embeddings (distinguishing image, historical question, historical answer, and current question). The fusion module uses a single-layer Transformer, and the input is processed in a single-layer Transformer for cross-modal interaction, using traditional multi-head self-attention to simultaneously capture fine-grained associations between visual regions and medical terms (e.g., \"upper lobe of the lung\" and the corresponding image region) and the semantic constraints of historical answers on the current question (e.g., activating the \"degree\" attribute prediction when \"pneumonia exists\") [41]. Finally, based on the question type, Softmax single classification or Sigmoid multi-label classification is used to generate predictions in the restricted answer space, and an autoregressive mechanism is employed to use high-level predictions as the historical context of lower-level questions, ensuring the clinical rationality of structured reports through logical consistency. This provides a baseline for medical hierarchical VQA.  
The SOTA method, context-VQA, builds upon the Hi-VQA architecture by first using a free-text report summarized by the GPT [42] model as additional context. After passing through the text encoder, it is aligned with the image features through an attention-basedalignment module, and then fused with the question features in the same single-layer Transformer, and finally input into an MLP for answer prediction. Compared to Hi-VQA, it shows performance improvement, but using GPT incurs additional costs and time, and using a large GPT model instead of a medical pre-trained model may lose important medical information in the free-text medical report.  
In summary, both Hi-VQA and context-VQA are traditional multi-modal fusion methods using Transformer, where features of different modalities are concatenated and then input into the self-attention module. This is different from the cross-attention-based fusion method of HiCA-VQA. Moreover, HiCA-VQA uses a hierarchical prompt approach rather than introducing additional context, saving time and cost.