AMANDA : Agentic Medical Knowledge Augmentation for Data-Efficient Medical Visual Question Answering (2025)

Abstract: Medical Multimodal Large Language Models (Med-MLLMs) have shown great promise in medical visual question answering (Med-VQA). However, when deployed in low-resource settings where abundant labeled data are unavailable, existing Med-MLLMs commonly fail due to their medical reasoning capability bottlenecks: (i) the intrinsic reasoning bottleneck that ignores the details from the medical image; (ii) the extrinsic reasoning bottleneck that fails to incorporate specialized medical knowledge. To address those limitations, we propose AMANDA, a training-free agentic framework that performs medical knowledge augmentation via LLM agents. Specifically, our intrinsic medical knowledge augmentation focuses on coarse-to-fine question decomposition for comprehensive diagnosis, while extrinsic medical knowledge augmentation grounds the reasoning process via biomedical knowledge graph retrieval. Extensive experiments across eight Med-VQA benchmarks demonstrate substantial improvements in both zero-shot and few-shot Med-VQA settings. The code is available at https://github.com/REAL-Lab-NU/AMANDA .

## introduction medical visual question answering (med-vqa) aims to automatically answer natural language questions about medical images, serving as an aipowered assistant to enhance healthcare professionals' diagnostic efficiency and accuracy @xcite . unlike general-domain vqa which focuses on everyday scenes and objects, med-vqa requires finegrained analysis of subtle pathological features, understanding of professional medical terminology, and integration of domain-specific medical knowledge @xcite . these unique characteristics make med-vqa particularly challenging yet * co-corresponding authors. recent advances in medical multimodal large language models (med-mllms) have demonstrated promising results in med-vqa through extensive pre-training and task-specific fine-tuning @xcite @xcite . however, obtaining a largescale medical dataset for med-mllm pre-training or fine-tuning requires labor-intensive expert annotations, making it impractical in data-efficient scenarios. when deployed in low-resource settings where abundant training or fine-tuning data are unavailable (i.e., zero-shot or few-shot settings), existing med-mllms commonly fail due to two bottlenecks of their medical reasoning capability: to address the aforementioned challenges, we present a training-free mllm agentic framework -amanda (agentic medical knowledge augmentation) for data-efficient medical visual question answering. in essence, our framework enhances med-mllms' reasoning capability through medical knowledge augmentation (med-ka) from both intrinsic and extrinsic reasoning perspectives. on the one hand, to enhance the medical reasoning depth, we propose intrinsic med-ka, which leverages a coarse-to-fine question decomposition strategy to fully utilize the intrinsic visual understanding capabilities within med-mllms, enabling comprehensive diagnosis through progressive examination. on the other hand, to bridge the gap between models' pre-trained knowledge and reliable medical expertise, we develop extrinsic med-ka, which retrieves relevant medical knowledge from biomedical knowledge graphs to ground the reasoning process. these complementary approaches are orchestrated by multiple llm agents that can adaptively control the depth of knowledge integration to maintain both effectiveness and efficiency. in addition, amanda can incorporate in-context learning examples, enabling further performance gains in few-shot settings. overall, our contributions can be summarized as follows: ## proposed approach -amanda while existing med-mllms have shown promise, they often struggle in data-efficient settings due to critical reasoning and knowledge bottlenecks. to address these limitations, we introduce amanda, a training-free, agentic framework specifically designed for med-vqa. in this section, we first formalize the med-vqa problem and present our amanda framework (sec. 3.1 and 3.2). we then detail our med-ka approaches (sec. 3.3) and present two extensions: the adaptive reasoning refinement mechanism and the few-shot enhancement strategies (sec. 3.4). ## conclusion in this work, we present amanda, a training-free agentic framework that addresses med-mllms' intrinsic and extrinsic bottlenecks in data-efficient scenarios. our framework enhances medical visual reasoning through coarse-to-fine question decomposition and grounds its analysis with extrinsic knowledge graphs, while maintaining efficiency through adaptive reasoning refinement. extensive experiments demonstrate substantial improvements on med-vqa in both zero-shot and few-shot settings, highlighting amanda's potential for reliable ai-assisted medical diagnosis in resourceconstrained environments.