Self-Training Large Language and Vision Assistant for Medical Question Answering (2024)

Abstract: Large Vision-Language Models (LVLMs) have shown significant potential in assisting medical diagnosis by leveraging extensive biomedical datasets. However, the advancement of medical image understanding and reasoning critically depends on building high-quality visual instruction data, which is costly and labor-intensive to obtain, particularly in the medical domain. To mitigate this data-starving issue, we introduce Self-Training Large Language and Vision Assistant for Medical (STLLaVA-Med). The proposed method is designed to train a policy model (an LVLM) capable of auto-generating medical visual instruction data to improve data efficiency, guided through Direct Preference Optimization (DPO). Specifically, a more powerful and larger LVLM (e.g., GPT-4o) is involved as a biomedical expert to oversee the DPO fine-tuning process on the auto-generated data, encouraging the policy model to align efficiently with human preferences. We validate the efficacy and data efficiency of STLLaVA-Med across three major medical Visual Question Answering (VQA) benchmarks, demonstrating competitive zero-shot performance with the utilization of only 9% of the medical data.

## introduction large vision-language models (lvlms) have demonstrated impressive performance across a wide range of medical challenges @xcite @xcite by fine-tuning through biomedical visual instruction data. similar to general lvlms @xcite , existing methods tailored for biomedical tasks primarily focus on collecting high-quality medical data to enhance task generalization and visual understanding. however, collecting medical data necessitates specialized expertise from physicians and raises privacy concerns, making the process both time-consuming and costly. to address vqa-rad slake pvqa 59.56 60.35 32.68 37.12 46.88 57.69 40.84 46.69 55.23 52.90 11.92 12.03 this data-starving issue, recent studies @xcite have explored leveraging larger models/apis (e.g., @xcite ) to generate medical data. nevertheless, this kind of method does not fully resolve the high api costs @xcite associated with building instructional data and still requires large-scale pre-training data to align medical images and text (see fig. 1 left). to bridge the gap in medical data acquisition, we propose self-training large language and vision assistant for medicine (stllava-med), a new training pipeline that enables lvlms to automatically generate medical instruction data governed by direct preference optimization (dpo) @xcite . different from previous self-training approaches @xcite , which generate answers for fixed/pre-defined questions (e.g., summarization and report), this work automatically generates open-ended questions and answers them, to enhance the diversity of self-training data and further improve medical image reasoning. moreover, achieving precise control of the generated model response is also challenging due to its unsupervised nature @xcite @xcite . existing methods for gaining such steerability, such as reinforcement learning @xcite and 20052 : what type of tissue is shown in the image? ## self-training datasets medical lvlms @xcite @xcite generally adopt pre-training on massive medical data, to realize medical image-text alignment. however, the proposed stllava-med does not involve such a medical corpus pre-training, providing new insights into data efficiency. to gpt-4o guided preference data collection. we prompt π θ to predict two answers to each generated question. specifically, to ensure the difference between answers, we set the temperature scaling to 1.2, t @xmath0, and t @xmath1.95, encouraging the model to generate more diverse and nonrepetitive output. in previous research @xcite , the preference data were annotated by human annotators. in contrast, this work utilizes gpt-4o as a simulated expert since we observe its excellent biomedical performance @xcite and the best downstream task performance in ## conclusion this work has proposed a self-training visionlanguage assistant for medicine (stllava-med), a novel training pipeline designed to enhance the data efficiency of training lvlms for medical tasks. our approach prompts the policy model to selfgenerate instruction-answer pairs and label them by a larger language model, such as gpt-4o, for preference optimization. this process aims to enhance the medical reasoning capabilities of a smaller vision-language model, reducing the reliance on extensively annotated medical data and alleviating human experts in the medical field. experimental results on three benchmarks demonstrate that stllava-med achieves exceptional medical reasoning capabilities using medical data at a minimum level. we aspire for our work to inspire future research aimed at enhancing the efficiency of training lvlms in broad medical domains.