That’s the Wrong Lung! Evaluating and Improving the Interpretability of Unsupervised Multimodal Encoders for Medical Data (2022)

Abstract: Pretraining multimodal models on Electronic Health Records (EHRs) provides a means of learning representations that can transfer to downstream tasks with minimal supervision. Recent multimodal models induce soft local alignments between image regions and sentences. This is of particular interest in the medical domain, where alignments might highlight regions in an image relevant to specific phenomena described in free-text. While past work has suggested that attention “heatmaps” can be interpreted in this manner, there has been little evaluation of such alignments. We compare alignments from a state-of-the-art multimodal (image and text) model for EHR with human annotations that link image regions to sentences. Our main finding is that the text has an often weak or unintuitive influence on attention; alignments do not consistently reflect basic anatomical information. Moreover, synthetic modifications — such as substituting “left” for “right” — do not substantially influence highlights. Simple techniques such as allowing the model to opt out of attending to the image and few-shot finetuning show promise in terms of their ability to improve alignments with very little or no supervision. We make our code and checkpoints open-source.

## introduction there has been a flurry of recent work on model architectures and self-supervised training objectives for multimodal representation learning, both generally @xcite @xcite @xcite and for medical data specifically @xcite @xcite . these methods yield representations that permit efficient learning on various multimodal downstream tasks (e.g., classification, captioning). given the inherently multimodal nature of much medical data -e.g., in radiology images and text are naturally paired -there has been particular interest in designing multimodal models for electronic health records (ehrs) data. however, one of the factors that currently stands in the way of broader adoption is interpretability. neural models that map image-text pairs to shared representations are opaque. consequently, doctors have no way of knowing whether such models rely on meaningful clinical signals or data artifacts @xcite . recent work has proposed models that soft-align text snippets to image regions. this may afford a type of interpretability by allowing practitioners to inspect what the model has "learned" or allow more efficient identification of relevant regions. past work has presented illustrative multimodal "saliency" maps in which such models highlight plausible regions. but such highlights also risk providing a false sense that the model "understands" more than it actually does, and irrelevant highlights would be antithetical to the goal of a efficiency in clinical decision support. multimodal models may fail in a few obvious ways; they may focus on the wrong part of an image, fail to localize by producing a high-entropy attention distribution, or localize too much and miss a larger region of interest. however, even when image attention appears reasonable, it may not in actuality reflect both modalities. figure 1 shows an example. here the model ostensibly succeeds at identifying the image region relevant to the given text (left). one may be tempted to conclude the model has "understood" the text and indicated the corresponding region. but this may be misleading: we can see that the same model yields a similar attention pattern when provided text with radically different semantics (e.g., when swapping "right" with "left"), or when providing sentences referencing an abnormality in another region. our contributions are as follows. (i) we appraise the interpretability of soft-alignments induced between images and texts by existing neural multimodal models for radiology, both retrospectively and via manual radiologist assessments. to the best of our knowledge, this is the first such evaluation. (ii) we propose methods that improve the ability of multimodal models for ehr to intuitively align image regions with texts. ## preliminaries we aim to evaluate the localization abilities of multimodal models for ehr. for this we focus on the recently proposed gloria model @xcite , which is representative of state-of-the-art, transformer-based multimodal architectures and accompanying pre-training methods. for completeness we also analyze (a modified version of) uniter @xcite . we next review details of these models, and then discuss the datasets we use to evaluate the alignments they induce. ## are alignment weights accurate? we first use the metrics defined above to evaluate the pretrained, publicly available weights for gloria @xcite . small left pleural effusion is stable. ## can we improve alignments? the above results indicate that image attention is unintuitive and less sensitive to the text modality than might be expected. next we propose simple methods to try to improve image/text alignments. ## related work work on multi-modal representation learning for medical data has proposed soft aligning modalities, but has focussed quantitative evaluation on the resultant performance that learned representations afford on downstream tasks @xcite @xcite . model interpretability is often suggested using only qualitative examples; our work aims to close this gap. a line of work in nlp evaluates the interpretability of neural attention mechanisms @xcite @xcite . elsewhere, work at the intersection of computer vision and radiology has critically evaluated use of saliency maps over images @xcite . recent work has sought to improve the ability of these models to identify fine-grained alignments via supervised attention @xcite , but have focused on downstream task performance. this differs from our focus on evaluating and improving localization itself, especially within the medical domain. we also do not assume access to large amounts of supervision, which is commonly lacking in this domain. ## conclusions we evaluated existing state-of-the-art unsupervised multimodal representation learning models for ehrs in terms of inducing fine-grained alignments between image regions and text. we found that the resultant heatmaps are often unintuitive and invariant to perturbations to the text that ought to change them substantially. we evaluated a number of methods aimed at improving this behavior, finding that: (1) allowing the model to refrain from attending to the image, and;