  We propose a selective learning method using meta-learning and deep
reinforcement learning for medical image interpretation in the setting of
limited labeling resources. Our method, MedSelect, consists of a trainable deep
learning selector that uses image embeddings obtained from contrastive
pretraining for determining which images to label, and a non-parametric
selector that uses cosine similarity to classify unseen images. We demonstrate
that MedSelect learns an effective selection strategy outperforming baseline
selection strategies across seen and unseen medical conditions for chest X-ray
interpretation. We also perform an analysis of the selections performed by
MedSelect comparing the distribution of latent embeddings and clinical
features, and find significant differences compared to the strongest performing
baseline. We believe that our method may be broadly applicable across medical
imaging settings where labels are expensive to acquire.
