  Disease classification relying solely on imaging data attracts great interest
in medical image analysis. Current models could be further improved, however,
by also employing Electronic Health Records (EHRs), which contain rich
information on patients and findings from clinicians. It is challenging to
incorporate this information into disease classification due to the high
reliance on clinician input in EHRs, limiting the possibility for automated
diagnosis. In this paper, we propose \textit{variational knowledge
distillation} (VKD), which is a new probabilistic inference framework for
disease classification based on X-rays that leverages knowledge from EHRs.
Specifically, we introduce a conditional latent variable model, where we infer
the latent representation of the X-ray image with the variational posterior
conditioning on the associated EHR text. By doing so, the model acquires the
ability to extract the visual features relevant to the disease during learning
and can therefore perform more accurate classification for unseen patients at
inference based solely on their X-ray scans. We demonstrate the effectiveness
of our method on three public benchmark datasets with paired X-ray images and
EHRs. The results show that the proposed variational knowledge distillation can
consistently improve the performance of medical image classification and
significantly surpasses current methods.
