  We propose and demonstrate a representation learning approach by maximizing
the mutual information between local features of images and text. The goal of
this approach is to learn useful image representations by taking advantage of
the rich information contained in the free text that describes the findings in
the image. Our method learns image and text encoders by encouraging the
resulting representations to exhibit high local mutual information. We make use
of recent advances in mutual information estimation with neural network
discriminators. We argue that, typically, the sum of local mutual information
is a lower bound on the global mutual information. Our experimental results in
the downstream image classification tasks demonstrate the advantages of using
local features for image-text representation learning.
