Learning to ground medical text in a 3 D human atlas (2020)

Abstract: In this paper, we develop a method for grounding medical text into a physically meaningful and interpretable space corresponding to a human atlas. We build on text embedding architectures such as Bert and introduce a loss function that allows us to reason about the semantic and spatial relatedness of medical texts by learning a projection of the embedding into a 3D space representing the human body. We quantitatively and qualitatively demonstrate that our proposed method learns a context sensitive and spatially aware mapping, in both the inter-organ and intra-organ sense, using a large scale medical text dataset from the “Large-scale online biomedical semantic indexing” track of the 2020 BioASQ challenge. We extend our approach to a self-supervised setting, and find it to be competitive with a classification based method, and a fully supervised variant of approach.

## introduction the quantity of available medical literature increases daily @xcite , however, it is often provided in a non-systematized, free form. the development of bert @xcite , and the increased popularity of transfer learning in natural language processing (nlp), prompted notable works that aim to leverage publicly available medical and scientific articles to develop domain specific pre-trained language models @xcite @xcite . high quality sentence representations that capture the semantics and structure of the text can be obtained by training models to solve the natural language inference (nli) task on open-domain datasets @xcite and predict whether two pieces of text entail, contradict or are neutral to * equal contribution figure 1 : given the text with implicit reference to lungs: "divided into two lobes, an upper and a lower lobe, by the oblique fissure, which extends from the costal to the mediastinal surface" @xcite , our model learns the grounding indicated by the star. each other @xcite . the aforementioned bert-based models can serve as the encoder backbone for such approaches @xcite , and the setup can be trivially extended to enable searching through and retrieving relevant documents from large datasets. despite proving useful in a variety of settings, these works suffer from the following limitations: by contrast, we propose a method that embeds medical text into a universal, small dimensional space corresponding to the human body that is easy to navigate and interpret (figure 1 ). the propensity of functionally similar organs towards being physically close represents an inductive bias that can be leveraged for computing compact, 3d text representations that are competitive with standard higher dimensional text embeddings. additionally, our approach allows to search through and retrieve documents grounded within the physical space of the human body. to that end, our contributions are: we perform extensive evaluation to measure the performance of our method in two scenarios, namely, grounding in the human atlas (relevant for visualization and navigation), and medical text-totext retrieval (directly assessing the performance of our model in an information retrieval setting). furthermore, we set up an experimental setting explicitly tailored to measure the spatial reasoning ability of our model within an organ, a setting never directly imposed during training. we empirically demonstrate that our method is highly successful in all aforementioned experimental settings, effectively addressing the previously stated limitations. the codebase and the trained models are released at: www.github.com/gorjanradevski/ text2atlas 2 related work (medical) text embeddings. before the development of bert, a common approach to embedding text was leveraging a pre-trained recurrent neural network (rnn) language model (lm) @xcite . an extension of such lm for the biomedical domain is bioelmo @xcite . despite being successful in a transfer learning setting, the usefulness of the generated embeddings for medical text navigation and retrieval is arguably limited. furthermore, bert-based medical language representation models such as biobert @xcite and clinicalbert @xcite , despite outperforming rnn based lms on a variety of downstream tasks, also make embedded text navigation impractical. we, on the other hand, directly focus on learning embeddings that are rich with visual information, i.e., are by default represented in a physically meaningful space of the human body. ## experimental setup we use bert base @xcite as the backbone of the trained models. we use adamw @xcite with a learning rate of 10 -5 as per @xcite , weight decay of 10 -2 and clip the gradients when the global norm exceeds 2.0. we perform early stopping by saving the model with the best performance on the validation set. we only tune the hyperparameters related to the sod loss function, and we keep everything else fixed as per the standard practice @xcite . our implementation utilizes pytorch @xcite and the huggingface transformers library @xcite . ## evaluation we quantitatively evaluate our trained models in two scenarios: (1) grounding to the human atlas -measuring to what extent our trained model can ground medical articles to the correct location.