Literature Retrieval for Precision Medicine with Neural Matching and Faceted Summarization (2020)

Abstract: Information retrieval (IR) for precision medicine (PM) often involves looking for multiple pieces of evidence that characterize a patient case. This typically includes at least the name of a condition and a genetic variation that applies to the patient. Other factors such as demographic attributes, comorbidities, and social determinants may also be pertinent. As such, the retrieval problem is often formulated as ad hoc search but with multiple facets (e.g., disease, mutation) that may need to be incorporated. In this paper, we present a document reranking approach that combines neural query-document matching and text summarization toward such retrieval scenarios. Our architecture builds on the basic BERT model with three specific components for reranking: (a). document-query matching (b). keyword extraction and (c). facet-conditioned abstractive summarization. The outcomes of (b) and (c) are used to essentially transform a candidate document into a concise summary that can be compared with the query at hand to compute a relevance score. Component (a) directly generates a matching score of a candidate document for a query. The full architecture benefits from the complementary potential of document-query matching and the novel document transformation approach based on summarization along PM facets. Evaluations using NIST’s TREC-PM track datasets (2017–2019) show that our model achieves state-of-the-art performance. To foster reproducibility, our code is made available here: https://github.com/bionlproc/text-summ-for-doc-retrieval .

## introduction the u.s. nih's precision medicine (pm) initiative @xcite calls for designing treatment and preventative interventions considering genetic, clinical, social, behavioral, and environmental exposure variability among patients. the initiative rests on the widely understood finding that considering individual variability is critical in tailoring healthcare interventions to achieve substantial progress in reducing disease burden worldwide. cancer was chosen as its near term focus with the eventual aim of expanding to other conditions. as the biomedical research enterprise strives to fulfill the initiative's goals, computing needs are also on the rise in drug discovery, predictive modeling for disease onset and progression, and in building nlp tools to curate information from the evidence base being generated. in a dovetailing move, the u.s. nist's trec (text retrieval conference) has been running a pm track since 2017 with a focus on cancer @xcite . the goal of the trec-pm task is to identify the most relevant biomedical articles and clinical trials for an input patient case. each case is composed of (1) a disease name, (2) a gene name and genetic variation type, and (3) demographic information (sex and age). ## background the basic reranking architecture we begin with is the bidirectional encoder representations from transformers (bert) @xcite model. bert is trained on a masked language modeling objective on a large text corpus such as wikipedia and bookscorpus. as a sequence modeling method, it has achieved state-of-the-art results in a wide range of natural language understanding (nlu) tasks, including machine translation @xcite ) and text summarization @xcite . with an additional layer on top of a pretrained bert model, we can fine-tune models for specific nlu tasks. in our study, we utilize this framework in all three components identified in section 1.3 by starting with a bert-base-uncased pretrained huggingface model @xcite . ## models and reranking in this effort, toward document reranking, we aim to measure the relevance match between a document and a faceted pm query. each training instance is a 3-tuple (d, q, y d q ) where q is a query, d is a candidate document, and y d q is a boolean human adjudicated outcome: whether d is relevant to q. as mentioned in section 1.3, first, we fine-tune bert for a query-document relevance matching task modeled as a classification goal to predict y d q (rel). next, we fine-tune bert for token-level relevance classification, different from rel, where a token in d is deemed relevant during training if it occurs as part of q. we name this model ext for keyword extraction. lastly, we train a bert model in the seq2seq setting where the encoder is initialized with a pretrained ext model. the encoder reads in d, and the decoder attends to the contextualized representations of d to generate a facet-specific pseudo-query sentence q d , which is then compared with the original query q. we conceptualize this process as text summarization from a document to query sentences foot_2 and refer to it as abs. all three models are used together to rerank a candidate d at test time for a specific input query. ## evaluations and results we conducted both quantitative and qualitative evaluations with example outcomes. the final evaluation was done on the 2019 trec-pm dataset while all hyperparameter tuning was done using a training and validation dataset split of a shuffled combined set of instances from 2017 and 2018 tracks (20% validation and the rest for training). ## conclusion in this paper, we proposed an ensemble document reranking approach for pm queries. it builds on pretrained bert models to combine strategies from document relevance matching and extractive/abstractive text summarization to arrive at document rankings that are complementary in eventual evaluations. our experiments also demonstrate that entity embeddings trained on an annotated domain specific corpus can help in document retrieval settings. both quantitative and qualitative analyses throw light on the strengths of our approach. one scope for advances lies in improving the summarizer to generate better pseudo-queries so that abs starts to perform better on its own. at a high level, training data is very hard to generate in large amounts for ir tasks in biomedicine and this holds for the trec-pm datasets too. to better train abs, it may be better to adapt other biomedical ir datasets. for example, the trec clinical decision support (cds) task that ran from 2014 to 2016 is related to the pm task @xcite . a future goal is to see if we can apply our neural transfer learning @xcite and domain adaptation @xcite efforts to repurpose the cds datasets for the pm task. another straightforward idea is to reuse generated pseudo-query sentences in the edismax query by solr, as a form of pseudo relevance feedback. the s cos expression in section 3.4 focuses on an asymmetric formulation that starts with a query term and looks for the best match in the pseudoquery. considering a more symmetric formulation, where, we also begin with the pseudo-query terms and average both summands may provide a better estimate for reranking. additionally, a thorough exploration of how external biomedical knowledge bases @xcite can be incorporated in the neural ir framework for pm is also important @xcite . gastric our cancer erbb2 carcinoma cases cases cases expression from gastric cancer tumors cancer cancer cancer (a) attention heatmap produced by [unused0] signal (topic of disease) amplification common erbb2 erbb2 amplification of erbb2 cancer amplification (b) attention heatmap produced by [unused1] signal (topic of generic variants and gene regulations) figure 4: comparison between the attention heatmaps on a sample document conditioned by field signals in abs model.