A Deep Metric Learning Method for Biomedical Passage Retrieval (2020)

Abstract: Passage retrieval is the task of identifying text snippets that are valid answers for a natural language posed question. One way to address this problem is to look at it as a metric learning problem, where we want to induce a metric between questions and passages that assign smaller distances to more relevant passages. In this work, we present a novel method for passage retrieval that learns a metric for questions and passages based on their internal semantic interactions. The method uses a similar approach to that of triplet networks, where the training samples are composed of one anchor (the question) and two positive and negative samples (passages). However,and in contrast with triplet networks, the proposed method uses a novel deep architecture that better exploits the particularities of text and takes into consideration complementary relatedness measures. Besides, the paper presents a sampling strategy that selects both easy and hard negative samples which improves the accuracy of the trained model. The method is particularly well suited for domain-specific passage retrieval where it is very important to take into account different sources of information. The proposed approach was evaluated in a biomedical passage retrieval task, the BioASQ challenge, outperforming standard triplet loss substantially by 10%,and state-of-the-art performance by 26%.

## introduction scientific documents are a valuable source of information for both physicians and researchers in medical sciences. relevant information is continuously produced, more than 3,000 articles are published every day @xcite . manually finding relevant information in this among of data is an enormous challenge @xcite . passage retrieval methods can alleviate the manual scanning of documents by automatically finding a subset of relevant passages that speed-up and improve the search process. their goal is to return the highest correlated passages that are a valid answer for a given question. metric learning has been broadly used in face identification and other image processing tasks. this approach has a powerful and simple mathematical formulation that allows to produce a compact representation in a metric space that can be used to identify image correspondences. the same idea can be applied to the passage retrieval task where answer passages should share semantic patterns with the question and this can be measured by a metric in an appropriate metric space. this idea has not been explored in depth in the context of passage retrieval, except for the work of @xcite , where a siamese network was used for learning a metric between questions and candidate answers in an open-domain question answering task on a proprietary dataset. this paper presents a novel deep metric learning method that learns a metric between question and passages bringing close semantically related pairs. most of the metric learning approaches learn to embed samples in a latent space where a metric (usually euclidean) captures relationships between samples. the proposed approach directly learns the metric fusing different similarity measures through a siamese convolutional deep learning architecture. also, the paper presents a sampling strategy that chooses easy and then hard negative samples in the training phase, improving the overall model performance. the experimental results show that the method is able to induce a metric between questions and passages that helps to discriminate relevant passages from non-relevant passages. the proposed architecture is similar to a triplet network (because of the three inputs: question, answer passage, non-answer passage) and also to a siamese architecture because it is composed of two convolutional neural networks with shared weights. however, different from these, it allows to extract important semantic features from several question-passage internal similarity measures that provide a complementary view of their relatedness. the similarity measures include a structured view of the question and passage, incorporating valuable information that is usually available in close domain problems. to validate the model performance we carried out a systematic evaluation considering a widely used domain-specific collection, the bioasq dataset @xcite , and comparing with stateof-the-art models. the results show that the performance of the proposed model outperforms previous approaches with a wide margin. the main contributions of this work are the following: it is important to highlight that the proposed model could be easily implemented, and the number of its parameters is much less than in the state-of-the-art models @xcite , which have in the order of millions while ours in the order of thousands. the paper is organized as follows: section 2 discusses the related work in biomedical passage retrieval; section 3 shows the details of the proposed metric learning method; section 4 present the sampling strategy; section 5 presents a systematic evaluation of the method; section 6 discusses the results against the state of the art models; finally, section 7 exposes some conclusions and discusses our future work ideas. ## related work passage retrieval methods analyze the content of documents to identify snippets that likely answer a given specific question. this is a sub-task of the more general problem of question answering (qa). it has been extensively studied in open domain scenarios @xcite , but has received less attention in the biomedical domain. the launch of the biomedical qa track at bioasq has boosted the research in this specific domain @xcite . bioasq challenge is the widest challenge for biomedical indexing and information retrieval; for five consecutive years, they have shared a set of questions and related documents with annotated answer passages that contribute to research in this area. numerous passage retrieval approaches have been proposed at bioasq. for example, @xcite proposed to apply a word embedding representation for question-passage sequences and then to compute their semantic relationship employing a weighted cosine distance. another relevant approach, which obtained the best results in the 2018 bioasq edition, was presented by the auen-nlp team @xcite . this approach is based on an abcnn architecture @xcite , which models pair of sentences with a convolutional neural model and an attention mechanism, and uses a linear classification layer to produce an output relevance score. the ustb team approach combine different strategies to enrich query terms, such as sequential dependence models, pseudorelevance, fielded sequential dependence models and divergence from randomness models @xcite . finally, @xcite used bert contextual word embeddings @xcite to represent question and passage pairs, and fine-tuned the model to produce a ranking score. most recent works have employed pre-trained transformers language models that are fine-tuned on the downstream classification task. it is remarkable that although the biomedical domain is plenty of structured knowledge as biomedical terminology databases and ontologies, most of the approaches have not made use of these resources @xcite ). an exception is the work presented by @xcite , where ontologies are used to expand query terms. structured resources offer information that is complementary to textual information and that can be used to alleviate problems as polysemy or synonymy disambiguation. this work proposes a deep learning approach that takes as input different similarity representations that came from text and structured information. the proposed representations offer a diverse and complete view of question and passage interactions, which are then transformed into patterns by convolutional filters, to be finally projected into a metric learning space which locates question and related passages close to each other. the non-related passages are distant from the question with a minimum margin constrain. metric learning has been used with success in medical diagnosis, image classification, voice recognition, among others @xcite . the use of metric learning in natural language processing tasks has gained attention in the last years with approaches like the one presented by @xcite , where a smoothed deep metric loss function is considered to identify repeated questions for open-domain community qa portals. this work is one of the few approaches that employ metric learning in the passage retrieval task. another approach was presented by @xcite where a metric learning approach based on a convolutional neural siamese architecture is applied to question and passage sequences. an important difference of this work with ours is that the convolutional neural network represent individually questions and answers following the architecture of a vanilla siamese network. in the model presented in this paper, the convolutional neural network learns to represent the interactions between questions and answers in an architecture that combines ideas from siamese and triplet networks. ## deep metric learning for passage retrieval (dmlpr) the traditional deep metric learning approach is composed of two steps. first, a deep neural model is trained to learn a mapping from a given data representation (commonly images) to an euclidean space, then euclidean distances in the learned spaces are expected to measure the dissimilarity between objects @xcite . the first deep metric learning approaches used a siamese architecture, where the model receives a pair question-answer and each component is mapped to the euclidean space by the same neural network. an evolution of this architecture was the triplet network, where the model receives triplets instead of pairs. the triplets consist of two matching examples (positive and anchor) and one non-matching sample (negative). for both siamese and triplet networks, each sample is individually mapped to to the embedding space. in contrast to the classic metric learning approach, which learns a metric embedding space for individual samples, our approach learns a combined question-passage embedding that codifies the pair relatedness. the proposed architecture is described in detail in the following sections. ## informative negative passage identification selecting informative training samples is very important in deep metric learning, as it is described in previous works @xcite . our approach discriminate hard negative samples based on the semantic relatedness of question and passage pairs using the cosine similarity over biosentvec sentence embeddings @xcite . during training, we first feed the model with easy negative samples, and then with hard negative samples that are more challenging to classify. the process to filter hard and easy training samples is as follows: ## conclusion we present a novel deep-metric learning approach for biomedical passages retrieval that surpasses previous approaches evaluated in the bioasq dataset. the model presents innovations in terms of the architecture that combines multi-similarity representation, a cnn, and a siamese design, as well as in terms of the training strategy that identify hard and easy negative samples which are used to gradually train the model. motivated by the results obtained, future work will focus on exploring other alternatives to fuse information coming from structured knowledge sources. it will also be important to experiment with other forms of metric learning approaches.