T rial2 V ec: Zero-Shot Clinical Trial Document Similarity Search using Self-Supervision (2022)

Abstract: Clinical trials are essential for drug development but are extremely expensive and time-consuming to conduct. It is beneficial to study similar historical trials when designing a clinical trial. However, lengthy trial documents and lack of labeled data make trial similarity search difficult. We propose a zero-shotclinical trial retrieval method, called Trial2Vec, which learns through self-supervision without the need for annotating similar clinical trials. Specifically, the meta-structure of trial documents (e.g., title, eligibility criteria, target disease) along with clinical knowledge (e.g., UMLS knowledge base) are leveraged to automatically generate contrastive samples. Besides, encodes trial documents considering meta-structure thus producing compact embeddings aggregating multi-aspect information from the whole document. We show that our method yields medically interpretable embeddings by visualization and it gets 15% average improvement over the best baselines on precision/recall for trial retrieval, which is evaluated on our labeled 1600 trial pairs. In addition, we prove the pretrained embeddings benefit the downstream trial outcome prediction task over 240k trials. Software is available at https://github.com/RyanWangZf/Trial2Vec.

## introduction clinical trials are essential for developing new medical interventions @xcite . many considerations come into the design of a clinical trial, including study population, target disease, outcome, drug candidates, trial sites, and eligibility criteria, as in recently, a burgeoning body of research @xcite @xcite proposes to execute self-supervised learning to train semantic-meaningful sentence embeddings free of labels. however, there are still challenges to apply them for document similarity search: in this work, we propose clinical trial to vectors, trial2vec, a zero-shot trial document similarity search using self-supervision. we design a trial encoding framework considering the meta-structure to rid the risk that semantic meaning vanishes due to the uniform average of token embeddings. meanwhile, the meta-structure is utilized to generate contrastive samples for efficient supervision. medical knowledge is introduced to further enhance the negative sampling for cl. our main contributions are: ## method in this section, we present the details of trial2vec. the main idea is to jointly learn the global and local representations from trial documents considering their meta-structure. specifically, observed in ## experiments in this section, we conduct five types of experiments to answer the following research questions: ## conclusion this paper investigated utilizing bert with selfsupervision for encoding trial into dense embeddings for similarity search. experiments show our method can succeed in zero-shot trial search under various settings. the embeddings are also useful for trial downstream predictive tasks. the qualitative analysis, including embedding space vi-sualization and case studies, further verifies that trial2vec gets a medically meaningful understanding of clinical trials.