Automatic Biomedical Term Clustering by Learning Fine-grained Term Representations (2022)

Abstract: Term clustering is important in biomedical knowledge graph construction. Using similarities between terms embedding is helpful for term clustering. State-of-the-art term embeddings leverage pretrained language models to encode terms, and use synonyms and relation knowledge from knowledge graphs to guide contrastive learning. These embeddings provide close embeddings for terms belonging to the same concept. However, from our probing experiments, these embeddings are not sensitive to minor textual differences which leads to failure for biomedical term clustering. To alleviate this problem, we adjust the sampling strategy in pretraining term embeddings by providing dynamic hard positive and negative samples during contrastive learning to learn fine-grained representations which result in better biomedical term clustering. We name our proposed method as CODER++, and it has been applied in clustering biomedical concepts in the newly released Biomedical Knowledge Graph named BIOS.

## introduction a critical step for building a biomedical knowledge graph is clustering synonyms terms into concepts @xcite . after mining terms from the biomedical corpus or electronic medical records, these terms may belong to an existing concept dictionary or newly discovered concepts. it is hard for humans to link terms to an existing concept dictionary since the volume of the concept dictionary is huge. furthermore, it is almost impossible for humans to determine if one term is a new concept. embedding-based entity linking methods encode terms into a dense space and use similarities among ⇤ contributed equally. † corresponded author. 1 our codes and model will be released at https:// github.com/ganjinzero/coder. in this paper, we propose a probing experiment to evaluate term clustering on umls automatically. this experiment shows sapbert and coder have insufficient ability in term clustering. for better term clustering, we propose a dynamic sampling strategy that provides both hard positive and negative pairs to learn fine-grained terms embeddings named coder++. coder++ not only reserves the ability to normalize terms but also can distinguish different concepts with similar term names. coder++ shows decent ability on biomedical entity linking and a significant improvement on biomedical term clustering evaluation. ## related work automatic term clustering has long been discussed. traditional methods use statistical approaches to define similar terms and perform clustering. @xcite defines term similarity based on distributions and @xcite forms clusters based on co-occurrence in semantically coherent documents. @xcite uses markov logic for unsupervised concept clustering. recent researches focus on deep learning approaches, where biomedical term embeddings can be used for term clustering. @xcite identifies biomedical synonyms using word embeddings. sapbert @xcite and coder @xcite learn synonyms knowledge from umls to provide close embeddings for synonyms. in this work, we improve these embeddings by providing dynamic hard negative samples. ## term clustering evaluation we introduce the term clustering evaluation on umls as the probing experiment, in which we find that both coder and sapbert show poor clustering performance. through the case study, we find the reason is that both models fail to distinguish between fine-grained biomedical terms, which suggests a refinement is needed to support biomedical term clustering. ## approach we introduce coder++ to address the abovementioned problem. the idea is simple, providing hard negative pairs to reduce false positive term pairs. we focus on how to construct mini-batches to learn fine-grained term representations. ## conclusions we propose coder++, a fine-grained biomedical term representation, which benefits from our dynamic sampling strategy that provides hard positive and negative pairs. we propose an automatic largescale clustering evaluation algorithm. through a combination of automatic evaluation and the case study, we find coder++ greatly outperforms coder and sapbert on umls term clustering and has a much better ability to distinguish different concepts with similar term names. the effectiveness of our dynamic sampling strategy is also proved through an ablation study. our work can be used for automatic term clustering or recommend candidate similar terms for experts and crowdsourcing participants in human term clustering. our work also suggests that biomedical term embedding models such as coder can be further pretrained by focusing on specific information.