Before revision:  We propose SapBERT, a pre-training scheme based on BERT. It self-aligns the representation space of biomedical entities with a metric learning objective function leveraging UMLS, a  collection of biomedical ontologies with > 4M concepts.
After revision:  We propose SapBERT, a pretraining scheme that self-aligns the representation space of biomedical entities with a metric learning objective function leveraging UMLS, a collection of biomedical ontologies with > 4M concepts.