A multilingual BERGAMOT model with pre-trained on UMLS (version 2020AB) using a Graph Attention Network (GAT) encoder.
The model is described in paper "Biomedical Entity Representation with Graph-Augmented Multi-Objective Transformer " which is accepted to NAACL 2024!
For pretraining code see our github: https://github.com/Andoree/BERGAMOT.
Please do not forget to cite our paper when bib will be available.
- Downloads last month
- 13