FremyCompany
commited on
Commit
•
05b79c0
1
Parent(s):
74fef1b
Update README.md
Browse files
README.md
CHANGED
@@ -16,9 +16,11 @@ widget:
|
|
16 |
---
|
17 |
|
18 |
# FremyCompany/BioLORD-STAMB2-v1
|
19 |
-
This model was trained using BioLORD, a new pre-training strategy for producing
|
20 |
|
21 |
-
|
|
|
|
|
22 |
|
23 |
This model is based on [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) and was further finetuned on the [BioLORD-Dataset](https://huggingface.co/datasets/FremyCompany/BioLORD-Dataset).
|
24 |
<img width="640" src="https://s3.amazonaws.com/moonup/production/uploads/1665568401241-5f04e8865d08220171a0ad3f.png" />
|
|
|
16 |
---
|
17 |
|
18 |
# FremyCompany/BioLORD-STAMB2-v1
|
19 |
+
This model was trained using BioLORD, a new pre-training strategy for producing meaningful representations for clinical sentences and biomedical concepts.
|
20 |
|
21 |
+
State-of-the-art methodologies operate by maximizing the similarity in representation of names referring to the same concept, and preventing collapse through contrastive learning. However, because biomedical names are not always self-explanatory, it sometimes results in non-semantic representations.
|
22 |
+
|
23 |
+
BioLORD overcomes this issue by grounding its concept representations using definitions, as well as short descriptions derived from a multi-relational knowledge graph consisting of biomedical ontologies. Thanks to this grounding, our model produces more semantic concept representations that match more closely the hierarchical structure of ontologies. BioLORD establishes a new state of the art for text similarity on both clinical sentences (MedSTS) and biomedical concepts (MayoSRS).
|
24 |
|
25 |
This model is based on [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) and was further finetuned on the [BioLORD-Dataset](https://huggingface.co/datasets/FremyCompany/BioLORD-Dataset).
|
26 |
<img width="640" src="https://s3.amazonaws.com/moonup/production/uploads/1665568401241-5f04e8865d08220171a0ad3f.png" />
|