Commit
•
f21d357
1
Parent(s):
c9972ce
Update README.md
Browse files
README.md
CHANGED
|
@@ -3,7 +3,7 @@ license: cc-by-nc-nd-4.0
|
|
| 3 |
---
|
| 4 |
# SurgicBERTa
|
| 5 |
SurgicBERTa is a language model based on RoBERTa-base (Liu et al., 2019) architecture.
|
| 6 |
-
We
|
| 7 |
We used the full text of the books and papers in training, not just abstracts.
|
| 8 |
Specific details of the adaptive pretraining procedure and evaluation tasks can be found in the paper below cited.
|
| 9 |
|
| 3 |
---
|
| 4 |
# SurgicBERTa
|
| 5 |
SurgicBERTa is a language model based on RoBERTa-base (Liu et al., 2019) architecture.
|
| 6 |
+
We adapted RoBERTa-base to different **surgical textbooks and academic papers** via continued pretraining. This amount to about 7 million words and 300k surgical sentences.
|
| 7 |
We used the full text of the books and papers in training, not just abstracts.
|
| 8 |
Specific details of the adaptive pretraining procedure and evaluation tasks can be found in the paper below cited.
|
| 9 |
|