|
--- |
|
license: cc-by-nc-nd-4.0 |
|
--- |
|
# SurgicBERTa |
|
SurgicBERTa is a language model based on RoBERTa-base (Liu et al., 2019) architecture. |
|
We adapted RoBERTa-base to different **surgical textbooks and academic papers** via continued pretraining. This amount to about 7 million words and 300k surgical sentences. |
|
We used the full text of the books and papers in training, not just abstracts. |
|
Specific details of the adaptive pretraining procedure and evaluation tasks can be found in the paper below cited. |
|
|
|
# Citation |
|
If using this model, please cite the following paper: |
|
<em> |
|
<br /> |
|
<br /> |
|
@article{bombieri_et_al_surgical_srl_2022, <br /> |
|
title = {Machine understanding surgical actions from intervention procedure textbooks},<br /> |
|
journal = {Computers in Biology and Medicine},<br /> |
|
pages = {106415},<br /> |
|
year = {2022},<br /> |
|
issn = {0010-4825},<br /> |
|
doi = {https://doi.org/10.1016/j.compbiomed.2022.106415},<br /> |
|
url = {https://www.sciencedirect.com/science/article/pii/S0010482522011234},<br /> |
|
author = {Marco Bombieri and Marco Rospocher and Simone Paolo Ponzetto and Paolo Fiorini},<br /> |
|
keywords = {Semantic role labeling, Surgical data science, Procedural knowledge, Information extraction, Natural language processing}<br /> |
|
} |
|
</em> |
