surgicberta /
marcobombieri's picture
license: cc-by-nc-nd-4.0
  - text: hemithyroidectomy is the removal of half of the <mask> gland.
    example_title: Example 1
  - text: >-
      Li-Fraumeni <mask> is a hereditary tumor with autosomal dominant
    example_title: Example 2
  - text: >-
      The fascia in front of the pancreas was cut to the spleen direction to
      <mask> the splenic artery and vein.
    example_title: Example 3


SurgicBERTa is a language model based on RoBERTa-base (Liu et al., 2019) architecture. We adapted RoBERTa-base to different surgical textbooks and academic papers via continued pretraining. This amount to about 7 million words and 300k surgical sentences. We used the full text of the books and papers in training, not just abstracts. Specific details of the adaptive pretraining procedure and evaluation tasks can be found in the paper below cited.


If using this model, please cite the following paper:

title = {Machine understanding surgical actions from intervention procedure textbooks},
journal = {Computers in Biology and Medicine},
pages = {106415},
year = {2022},
issn = {0010-4825},
doi = {},
url = {},
author = {Marco Bombieri and Marco Rospocher and Simone Paolo Ponzetto and Paolo Fiorini},
keywords = {Semantic role labeling, Surgical data science, Procedural knowledge, Information extraction, Natural language processing}