Back to all models
Model card Files and versions Use in transformers
fill-mask mask_token: [MASK]
Query this model
πŸ”₯ This model is currently loaded and running on the Inference API. ⚠️ This model could not be loaded by the inference API. ⚠️ This model can be loaded on the Inference API on-demand.
JSON Output
API endpoint  

⚑️ Upgrade your account to access the Inference API

Share Copied link to clipboard

Contributed by

smanjil Manjil Shrestha
1 model

German Medical BERT

This is a fine-tuned model on Medical domain for German language and based on German BERT. This model has only been trained to improve on target task (Masked Language Model). It can later be used to perform a downstream task of your needs, while I performed it for NTS-ICD-10 text classification task.

Overview

Language model: bert-base-german-cased

Language: German

Fine-tuning: Medical articles (diseases, symptoms, therapies, etc..)

Eval data: NTS-ICD-10 dataset (Classification)

Infrastructure: Gogle Colab

Details

  • We fine-tuned using Pytorch with Huggingface library on Colab GPU.
  • With standard parameter settings for fine-tuning as mentioned in original BERT's paper.
  • Although had to train for upto 25 epochs for classification.

Performance (Micro precision, recall and f1 score for multilabel code classification)

Models P R F1
German BERT 86.04 75.82 80.60
German MedBERT-256 87.41 77.97 82.42
German MedBERT-512 87.75 78.26 82.73

Author

Manjil Shrestha: shresthamanjil21 [at] gmail.com

Get in touch: LinkedIn