🚀 Models ⚡️ Inference API
🔥 Sign in
Back to all models
Model: smanjil/German-MedBERT
pytorch bert masked-lm de exbert German
Model card Files and versions Use in transformers

How to use this model directly from the 🤗/transformers library: 

					
Copy to clipboard

					from transformers import AutoTokenizer, AutoModelForMaskedLM

					tokenizer = AutoTokenizer.from_pretrained("smanjil/German-MedBERT")
					
					model = AutoModelForMaskedLM.from_pretrained("smanjil/German-MedBERT")
See raw config file

How to clone the model repo 

					
Copy to clipboard

					git lfs install
					git clone https://huggingface.co/smanjil/German-MedBERT
Done
fill-mask mask_token: [MASK]
Query this model
🔥 This model is currently loaded and running on the Inference API. ⚠️ This model could not be loaded by the inference API. ⚠️ This model can be loaded on the Inference API on-demand.
JSON Output
API endpoint  

⚡️ Upgrade your account to access the Inference API

Share Copied link to clipboard

Contributed by

smanjil Manjil Shrestha
1 model
Update on GitHub

German Medical BERT

This is a fine-tuned model on Medical domain for German language and based on German BERT. This model has only been trained to improve on target task (Masked Language Model). It can later be used to perform a downstream task of your needs, while I performed it for NTS-ICD-10 text classification task.

Overview

Language model: bert-base-german-cased

Language: German

Fine-tuning: Medical articles (diseases, symptoms, therapies, etc..)

Eval data: NTS-ICD-10 dataset (Classification)

Infrastructure: Gogle Colab

Details

  • We fine-tuned using Pytorch with Huggingface library on Colab GPU.
  • With standard parameter settings for fine-tuning as mentioned in original BERT's paper.
  • Although had to train for upto 25 epochs for classification.

Performance (Micro precision, recall and f1 score for multilabel code classification)

Models P R F1
German BERT 86.04 75.82 80.60
German MedBERT-256 87.41 77.97 82.42
German MedBERT-512 87.75 78.26 82.73

Author

Manjil Shrestha: shresthamanjil21 [at] gmail.com

Get in touch: LinkedIn