metadata
license: mit
base_model: deepset/gbert-base
GBERT-BioM-Translation-base
This model is a medically continuously pre-trained version of deepset/gbert-base.
Training data
The model was trained on German PubMed abstracts, translated English PubMed abstracts, and translated MIMIC-III reports.
| Dataset | Tokens | Documents |
|---|---|---|
| German PubMed | 5M | 16K |
| PubMed | 1,700M | 21M |
| MIMIC-III | 695M | 24M |
| Total | 2,400M | 45M |
Evaluation
| Model | CLEF eHealth 2019 | RadQA | GraSCCo | BRONCO150 | GGPONC 2.0 | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| F1 | P | R | F1 | EM | F1 | P | R | F1 | P | R | F1 | P | R | |
| GBERT-base | .816 | .818 | .815 | .794 | .707 | .642 | .617 | .676 | .833 | .818 | .849 | .770 | .761 | .780 |
| GBERT-large | .832 | .802 | .865 | .809 | .718 | .647 | .617 | .680 | .835 | .820 | .852 | .772 | .758 | .786 |
| GBERT-BioM-Translation-base | .825 | .851 | .801 | .808 | .716 | .661 | .642 | .681 | .842 | .824 | .861 | .780 | .766 | .794 |
| GBERT-BioM-Translation-large | .833 | .860 | .807 | .811 | .714 | .692 | .677 | .707 | .844 | .825 | .864 | .786 | .779 | .793 |