---
license: mit
base_model: deepset/gbert-base
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# GBERT-BioM-Translation-base

This model is a medically continuously pre-trained version of [deepset/gbert-base](https://huggingface.co/deepset/gbert-base).

## Training data

The model was trained on German PubMed abstracts, translated English PubMed abstracts, and translated MIMIC-III reports.

| Dataset    | Tokens   | Documents |
|------------|----------|-----------|
| German PubMed | 5M      | 16K       |
| PubMed     | 1,700M   | 21M       |
| MIMIC-III  | 695M     | 24M       |
| **Total**  | **2,400M** | **45M**     |

## Evaluation

| Model                        | CLEF eHealth 2019 |      |      | RadQA |      | GraSCCo |      |      | BRONCO150 |      |      | GGPONC 2.0 |      |      |
|------------------------------|-------------------|------|------|-------|------|---------|------|------|-----------|------|------|------------|------|------|
|                              | F1                | P    | R    | F1    | EM   | F1      | P    | R    | F1        | P    | R    | F1         | P    | R    |
| [GBERT-base](https://huggingface.co/deepset/gbert-base)                   | .816              | .818 | .815 | .794  | .707 | .642    | .617 | .676 | .833      | .818 | .849 | .770       | .761 | .780 |
| [GBERT-large](https://huggingface.co/deepset/gbert-large)                  | .832              | .802 | .865 | .809  | .718 | .647    | .617 | .680 | .835      | .820 | .852 | .772       | .758 | .786 |
| **GBERT-BioM-Translation-base**  | .825              | .851 | .801 | .808  | .716 | .661    | .642 | .681 | .842      | .824 | .861 | .780       | .766 | .794 |
| GBERT-BioM-Translation-large | .833              | .860 | .807 | .811  | .714 | .692    | .677 | .707 | .844      | .825 | .864 | .786       | .779 | .793 |