scheun/biobert-ade-classifier

BioBERT for Adverse Drug Effect (ADE) Classification

This model is a fine-tuned version of dmis-lab/biobert-base-cased-v1.2 for binary sentence classification: Does a sentence describe an adverse drug effect (ADE)? It was fine-tuned on the ADE Corpus V2 dataset and compared against a classical TF-IDF + Logistic Regression baseline as part of a broader project benchmarking classical vs. transformer approaches on imbalanced biomedical text.

Project Repo: GitHub

Results (Test Set: N=3,528)

Model	Weighted F1	ADE Class F1	Accuracy	Total Errors
TF-IDF + Logistic Regression	0.90	0.84	90%	349
BioBERT (this model)	0.96	0.93	96%	145

BioBERT reduced misclassifications by 58% (349 → 145 errors) compared to the classical baseline.

Training Details

• Base model: dmis-lab/biobert-base-cased-v1.2 (110M parameters)
• Epochs: 3 (Best checkpoint selected by validation F1)
• Learning rate: 2e-5
• Batch size: 16
• Max sequence length: 128
• Precision: fp16
• Data split: stratified 70/15/15 train/val/test (seed=42)

Epoch	Train Loss	Val F1	Val Accuracy
1	0.175	0.943	0.943
2	0.114	0.952	0.952
3	0.043	0.952	0.952

Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained("scheun/biobert-ade-classifier")
tokenizer = AutoTokenizer.from_pretrained("scheun/biobert-ade-classifier")

inputs = tokenizer("Patient developed severe nausea after taking the medication.", return_tensors="pt")
outputs = model(**inputs)
prediction = outputs.logits.argmax(-1).item()
print(prediction)  # 0 = not ADE, 1 = ADE

Limitations

• Trained on MEDLINE case report sentences. Performance may vary on other text domains.
• Binary classification only. It does not extract which drug or which effect is mentioned.

References

• Gurulingappa et al. (2012), Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports
• Lee et al. (2020), BioBERT: a pre-trained biomedical language representation model for biomedical text mining