CLTL-VUAmsterdam/multiClinNER26-xlmrb

Model Card for Model ID

A XLMRoberta-base model finetuned for the NER subtask of MultiClinAI 2026.

Model Details

Model Description

The model is finetuned for multiclass classification over the three labels (diseases, procedures, symptoms), using the training data for the seven languages in the task (Czech, English, Spanish, Dutch, Italian, Romanian and Swedish).

• Developed by: Sophie Arnoult
• Funded by: NWO
• Shared by: CLTL
• Model type: XLMRobertaForTokenClassification
• Language(s) (NLP): Czech, English, Spanish, Dutch, Italian, Romanian and Swedish
• License: MIT
• Finetuned from model: xlm-roberta-base

Model Sources

• Repository: https://github.com/cltl/MultiClinNER-2026
• Paper: LotusOrchid at #SMM4H–HeaRD 2026: Fitting pretrained encoders for Dutch medical data
• Task homepage: https://temu.bsc.es/MultiClinAI/

The model corresponds to system 'all_xlmrb' in the paper.

Bias, Risks, and Limitations

The model was only tested on workshop data, and it is unknown how well it will perform on other datasets.

Citation

@inproceedings{arnoult-etal-2026-lotusorchid, title = "{L}otus{O}rchid at {#}{SMM}4{H}{--}{H}ea{RD} 2026: Fitting pretrained encoders for {D}utch medical data", author = "Arnoult, Sophie and Chen, Shutao and Vossen, Piek", editor = "Lopez-Garcia, Guillermo and Gonzalez-Hernandez, Graciela", booktitle = "Proceedings of the 11th Social Media Mining for Health Research and Applications ({SMM}4{H}-{H}ea{RD} 2026) Workshop and Shared Tasks", month = jul, year = "2026", address = "San Diego, United States", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2026.smm4h-1.23/", doi = "10.18653/v1/2026.smm4h-1.23", pages = "139--145", ISBN = "979-8-89176-432-3", abstract = "This paper presents our submission to MultiClinAI{'}s NER subtask for {#}SMM4H-HeaRD 2026. We focus on the questions 1) which Language Model represents the clinical notes best and 2) which annotations can help training these models. To get answers for these questions, we follow a token-based classification approach with pretrained encoder language models, where we compare models that were pretrained on generic data against medical data, and on a single language, Dutch, against many languages. In addition, we present two data-augmented systems: one with data from the other languages of the workshop for multilingual training, and one with synthetic annotations." }

~
~