Nicolas-BZRD

Nicolas-BZRD

AI & ML interests

PhD Student | NLP - LLMs - Adaptation real-world problem Optimization

Recent Activity

Organizations

CroissantLLM's profile picture UTTER - Unified Transcription and Translation for Extended Reality's profile picture Diabolocom's profile picture EuroBERT's profile picture

Nicolas-BZRD's activity

New activity in EuroBERT/EuroBERT-210m about 6 hours ago
New activity in nickprock/multi-sentence-BERTino 1 day ago

EuroBERT

#3 opened 1 day ago by
Nicolas-BZRD

Discussion

4
#2 opened 2 days ago by
Nicolas-BZRD
view reply

Hey @CorentinAmbroise , we are currently working on the modeling file to add the different tasks required to execute the MTEB benchmark. We hope to achieve it soon.

view reply

We are working on the next model, which covers all European languages. Training the previous model with a restricted number of languages helped us better understand the impact of their distribution during training and the curse of multilinguality while maximizing population coverage.

We also released the code base and look forward to see the community adding more languages 🤗

view reply

ModernBERT is English-only. We achieve similar performance in English with our small model (which is slightly larger than ModernBERT) and better performance with our medium and large models. For multilingual tasks, we obtain superior results. However, since comparing ModernBERT on multilingual data is less meaningful, we chose not to report those results. For math and code, the comparison is more relevant, so we included it. However, you are right—we will add the results in the appendix.