Manjot PRO

morsczx

AI & ML interests

None yet

Recent Activity

reacted to MoritzLaurer's post with 👍 21 days ago

Quite excited by the ModernBERT release! 0.15/0.4B small, 2T modern pre-training data and tokenizer with code, 8k context window, great efficient model for embeddings & classification! This will probably be the basis for many future SOTA encoders! And I can finally stop using DeBERTav3 from 2021 :D Congrats @answerdotai, @LightOnIO and collaborators like @tomaarsen ! Paper and models here 👇https://huggingface.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb

View all activity

Organizations

morsczx's activity

reacted to MoritzLaurer's post with 👍 21 days ago

Post

2597

Quite excited by the ModernBERT release! 0.15/0.4B small, 2T modern pre-training data and tokenizer with code, 8k context window, great efficient model for embeddings & classification!

This will probably be the basis for many future SOTA encoders! And I can finally stop using DeBERTav3 from 2021 :D

Congrats @answerdotai , @LightOnIO and collaborators like @tomaarsen !

Paper and models here 👇https://huggingface.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb