How to use ModernBERT as a sentence transformer?
How to do the equivalent of this with ModernBERT ? I know its not a sentence transformer and AutoModel, BERTModel doesnt recognize this correctly.
sentences_1 = 'How is the weather today?'
sentences_2 = 'What is the current weather like today?'
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('sentence-transformers/msmarco-MiniLM-L-12-v3',trust_remote_code=True)
embeddings_1 = model.encode(sentences_1, normalize_embeddings=True)
embeddings_2 = model.encode(sentences_2, normalize_embeddings=True)
print(cos_sim(embeddings_1, embeddings_2))
The base model has not been fine-tuned for retrieval tasks off-the-shelf.
I have a sentence-transformer finetuned version based on the offical finetuning script with a larger batch size. It's working better than the numbers reported in the paper. Please give it a try and share your experience!
@hungrybiker
https://huggingface.co/blog/train-sentence-transformers may be able to help :) Do note that you need to install transformers from main using
pip install --upgrade git+https://github.com/huggingface/transformers.git