nomic-ai/nomic-embed-text-v1.5 · Unknown task sentence-similarity

May 21

Hello,

I am trying to use transformers library to run this model, I am using the code snippet provided:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("sentence-similarity", model="nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)

But I got the following error:

Unknown task sentence-similarity

zpn

Nomic AI org May 21

I will need to do a little more digging but it seems that you're right there is not sentence-similarity pipeline and it's suggested to use SentenceTransformers instead: https://github.com/huggingface/transformers/issues/22923#issuecomment-1517996334

let me know if you still run into issues running this code

import torch.nn.functional as F
from sentence_transformers import SentenceTransformer

matryoshka_dim = 512

model = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)
sentences = ['search_query: What is TSNE?', 'search_query: Who is Laurens van der Maaten?']
embeddings = model.encode(sentences, convert_to_tensor=True)
embeddings = F.layer_norm(embeddings, normalized_shape=(embeddings.shape[1],))
embeddings = embeddings[:, :matryoshka_dim]
embeddings = F.normalize(embeddings, p=2, dim=1)
print(embeddings)

BenTouss

May 21

That's working!
Thanks!
Maybe there is a way to update the snippet when you select "Use this model" in the model page..
I was not aware of the package, sentenceTransformers..

Thank you!

BenTouss changed discussion status to closed May 21