nomic-ai

/

nomic-embed-text-v1.5

like 441

Follow

Nomic AI 217

How can I infer with different embedding dimension?

model_path = "./Nomic_Embed_V1.5/onnx/model_quantized.onnx"
tokenizer_path = "./Nomic_Embed_V1.5"
tokenizer = AutoTokenizer.from_pretrained(tokenizer_path)
session = ort.InferenceSession(model_path)
inputs = tokenizer(query_sentence, return_tensors="np")

onnx_inputs = {session.get_inputs()[i].name: inputs[input_name].astype(
np.int64) for i, input_name in enumerate(inputs.keys())}
outputs = session.run(None, onnx_inputs)
embeddings = outputs[0]

I don't believe the model was converted to Onnx with that ability. You will have to do that yourself unfortunately

Thanks. Already working on one

Let me know if you get something working!