Hugging Face's logo

McGill-NLP
/
LLM2Vec-Mistral-7B-Instruct-v2-mntp-unsup-simcse

Sentence Similarity
PEFT
Safetensors
English
text-embedding
embeddings
information-retrieval
beir
text-classification
language-model
text-clustering
text-semantic-similarity
text-evaluation
text-reranking
feature-extraction
Sentence Similarity
natural_questions
ms_marco
fever
hotpot_qa
mteb
Eval Results
Model card Files Community
3

Mac different results

#2
by l0d0v1c - opened
Discussion
l0d0v1c

I tried with mac M2 (using https://download.pytorch.org/whl/nightly/cpu becaus bfloat16 is not supported on MPS). It works but I get different results:

tensor([[0.5709, 0.1221],
[0.0952, 0.3745]])

insead of

tensor([[0.6175, 0.2535],
[0.2298, 0.5792]])

in your example.
Anyway very interesting work! Thank you.

vaibhavad
 McGill NLP Group org

Hi @l0d0v1c ,

Apologies for the late response. Unfortunately, we haven't extensively tested our models beyond NVIDIA GPUs using BF16, so the behaviour might be unpredictable. However, using float32 should give you similar results on M2 and NVIDIA.

Sign up or log in to comment