Issue: Semantic Similarity Score
#55
by
linhphanff
- opened
Firstly, I want to thank you for the great job you guys have done with the extremely powerful model, I have an issue right now. When I calculate the similarity between two identical sentences using dense vectors, why is the score 0.9995 (should be 1)? Can you explain?
Sentence 1: "We recommend"
Sentence 2: "We recommend"
Thanks for your attention to our work!
If you use fp16, numerical overflow is likely to occur, which may cause slight differences. However, the error is very small and does not significantly affect the ranking results.
If you load model with transformers package, you need to set model.eval(), otherwise the dropout operation will cause two different embeddings for the same sentence.
Thanks for your answer !!