How to interpret the score?

by Jr92 - opened Mar 12

Jr92

Mar 12

Hi,

I am trying the rerank-xsmall model. I can see even for 2 same sentences the score is under 0.25.
So, I was wondering if I am interpreting it incorrectly.
I was trying "What a day. What a day"

Thanks

juliuslipp

mixedbread ai org Mar 12

•

edited Mar 12

Hey @Jr92 ,
our reranker is mostly designed to work like a search engine and in this iteration is not mean for STS tasks. This means you should design your query to be a question. E.g. "Who wrote 'To Kill a Mockingbird'?" + "'To Kill a Mockingbird' is a novel by Harper Lee published in 1960. It was immediately successful, winning the Pulitzer Prize, and has become a classic of modern American literature.". We're aware that this is a downfall and are working to fix this in V2.

Have a nice day!

juliuslipp changed discussion status to closed Mar 12

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment