what score is a good score

by narenzen - opened

what score is a good score?

Here are few outputs from the model:

Screenshot from 2023-05-11 20-56-27.png
Output from model:
{'score': 0.5341075658798218, 'start': 11, 'end': 15, 'answer': 'john'}
Even for this easy question, models score is 0.5

Screenshot from 2023-05-11 20-58-33.png
Output from model:
{'score': 0.03491657227277756, 'start': 6, 'end': 15, 'answer': 'bar chart'}

Why the scores are very low, even though the model's output is correct?
What could be the threshold to reject false outputs?

Sign up or log in to comment