QQs about score and class beyond “candidate labels”
When using inference API with multiple labels, how should I interpret scores from this model? I would appreciate any pointers to underlying scoring function.
When scores for all classes are low, is it safe to assume the input is about some other class? Any minimum score threshold could be used as cut-off for this purpose?
Hey @pradhuman-fiddler ,
Re 1: do you mean with multi_label=True ? You can find more details here: https://discuss.huggingface.co/t/new-pipeline-for-zero-shot-text-classification/681/3?u=moritzlaurer
Re 2: Yes, when you use multi_label=True and all values are very low, the model thinks that the text does not belong to any of the classes you provided. Regarding thresholds: there is not golden answer for this and it strongly depends on your use case. I'd recommend testing a few edge cases and determining a threshold for your use case like that.
This is very helpful. Thanks!