Spaces:

evaluate-metric

/

f1

like 8

Runtime error

Check here for related issue: https://github.com/huggingface/evaluate/issues/234

This works:

metric=evaluate.load('f1')
metric.compute(references=[0, 1, 0, 1, 0], predictions=[0, 0, 1, 1, 0], average=None)

This won't work:

metric=evaluate.combine(["f1"])
metric.compute(references=[0, 1, 0, 1, 0], predictions=[0, 0, 1, 1, 0], average=None)

Reason:
average is not included in f1 score features: https://github.com/huggingface/evaluate/blob/eaf34a7d04e7ab3e6155a046f6d7fda01d9ead84/metrics/f1/f1.py#L112

and CombinedEvaluations.compute ignore if argument is not included in features: https://github.com/huggingface/evaluate/blob/eaf34a7d04e7ab3e6155a046f6d7fda01d9ead84/src/evaluate/module.py#L858

Is this expected or a bug?