Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
some models get close to zero on some topics
#7
by
gblazex
- opened
especially finance and legal.
Is this normal? Doesn't look right to me
Hey! Thanks for flagging this. We did notice this issue before and we will be updating the financebench and legalbench evaluations over the weekend for all the models on the leaderboard. We realized the max_new_tokens was too short of the legal confidentiality task for a few models.