Spaces:

openlifescienceai

/

open_medical_llm_leaderboard

like 293

Running on CPU Upgrade

Hi @tosaddler , thanks for reporting! The leaderboard seems to be running now, or were you referring to a specific component that is not working?

@aryopg It was giving an error in the morning, fixed it.

I think,the CPU is running 0%

We submitted a few models a days ago and they are still listed as "Running Evaluation." There are 11 models that have been under evaluation for a week now :)

Thanks for letting us know! We just restarted all of the evaluations, hopefully everything will go smoothly this time!

do you see your models on the leaderboard, @wassemgtk ? If not, I'll investigate again!

Nope - should be under Writer/Palmyra-Med-70b name. Thanks for checking!!

Hi @wassemgtk , i think it's due to the limitation of GPU that we have. I just noticed that you requested the evaluation on float32, would you mind re-submitting the evaluation in bfloat16? It shouldn't affect the performance that much

Noted - i resubmit the model with 70b size. Also, I am happy to donate or help with the GPU capacity if you require it.

@wassemgtk That would be a great help, can you send an email to openlifescience [at] gmail

Unfortunately, we still can't see the model. Could you please check?

@aryopg anything we can do to help?