Spaces:
Running
on
CPU Upgrade
Three failed evaluations
Hello,
It seems my recent 3 submitted models failed evaluations. But I had all of them loaded locally with AutoModel and even had a nice chat with one of them.
Can you take a look at what at what happened? These are the requests:
Pretergeek/OpenChat-3.5-0106_8.11B_36Layers-Interleaved_eval_request_False_bfloat16_Original.json
Pretergeek/OpenChat-3.5-0106_8.99B_40Layers-Interleaved_eval_request_False_bfloat16_Original.json
Pretergeek/OpenChat-3.5-0106_10.7B_48Layers-Interleaved_eval_request_False_bfloat16_Original.json
Best Regards,
@Pretergeek
Hi @Pretergeek ,
Could you please provide links to the request files for these three models according to our F.A.Q.? That way we will check these models faster
Hi,
Of course, my bad, I sent only partial references. Here are proper links:
https://huggingface.co/datasets/open-llm-leaderboard/requests/resolve/main/Pretergeek/OpenChat-3.5-0106_8.11B_36Layers-Interleaved_eval_request_False_bfloat16_Original.json
https://huggingface.co/datasets/open-llm-leaderboard/requests/resolve/main/Pretergeek/OpenChat-3.5-0106_8.99B_40Layers-Interleaved_eval_request_False_bfloat16_Original.json
https://huggingface.co/datasets/open-llm-leaderboard/requests/resolve/main/Pretergeek/OpenChat-3.5-0106_10.7B_48Layers-Interleaved_eval_request_False_bfloat16_Original.json
Thank you.
Hi @Pretergeek ,
I've resubmitted these three models. I will keep this discussion open for a while to check how the evaluation is going
Thank you @alozowski , I appreciate you keeping an eye on it.
I see that the first model failed and the problem is on our side, I'll fix it on Monday 🤝
Have a pleasant weekend.
UPD: I've resubmitted your models again, seems like it was a network issue, but should be fine now
I close this issue, please, feel free to ping me here in case of any questions about these models or open a new discussion