Spaces:

open-llm-leaderboard

/

open_llm_leaderboard

like 11.4k

Running on CPU Upgrade

Hi!
Thank you for your issue! Yes, we need to make the checking system a bit better - I can remove your results and requests to allow you to resubmit, would that work for you?

yes, that works for me, thanks!

Done! Please retry submitting in about an hour :)

thanks, I tried to re-submit the model (under a new name: HWERI/Llama2-7b-sharegpt4), but keeps getting "Something went wrong Connection errored out." Is there something that I am doing incorrectly? Attached is a screenshot.

Hm, this is highly surprising but it might be a temporary network problem. I'm going to reboot the space, please try again once it's up!

Hmm, very strange, I still get the same error.

My teammate is creating a new model repo with the same files, named HWERI/Llama2-7b-sharegpt4-v2, and will try it again.

I just filled the form using the above template and it worked just fine, I have no idea what went wrong.

You rmodel should appear in the pending queue in less than an hour, can you close this issue once it's good?

Maybe it’s related to browser cache. Thanks for submitting it for us! I’ll keep you posted

The issue has been very strange, I saw the model HWERI/Llama2-7b-sharegpt4 was in the pending list, as shown in the screenshot, but now it vanished, and is nowhere to be found in either pending, running or finished list. How could this be?

Hi! If a model disappears from all queues, it's likely it failed, see the FAQ

Thanks, just found the failed requests!

I am able to run the harness evaluation on these model files locally, and couldn't understand why it failed by submission. Is there more detailed information on why the it has failed?

This is one of the logs:
{
"model": "beaugogh/Llama2-7b-sharegpt4-resubmit",
"base_model": "",
"revision": "main",
"private": false,
"precision": "float16",
"weight_type": "Original",
"status": "FAILED",
"submitted_time": "2023-08-13T12:30:26Z",
"model_type": "🔶 : fine-tuned",
"job_id": "364137"
}

@clefourrier may I get an update on this issue?

Hi @beaugogh
We had a connectivity issue this weekend, where a lot of model results where not uploaded to the dataset, and appeared as failed. It has been fixed now.

@clefourrier thanks for the reply. Can we remove the failure records so that I can resubmit the models? Currently it errored on “this model has been submitted ”.

Hi! I think your model has results, so there is no need to resubmit it, unless you are talking about another model?

Sorry, yes, I see the models have been online! Thanks!