open-llm-leaderboard/open_llm_leaderboard

Sao10K

Dec 21, 2023

Hi again, it's me. I'm wondering why these two failed.

Wintergoddess was in the queue with multiple other models for a few days, but when I checked this morning it failed?

Same with Frostwind, though I just added that like yesterday.

Wintergoddess is a Llama2 70B model.

Frostwind is a Solar 10.7B Finetune.

Below are the files: I was able to run the eval harness locally on runpod.

https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/Sao10K/WinterGoddess-1.4x-70B-L2_eval_request_False_float16_Original.json

https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/Sao10K/Frostwind-10.7B-v1_eval_request_False_bfloat16_Original.json

clefourrier

Open LLM Leaderboard org Dec 21, 2023

Hi! The first one was cancelled and not relaunched while we were changing clusters, the second was blocked by a connection problem - I passed both back to pending

clefourrier changed discussion status to closed Dec 21, 2023

Spaces:

open-llm-leaderboard
/

open_llm_leaderboard

Running on CPU Upgrade

Model Eval Failure