Spaces:
Running
on
CPU Upgrade
cannot load results
datasets.load_dataset("open-llm-leaderboard/details_tiiuae__falcon-7b", name="results", split="latest",)
datasets.exceptions.DatasetGenerationError: An error occurred while generating the dataset
"because column names don't match"
regarding the new leaderboard, it seems even llama3 isn't loading (also tried removing the "-private" suffix)
datasets.load_dataset("HuggingFaceEvalInternal/meta-llama__Meta-Llama-3-70B-Instruct-details-private", name="results", split="latest",)
FileNotFoundError: Couldn't find a dataset script at ../HuggingFaceEvalInternal/meta-llama__Meta-Llama-3-70B-Instruct-details/meta-llama__Meta-Llama-3-70B-Instruct-details.py or any data file in the same directory. Couldn't find 'HuggingFaceEvalInternal/meta-llama__Meta-Llama-3-70B-Instruct-details' on the Hugging Face Hub either: FileNotFoundError: Dataset 'HuggingFaceEvalInternal/meta-llama__Meta-Llama-3-70B-Instruct-details' doesn't exist on the Hub. If the repo is private or gated, make sure to log in with huggingface-cli login.
Hi @ysharma1126 ,
You can download the open-llm-leaderboard/tiiuae__falcon-7b-details dataset like this:
from datasets import load_dataset
falcon_ds = load_dataset("open-llm-leaderboard/tiiuae__falcon-7b-details",
name="tiiuae__falcon-7b__leaderboard_arc_challenge",
split="latest",
use_auth_token = "your_token",
streaming=True,
)
And check the samples inside:
for sample in falcon_ds.take(5):
print(sample)
Note, that this dataset is gated, so you need to request the access. Plus, here I check one of the tasks – "arc_challenge". You can check other tasks by passing them into name parameter.
Considering Meta-Llama-3-70B-Instruct, we use HuggingFaceEvalInternal organisation as our private one, so you won't have access to it. You can check the details for open-llm-leaderboard/meta-llama__Meta-Llama-3-70B-Instruct-details the same way as for falcon-7b
I close this discussion, please, write here if you have any other questions