Spaces:

ought

/

raft-leaderboard

like 45

Running

Hey @JeanneRbs , thanks for reporting this! I'll take a look at the logs and report back soon

Hey @JeanneRbs it seems that there's a problem loading your submission dataset. From the logs we see the following error:

submission_ds = load_dataset(path=submission_dataset, name=task, use_auth_token=use_auth_token, split="test")
File "/app/env/lib/python3.8/site-packages/datasets/load.py", line 1679, in load_dataset
Json files may be downloaded completely. Also streaming from remote zip or gzip files is supported but other compressed formats
File "/app/env/lib/python3.8/site-packages/datasets/builder.py", line 704, in download_and_prepare
raise OSError(
File "/app/env/lib/python3.8/site-packages/datasets/builder.py", line 1221, in _download_and_prepare
Example::
File "/app/env/lib/python3.8/site-packages/datasets/builder.py", line 793, in _download_and_prepare
if isinstance(datasets, dict):
File "/app/env/lib/python3.8/site-packages/datasets/builder.py", line 1212, in _prepare_split
in a Beam pipeline. It is called once for each `SplitGenerator` defined in
File "/app/env/lib/python3.8/site-packages/datasets/features/features.py", line 1579, in encode_example
File "/app/env/lib/python3.8/site-packages/datasets/features/features.py", line 1148, in encode_nested_example
return Sequence(feature=generate_from_arrow_type(pa_type.value_type), length=pa_type.list_size)
File "/app/env/lib/python3.8/site-packages/datasets/features/features.py", line 1149, in <dictcomp>
elif isinstance(pa_type, pa.ListType):
File "/app/env/lib/python3.8/site-packages/datasets/features/features.py", line 1203, in encode_nested_example
feature (FeatureType): the feature type to be checked
File "/app/env/lib/python3.8/site-packages/datasets/features/features.py", line 995, in encode_example
return schema()
File "/app/env/lib/python3.8/site-packages/datasets/features/features.py", line 934, in str2int
Array2D,
File "/app/env/lib/python3.8/site-packages/datasets/features/features.py", line 934, in <listcomp>
Array2D,
File "/app/env/lib/python3.8/site-packages/datasets/features/features.py", line 955, in _strval2int
schema = schema[0]
ValueError: Invalid string class label

Could you please check that you can load your submission dataset for all the RAFT tasks? I'm a bit surprised this passed the validation tests, but maybe something has changed recently on the datasets side. Apologies for the inconvenience!

Thank you for your reply, everything is good now !