Spaces:
Running
on
CPU Upgrade
Can not use API to access leaderboard
I am trying to access the leaderboard using the API specified at the bottom of the leaderboard page. However, the request always runs into errors.
My code to access the leaderboard is:
from gradio_client import Client
client = Client("https://huggingfaceh4-open-llm-leaderboard.hf.space/")
result = client.submit(
"./empty.json", # str (filepath to JSON file)
#in 'parameter_23' Dataframe component
["Average ⬆️"], # List[str] in 'Select columns to show' Checkboxgroup component
["🟢 pretrained"], # List[str] in 'Model types' Checkboxgroup component
["torch.float16"], # List[str] in 'Precision' Checkboxgroup component
["?"], # List[str] in 'Model sizes (in billions of parameters)' Checkboxgroup component
True, # bool in 'Show gated/private/deleted models' Checkbox component
"", # str in 'parameter_8' Textbox component
fn_index=0
)
print(result)
Hi! Could you please elaborate on the error messages you get? :)
Hi! Could you please elaborate on the error messages you get? :)
Yes. I followed the code template that is provided by clicking the "Use via API" botton at the bottom of the main leaderboard page (https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
I substituted the first parameter with the path to an empty json file in the same folder of the code file. After running the code, I got the following error message:
It would be great if you can provide some explanations for the usage of the first and last parameter (# str (filepath to JSON file) and fn_index). I couldn't find any in the web.
Hi!
Some context: the API comes natively with Gradio Spaces, and the leaderboard team is not providing support for it since it's not the recommended way to use the leaderboard (unless the issue is very fast to fix for us) .
In your case, I suspect that the filepath to the JSON file is a file path to a request file (see the requests dataset), but I'm unsure. No idea about the last parameter.
However, you could go see what other users did in the Tools section, as some tools created by the community rely on the API. Community members such as @felixz also used it in the past, so maybe they could give you a hand if they want/have the time to?
Good luck with this!