Spaces:

ought

/

raft-leaderboard

like 45

Running

Hi @OE-Heart ! Thanks for raising the issue - there's been some changes in our backend that affected the evaluation pipeline. I'll post a fix this week and report back here when it's done. Apologies for the inconvenience!

Hi @OE-Heart , we're just deployed the fix for the evaluation pipeline and your submission has now been evaluated. Apologies for the delay!

Hi @lewtun ! Thanks for the fix and I have another submission on Jun 11, will it be evaluated on Jun 12 as expected?

Hi @OE-Heart , looking at the logs, it seems there's a problem with one of the classes in the banking77 subset, specifically about the reverted_card_payment? class. Could you please check that your submission passes the validation test with:

python cli.py validate

Thanks!

@lewtun Thanks for the reminder, you are right. I made a little mistake on the label. May I have another evaluation please? It's important for me. Appreciate.

Just in case someone else encounters this problem in the future, pay attention to one particular label name is reverted_card_payment? but not reverted_card_payment. The question mark at the end is not an error.

@lewtun Thanks for the reminder, you are right. I made a little mistake on the label. May I have another evaluation please? It's important for me. Appreciate.

Sure! I'll run the evaluation pipeline ASAP

Hey @OE-Heart , I've run the evaluation pipeline and there seems to be a problem with the ID column in one of your datasets. Could you confirm whether the validation script passes for your submission?

python cli.py validate

Thanks for checking. Last question: can you also make a new submission by running

python cli.py submit

Hey @OE-Heart , I've run the evaluation pipeline and there seems to be a problem with the ID column in one of your datasets. Could you confirm whether the validation script passes for your submission?

python cli.py validate

I just checked again and it looks ok for me.

Thanks for checking. Last question: can you also make a new submission by running

python cli.py submit

I just submitted.

Hi, @lewtun .Is there anything else I can do, please? Sorry for the trouble.

Looks like it worked! Great job on beating the human baseline :D