{{cookiecutter.repo_name}}/README.md

metadata

tags:
  - benchmark:superb
  - type:model
  - submission_name:none

SUPERB submissions for test-submission-with-weights

Submitting to the leaderboard

To make a submission to the leaderboard), there are three main steps:

1. Pretrain a model and store the weights in this repository as a model.pt file
2. Validate the model is compatible with the fine-tuning and evaluation framework
3. Push the model to the Hub!

See the instructions below for more details.

Rules

1. To prevent overfitting to the public leaderboard, we only evaluate one submission per week. You can push upstream models to the Hub as many times as you wish, but we will only evaluate the most recent commit in a given week.

Submission file format

TODO

Validate your submission

To ensure that your submission files are correctly formatted, run the following command from the root of the repository:

python cli.py validate

If everything is correct, you should see the following message:

All submission files validated! ✨ 🚀 ✨
Now you can make a submission 🤗

Push your submission to the Hugging Face Hub!

The final step is to commit your files and push them to the Hub:

python cli.py submit

If there are no errors, you should see the following message:

Submission successful! 🎉 🥳 🎉
Your submission will be fine-tuned and evaulated on Sunday 05 September 2021 ⏳

and your results will be visible on the leaderboard once the model is fine-tuned and evaluated.s