What is this?

This repository is a demo leaderboard template. You can copy the leaderboard space and the two datasets (results and requests) to your org to get started with your own leaderboard!

The space does 3 things:

• stores users submissions, and sends them to the requests dataset
• reads the submissions depending on their status/date of creation.
• reads the results (results of running evaluations should be sent to results) and displays them in a leaderboard.

You should use this leaderboard if you have your own backend and plan to run it elsewhere.

Getting started

Defining environment variables

To get started on your own leaderboard, you will need to edit 2 files:

• src/envs.py to define your own environment variable (like the org name in which this has been copied)
• src/display/about.py with the tasks and number of few_shots you want for your tasks

Setting up fake results to initialize the leaderboard

Once this is done, you need to edit the "fake results" file to fit the format of your tasks: in the sub dictionary results, replace task_name1 and metric_name by the correct values you defined in Tasks above.

    "results": {
        "task_name1": {
            "metric_name": 0
        }
    }