README.md

metadata

title: Photographic memory leaderboard
emoji: 🥇
colorFrom: green
colorTo: indigo
sdk: gradio
sdk_version: 4.4.0
app_file: app.py
pinned: true
license: apache-2.0

Run commands:

To run this code locally:

1. Change the environment variables in src/env to add your API keys.

2. To run the backend server + UI locally, run the following command:

./start.sh RUN_MODE="LOCAL" LOCAL_MODEL_NAME="<any huggingface or openai model you want to try>"

Eg:
./start.sh RUN_MODE="LOCAL" LOCAL_MODEL_NAME="hf-internal-testing/tiny-random-gpt2"

Adhoc evaluation:

To run only the backend server locally to quickly get the evaluation results, run the following command:

./run-adhoc.sh LOCAL_MODEL_NAME="<any huggingface or openai model you want to try>"

This will still allow you to save your benchmarking results in the folder output-data.