Spaces:
Running
Running
| title: Model Evaluator | |
| emoji: 📊 | |
| colorFrom: red | |
| colorTo: red | |
| sdk: streamlit | |
| sdk_version: 1.10.0 | |
| app_file: app.py | |
| # Model Evaluator | |
| > Submit evaluation jobs to AutoTrain from the Hugging Face Hub | |
| ## Supported tasks | |
| The table below shows which tasks are currently supported for evaluation in the AutoTrain backend: | |
| | Task | Supported | | |
| |:-----------------------------------|:---------:| | |
| | `binary_classification` | ✅ | | |
| | `multi_class_classification` | ✅ | | |
| | `multi_label_classification` | ❌ | | |
| | `entity_extraction` | ✅ | | |
| | `extractive_question_answering` | ✅ | | |
| | `translation` | ✅ | | |
| | `summarization` | ✅ | | |
| | `image_binary_classification` | ✅ | | |
| | `image_multi_class_classification` | ✅ | | |
| ## Installation | |
| To run the application locally, first clone this repository and install the dependencies as follows: | |
| ``` | |
| pip install -r requirements.txt | |
| ``` | |
| Next, copy the example file of environment variables: | |
| ``` | |
| cp .env.examples .env | |
| ``` | |
| and set the `HF_TOKEN` variable with a valid API token from the `autoevaluator` user. Finally, spin up the application by running: | |
| ``` | |
| streamlit run app.py | |
| ``` | |
| ## AutoTrain configuration details | |
| Models are evaluated by AutoTrain, with the payload sent to the `AUTOTRAIN_BACKEND_API` environment variable. The current configuration for evaluation jobs running on Spaces is: | |
| ``` | |
| AUTOTRAIN_BACKEND_API=https://api.autotrain.huggingface.co | |
| ``` | |
| To evaluate models with a _local_ instance of AutoTrain, change the environment to: | |
| ``` | |
| AUTOTRAIN_BACKEND_API=http://localhost:8000 | |
| ``` |