File size: 1,743 Bytes
9094cc2
5111f27
675f890
9094cc2
 
 
31b9ddb
9094cc2
 
 
a92816b
9094cc2
9ef3bbd
 
 
 
 
 
a92816b
 
 
 
 
 
e6fac54
a92816b
 
 
 
 
 
 
e6fac54
a92816b
 
 
 
 
e6fac54
 
 
85a8429
e6fac54
 
 
a92816b
 
 
e6fac54
 
 
 
 
 
 
a37788b
580b4e4
 
 
 
 
 
a92816b
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
---
title: Model Evaluator
emoji: πŸ“Š
colorFrom: red
colorTo: red
sdk: streamlit
sdk_version: 1.10.0
app_file: app.py
---

# Model Evaluator

> Submit evaluation jobs to AutoTrain from the Hugging Face Hub

## Supported tasks

The table below shows which tasks are currently supported for evaluation in the AutoTrain backend:

| Task                               | Supported |
|:-----------------------------------|:---------:|
| `binary_classification`            |     βœ…     |
| `multi_class_classification`       |     βœ…     |
| `multi_label_classification`       |     ❌     |
| `entity_extraction`                |     βœ…     |
| `extractive_question_answering`    |     βœ…     |
| `translation`                      |     βœ…     |
| `summarization`                    |     βœ…     |
| `image_binary_classification`      |     βœ…     |
| `image_multi_class_classification` |     βœ…     |

## Installation

To run the application locally, first clone this repository and install the dependencies as follows:

```
pip install -r requirements.txt
```

Next, copy the example file of environment variables:

```
cp .env.template .env
```

and set the `HF_TOKEN` variable with a valid API token from the `autoevaluator` user. Finally, spin up the application by running:

```
streamlit run app.py
```

## AutoTrain configuration details

Models are evaluated by AutoTrain, with the payload sent to the `AUTOTRAIN_BACKEND_API` environment variable. The current configuration for evaluation jobs running on Spaces is:

```
AUTOTRAIN_BACKEND_API=https://api-staging.autotrain.huggingface.co
```

To evaluate models with a _local_ instance of AutoTrain, change the environment to:

```
AUTOTRAIN_BACKEND_API=http://localhost:8000
```