Leyo HF staff commited on
Commit
00e1183
1 Parent(s): 87c2f77

update fairness std

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -358,12 +358,12 @@ The [notebook](https://huggingface.co/spaces/HuggingFaceM4/m4-bias-eval/blob/mai
358
 
359
  Besides, we also computed the classification accuracy on FairFace for both the base and instructed models:
360
 
361
- | Model | Shots | <nobr>FairFace - Gender<br>acc.</nobr> | <nobr>FairFace - Gender<br>per bucket std.</nobr> | <nobr>FairFace - Race<br>acc.</nobr> | <nobr>FairFace - Race<br>per bucket std.</nobr> | <nobr>FairFace - Age<br>acc.</nobr> | <nobr>FairFace - Age<br>per bucket std.</nobr> |
362
- |:---------------------|--------:|----------------------------:|-----------------------:|--------------------------:|---------------------:|-------------------------:|--------------------:|
363
- | IDEFICS 80B | 0 | 95.8 | 1.0 | 64.1 | 16.1 | 51.0 | 2.9 |
364
- | IDEFICS 9B | 0 | 94.4 | 2.2 | 55.3 | 13.0 | 45.1 | 2.9 |
365
- | IDEFICS 80B Instruct | 0 | 95.7 | 2.4 | 63.4 | 25.6 | 47.1 | 2.9 |
366
- | IDEFICS 9B Instruct | 0 | 92.7 | 6.3 | 59.6 | 22.2 | 43.9 | 3.9 |
367
 
368
 
369
  | Model | Shots | <nobr>Task<br> | <nobr>Gender<br> | <nobr>White<br> | <nobr>Black<br> | <nobr>Latino_Hispanic<br> | <nobr>East<br> | <nobr>Southeast Asian<br> | <nobr>Indian<br> | <nobr>Middle Eastern<br> | <nobr>Average<br> |
 
358
 
359
  Besides, we also computed the classification accuracy on FairFace for both the base and instructed models:
360
 
361
+ | Model | Shots | <nobr>FairFaceGender<br>acc. (std)</nobr> | <nobr>FairFaceRace<br>acc. (std)</nobr> | <nobr>FairFaceAge<br>acc. (std)</nobr> |
362
+ | :--------------------- | --------: | ----------------------------: | --------------------------: | -------------------------: |
363
+ | IDEFICS 80B | 0 | 95.8 (1.0) | 64.1 (16.1) | 51.0 (2.9) |
364
+ | IDEFICS 9B | 0 | 94.4 (2.2) | 55.3 (13.0) | 45.1 (2.9) |
365
+ | IDEFICS 80B Instruct | 0 | 95.7 (2.4) | 63.4 (25.6) | 47.1 (2.9) |
366
+ | IDEFICS 9B Instruct | 0 | 92.7 (6.3) | 59.6 (22.2) | 43.9 (3.9) |
367
 
368
 
369
  | Model | Shots | <nobr>Task<br> | <nobr>Gender<br> | <nobr>White<br> | <nobr>Black<br> | <nobr>Latino_Hispanic<br> | <nobr>East<br> | <nobr>Southeast Asian<br> | <nobr>Indian<br> | <nobr>Middle Eastern<br> | <nobr>Average<br> |