VictorSanh
commited on
Commit
•
27780fc
1
Parent(s):
baf1ac8
Update README.md
Browse files
README.md
CHANGED
@@ -305,7 +305,7 @@ Alongside this evaluation, we also computed the classification accuracy on FairF
|
|
305 |
|
306 |
| Model | Shots | <nobr>FairFaceGender<br>acc. (std*)</nobr> | <nobr>FairFaceRace<br>acc. (std*)</nobr> | <nobr>FairFaceAge<br>acc. (std*)</nobr> |
|
307 |
| :--------------------- | --------: | ----------------------------: | --------------------------: | -------------------------: |
|
308 |
-
| IDEFICS-1
|
309 |
| IDEFICS-2 8B (Instructed) | 0 | 96.3 (3.0) | 41.6 (40.9) | 53.5 (3.0) |
|
310 |
|
311 |
*Per bucket standard deviation. Each bucket represents a combination of ethnicity and gender from the [FairFace](https://huggingface.co/datasets/HuggingFaceM4/FairFace) dataset. The standard deviation within each demographic group indicates the disparity in the model's ability to recognize gender, ethnicity, or age across different groups. Specifically, for the IDEFICS-2 model, we notice a notably higher standard deviation in predicting ethnicity. This is evident in its near-zero accuracy for images depicting individuals of Middle Eastern, Latino/Hispanic, and Southeast Asian descent.
|
|
|
305 |
|
306 |
| Model | Shots | <nobr>FairFaceGender<br>acc. (std*)</nobr> | <nobr>FairFaceRace<br>acc. (std*)</nobr> | <nobr>FairFaceAge<br>acc. (std*)</nobr> |
|
307 |
| :--------------------- | --------: | ----------------------------: | --------------------------: | -------------------------: |
|
308 |
+
| IDEFICS-1 80B (Instructed) | 0 | 92.7 (6.3) | 59.6 (22.2) | 43.9 (3.9) |
|
309 |
| IDEFICS-2 8B (Instructed) | 0 | 96.3 (3.0) | 41.6 (40.9) | 53.5 (3.0) |
|
310 |
|
311 |
*Per bucket standard deviation. Each bucket represents a combination of ethnicity and gender from the [FairFace](https://huggingface.co/datasets/HuggingFaceM4/FairFace) dataset. The standard deviation within each demographic group indicates the disparity in the model's ability to recognize gender, ethnicity, or age across different groups. Specifically, for the IDEFICS-2 model, we notice a notably higher standard deviation in predicting ethnicity. This is evident in its near-zero accuracy for images depicting individuals of Middle Eastern, Latino/Hispanic, and Southeast Asian descent.
|