Image-Text-to-Text
Transformers
Safetensors
English
idefics2
pretraining
multimodal
vision
Inference Endpoints
5 papers
VictorSanh HF staff commited on
Commit
27780fc
1 Parent(s): baf1ac8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -305,7 +305,7 @@ Alongside this evaluation, we also computed the classification accuracy on FairF
305
 
306
  | Model | Shots | <nobr>FairFaceGender<br>acc. (std*)</nobr> | <nobr>FairFaceRace<br>acc. (std*)</nobr> | <nobr>FairFaceAge<br>acc. (std*)</nobr> |
307
  | :--------------------- | --------: | ----------------------------: | --------------------------: | -------------------------: |
308
- | IDEFICS-1 9B (Instructed) | 0 | 92.7 (6.3) | 59.6 (22.2) | 43.9 (3.9) |
309
  | IDEFICS-2 8B (Instructed) | 0 | 96.3 (3.0) | 41.6 (40.9) | 53.5 (3.0) |
310
 
311
  *Per bucket standard deviation. Each bucket represents a combination of ethnicity and gender from the [FairFace](https://huggingface.co/datasets/HuggingFaceM4/FairFace) dataset. The standard deviation within each demographic group indicates the disparity in the model's ability to recognize gender, ethnicity, or age across different groups. Specifically, for the IDEFICS-2 model, we notice a notably higher standard deviation in predicting ethnicity. This is evident in its near-zero accuracy for images depicting individuals of Middle Eastern, Latino/Hispanic, and Southeast Asian descent.
 
305
 
306
  | Model | Shots | <nobr>FairFaceGender<br>acc. (std*)</nobr> | <nobr>FairFaceRace<br>acc. (std*)</nobr> | <nobr>FairFaceAge<br>acc. (std*)</nobr> |
307
  | :--------------------- | --------: | ----------------------------: | --------------------------: | -------------------------: |
308
+ | IDEFICS-1 80B (Instructed) | 0 | 92.7 (6.3) | 59.6 (22.2) | 43.9 (3.9) |
309
  | IDEFICS-2 8B (Instructed) | 0 | 96.3 (3.0) | 41.6 (40.9) | 53.5 (3.0) |
310
 
311
  *Per bucket standard deviation. Each bucket represents a combination of ethnicity and gender from the [FairFace](https://huggingface.co/datasets/HuggingFaceM4/FairFace) dataset. The standard deviation within each demographic group indicates the disparity in the model's ability to recognize gender, ethnicity, or age across different groups. Specifically, for the IDEFICS-2 model, we notice a notably higher standard deviation in predicting ethnicity. This is evident in its near-zero accuracy for images depicting individuals of Middle Eastern, Latino/Hispanic, and Southeast Asian descent.