Update readme and doc from the 80b repo

README.md

@@ -4,6 +4,7 @@ tags:
 - multimodal
 - text
 - image
+- image-to-text
 license: other
 datasets:
 - HuggingFaceM4/OBELICS
@@ -305,9 +306,9 @@ Similarly to the base IDEFICS models, we performed checkpoint selection to stop
 
 ## Hardware
 
-The IDEFICS models were trained on an AWS SageMaker cluster with 8x80GB A100 GPUs nodes and EFA network. 
+The IDEFICS models were trained on an AWS SageMaker cluster with 8x80GB A100 GPUs nodes and EFA network.
 
-- IDEFICS-80B took ~28 days of training on 64 nodes (512 GPUs). 
+- IDEFICS-80B took ~28 days of training on 64 nodes (512 GPUs).
 - IDEFICS-80b-instruct finetuned the base model for ~3 days on 48 nodes (384 GPUs).
 
 
@@ -316,6 +317,47 @@ The IDEFICS models were trained on an AWS SageMaker cluster with 8x80GB A100 GPU
 The training software is built on top of HuggingFace Transformers + Accelerate, and [DeepSpeed ZeRO-3](https://github.com/microsoft/DeepSpeed) for training, and [WebDataset](https://github.com/webdataset/webdataset) for data loading.
 
 
+## Environmental Impact
+
+We distinguish the 3 phases of the creation of IDEFICS and report our carbon emissions separately for each one of them:
+
+*Preliminary experimentation*
+- **Hardware Type:** Intel Cascade Lake CPUs, NVIDIA V100 and A100 GPUs
+- **Hours used:** 460,000 CPU hours, 385,000 V100 GPU hours, and 300,000 A100 GPU hours
+- **Cloud Provider:** N/A (Jean Zay cluster)
+- **Compute Region:** France (57g CO2eq/kWh)
+- **Carbon Emitted:** 16,714 kgs of CO2eq
+
+*IDEFICS-9b pretraining*
+- **Hardware Type:** 128 NVIDIA A100 GPUs
+- **Hours used:** 350 hours
+- **Cloud Provider:** AWS
+- **Compute Region:** US-West 2 (288g CO2eq/kWh)
+- **Carbon Emitted:** 5,160 kg of CO2eq
+
+*IDEFICS-9b-instruct finetuning*
+- **Hardware Type:** 128 NVIDIA A100 GPUs
+- **Hours used:** 70 hours
+- **Cloud Provider:** AWS
+- **Compute Region:** US-West 2 (288g CO2eq/kWh)
+- **Carbon Emitted:** 1,032 kg of CO2eq
+
+*IDEFICS-80b pretraining*
+- **Hardware Type:** 512 NVIDIA A100 GPUs
+- **Hours used:** 672 hours (28 days)
+- **Cloud Provider:** AWS
+- **Compute Region:** US-West 2 (288g CO2eq/kWh)
+- **Carbon Emitted:** 39,498 kg of CO2eq
+
+*IDEFICS-80b-instruct finetuning*
+- **Hardware Type:** 384 NVIDIA A100 GPUs
+- **Hours used:** 72 hours (3 days)
+- **Cloud Provider:** AWS
+- **Compute Region:** US-West 2 (288g CO2eq/kWh)
+- **Carbon Emitted:** 3,174 kg of CO2eq
+
+This means that the total carbon footprint of the entire IDEFICS project can be estimated at **65.57 tons of CO2eq**, which is roughly equal to 168,092 miles driven by an average gasoline-powered car or 8.3 homes' energy use for one year, according to the [US Environmental Protection Agency](https://www.epa.gov/energy/greenhouse-gas-equivalencies-calculator).
+
 # Bias, Risks, and Limitations
 
 Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)).
@@ -438,4 +480,4 @@ Stas Bekman*, Léo Tronchon*, Hugo Laurençon*, Lucile Saulnier*, Amanpreet Sing
 
 # Model Card Contact
 
-Please open a discussion on the Community tab!
+Please open a discussion on the Community tab!