Update README.md

README.md

@@ -7,7 +7,7 @@ tags:
 - landcover
 - IGN
 model-index:
-- name: FLAIR-INC_rgb_12cl_resnet34-unet
+- name: FLAIR-INC_rgbie_12cl_resnet34-unet
   results:
   - task:
       type: semantic-segmentation
@@ -17,62 +17,56 @@ model-index:
     metrics:
     - name: mIoU
       type: mIoU
-      value: 58.63
+      value: 62.716
     - name: Overall Accuracy
       type: OA
-      value: 76.3711
+      value: 76.509
     - name: Fscore
       type: Fscore
-      value: 72.4353
+      value: 75.907
     - name: Precision
       type: Precision
-      value: 74.3015
+      value: 76.525
     - name: Recall
       type: Recall
-      value: 72.4891
+      value: 75.714
       
     - name: IoU Buildings
       type: IoU
-      value: 82.6313
+      value: 82.564 
     - name: IoU Pervious surface
       type: IoU
-      value: 53.2351
+      value: 54.149
     - name: IoU Impervious surface
       type: IoU
-      value: 74.1742
+      value: 73.807
     - name: IoU Bare soil
       type: IoU
-      value: 60.3958
+      value: 59.013
     - name: IoU Water
       type: IoU
-      value: 87.5887
+      value: 87.216
     - name: IoU Coniferous
       type: IoU
-      value: 46.3504
+      value: 61.591
     - name: IoU Deciduous
       type: IoU
-      value: 67.4473
+      value: 72.225
     - name: IoU Brushwood
       type: IoU
-      value: 30.2346
+      value: 31.187
     - name: IoU Vineyard
       type: IoU
-      value: 82.9251
+      value: 76.105
     - name: IoU Herbaceous vegetation
       type: IoU
-      value: 55.0283      
+      value: 51.340      
     - name: IoU Agricultural land
       type: IoU
-      value: 52.0145
+      value: 57.558
     - name: IoU Plowed land
       type: IoU
-      value: 40.8387   
-    - name: IoU Swimming pool
-      type: IoU
-      value: 48.4433    
-    - name: IoU Greenhouse
-      type: IoU
-      value: 39.4447
+      value: 45.840   
       
 pipeline_tag: image-segmentation
 ---
@@ -91,13 +85,13 @@ pipeline_tag: image-segmentation
 <br>
 
 <div style="border:1px solid black; padding:25px; background-color:#FDFFF4 ; padding-top:10px; padding-bottom:1px;">
-  <h1>FLAIR-INC_rgbie_15cl_resnet34-unet</h1> 
-  <p>The general characteristics of this specific model <strong>FLAIR-INC_rgbie_15cl_resnet34-unet</strong> are :</p>
+  <h1>FLAIR-INC_rgbie_12cl_resnet34-unet</h1> 
+  <p>The general characteristics of this specific model <strong>FLAIR-INC_rgbie_12cl_resnet34-unet</strong> are :</p>
   <ul style="list-style-type:disc;">
     <li>Trained with the FLAIR-INC dataset</li>
     <li>RGBIE images (true colours + infrared + elevation)</li>
     <li>U-Net with a Resnet-34 encoder</li>
-    <li>15 class nomenclature : [building, pervious surface, impervious surface, bare soil, water, coniferous, deciduous, brushwood, vineyard, herbaceous, agricultural land, plowed land, swimming pool, snow, greenhouse]</li>
+    <li>12 class nomenclature : [building, pervious surface, impervious surface, bare soil, water, coniferous, deciduous, brushwood, vineyard, herbaceous, agricultural land, plowed land]</li>
   </ul>
 </div>
 
@@ -119,7 +113,7 @@ The product called ([BD ORTHO®](https://geoservices.ign.fr/bdortho)) has its ow
 Consequently, the model’s prediction would improve if the user images are similar to the original ones.
 
 _**Radiometry of input images**_ :
-The BD ORTHO input images are distributed in 8-bit encoding format per channel. When traning the model, input normalization was performed (see section **Trainingg Details**). 
+The BD ORTHO input images are distributed in 8-bit encoding format per channel. When traning the model, input normalization was performed (see section **Training Details**). 
 It is recommended that the user apply the same type of input normalization while inferring the model.
 
 _**Multi-domain model**_ :
@@ -133,23 +127,23 @@ When decoded to [0,255] ints, a difference of 1 should coresponds to 0.2 meters
 
 _**Land Cover classes of prediction**_ :
 The orginial class nomenclature of the FLAIR Dataset encompasses 19 classes (See the [FLAIR dataset](https://huggingface.co/datasets/IGNF/FLAIR) page for details).
-However 3 classes corresponding to uncertain labelisation (Mixed (16), Ligneous (17) and Other (19)) and 1 class with very poor labelling (Clear cut (15)) were desactivated during training.
-As a result, the logits produced by the model are of size 19x1, but classes n° 15, 16, 17 and 19 should appear at 0 in the logits and should not be present in the final argmax product.
+This model was trained to be coherent withe the FLAIR#1 scientific challenge in which contestants were evaluated of the first 12 classes of the nomenclature. Classes with label greater than 12 were desactivated during training. 
+As a result, the logits produced by the model are of size 19x1, but classes n° 13, 14, 15, 16, 17, 18 and 19 should appear at 0 in the logits and should not be present in the final argmax product.
 
 
 
 ## Bias, Risks, Limitations and Recommendations
 
 _**Using the model on input images with other spatial resolution**_ :
-The FLAIR-INC_rgbie_15cl_resnet34-unet model was trained with fixed scale conditions. All patches used for training are derived from aerial images with 0.2 meters spatial resolution. Only flip and rotate augmentations were performed during the training process.  
+The FLAIR-INC_rgbie_12cl_resnet34-unet model was trained with fixed scale conditions. All patches used for training are derived from aerial images with 0.2 meters spatial resolution. Only flip and rotate augmentations were performed during the training process.  
 No data augmentation method concerning scale change was used during training. The user should pay attention that generalization issues can occur while applying this model to images that have different spatial resolutions.
 
 _**Using the model for other remote sensing sensors**_ :
-The FLAIR-INC_rgbie_15cl_resnet34-unet model was trained with aerial images of the ([BD ORTHO® product](https://geoservices.ign.fr/bdortho)) that encopass very specific radiometric image processing. 
+The FLAIR-INC_rgbie_12cl_resnet34-unet was trained with aerial images of the ([BD ORTHO® product](https://geoservices.ign.fr/bdortho)) that encopass very specific radiometric image processing. 
 Using the model on other type of aerial images or satellite images may imply the use of transfer learning or domain adaptation techniques.
 
 _**Using the model on other spatial areas**_ :
-The FLAIR-INC_rgbie_15cl_resnet34-unet model was trained on patches reprensenting the French Metropolitan territory. 
+The FLAIR-INC_rgbie_12cl_resnet34-unet model was trained on patches reprensenting the French Metropolitan territory. 
 The user should be aware that applying the model to other type of landscapes may imply a drop in model metrics.  
 
 ---
@@ -166,7 +160,7 @@ Fine-tuning and prediction tasks are detailed in the README file.
 
 ### Training Data
 
-218 400 patches of 512 x 512 pixels were used to train the **FLAIR-INC_RVBIE_resnet34_unet_15cl_norm** model. 
+218 400 patches of 512 x 512 pixels were used to train the **FLAIR-INC_rgbie_12cl_resnet34-unet** model. 
 The train/validation split was performed patchwise to obtain a 80% / 20% distribution between train and validation. 
 Annotation was performed at the _zone_ level (~100 patches per _zone_). Spatial independancy between patches is guaranted as patches from the same _zone_ were assigned to the same set (TRAIN or VALIDATION). 
 The following number of patches were used for train and validation :
@@ -218,17 +212,17 @@ Statistics of the TRAIN+VALIDATION set :
 
 #### Speeds, Sizes, Times
 
-The FLAIR-INC_rgbie_15cl_resnet34-unet model was trained on a HPC/AI resources provided by GENCI-IDRIS (Grant 2022-A0131013803). 
+The FLAIR-INC_rgbie_12cl_resnet34-unet model was trained on a HPC/AI resources provided by GENCI-IDRIS (Grant 2022-A0131013803). 
 16 V100 GPUs were used ( 4 nodes, 4 GPUS per node). With this configuration the approximate learning time is 6 minutes per epoch.
 
-FLAIR-INC_rgbie_15cl_resnet34-unet was obtained for num_epoch=76 with corresponding val_loss=0.56. 
+FLAIR-INC_rgbie_12cl_resnet34-unet was obtained for num_epoch=65 with corresponding val_loss=0.55. 
 
 
 <div style="position: relative; text-align: center;">
     <p style="margin: 0;">TRAIN loss</p>
-    <img src="figs/train_loss_FLAIR-INC_RGBIE_resnet34_unet_15cl_norm.png" alt="TRAIN loss" style="width: 60%; display: block; margin: 0 auto;"/>
+    <img src="FLAIR-INC_rgbie_12cl_resnet34-unet_train-loss.png" alt="TRAIN loss" style="width: 60%; display: block; margin: 0 auto;"/>
     <p style="margin: 0;">VALIDATION loss</p>
-    <img src="figs/val_loss_FLAIR-INC_RGBIE_resnet34_unet_15cl_norm.png" alt="VALIDATION loss" style="width: 60%; display: block; margin: 0 auto;"/>
+    <img src="FLAIR-INC_rgbie_12cl_resnet34-unet_val-loss.png" alt="VALIDATION loss" style="width: 60%; display: block; margin: 0 auto;"/>
 </div>
 
 
@@ -243,37 +237,29 @@ The evaluation was performed on a TEST set of 31 750 patches that are independan
 The TEST set corresponds to the reunion of the TEST set of scientific challenges FLAIR#1 and FLAIR#2. See the [FLAIR challenge page](https://ignf.github.io/FLAIR/) for more details.
 
 The choice of a separate TEST set instead of cross validation was made to be coherent with the FLAIR challenges. 
-However the metrics for the Challenge were calculated on 12 classes and the TEST set acordingly. 
-As a result the _Snow_ class is absent from the TEST set.
+
 
 #### Metrics
 
-With the evaluation protocol, the **FLAIR-INC_RVBIE_resnet34_unet_15cl_norm** have been evaluated to **OA= 76.37%** and **mIoU=58.63%**. 
-The _snow_ class is discarded from the average metrics.
+With the evaluation protocol, the **FLAIR-INC_rgbie_12cl_resnet34-unet** have been evaluated to **OA=76.509%** and **mIoU=62.716%**. 
 
 The following table give the class-wise metrics :
 
-| Modalities              |   IoU (%)    | Fscore (%)  | Precision (%)  | Recall (%)  |
-| ----------------------- | ----------|---------|---------|---------|
-| building        	      |  82.63    |  90.49  |  90.26  |  90.72  |
-| pervious surface        |  53.24    |  69.48  |  68.97  |  70.00  |
-| impervious surface      |  74.17	  |  85.17  |  86.28  |  84.09  |
-| bare soil               |  60.40	  |  75.31  |  80.49  |  70.75  |
-| water                   |  87.59    |  93.38  |  93.16  |  93.61  |
-| coniferous        	  |  46.35    |  63.34  |  63.52  |  63.16  |
-| deciduous               |  67.45    |  80.56  |  77.44  |  83.94  |
-| brushwood               |  30.23	  |  46.43  |  63.55  |  36.58  |
-| vineyard                |  82.93	  |  90.67  |  91.35  |  89.99  |
-| herbaceous vegetation   |  55.03    |  70.99  |  70.59  |  71.40  |
-| agricultural land       |  52.01    |  68.43  |  59.18  |  81.12  |
-| plowed land             |  40.84    |  57.99  |  68.28  |  50.40  |
-| swimming_pool           |  48.44    |  65.27  |  81.62  |  54.37  |
-| _snow_                    |  _00.00_    |  _00.00_  |  _00.00_  |  _00.00_  |
-| greenhouse              |  39.45    |  56.57  |  45.52  |  74.72  |
-| **average**             |  **58.63**    |  **72.44**  |  **74.3**  |  **72.49**  |
-
-
-
+| Classes             |   IoU (%)    | Fscore (%)  | Precision (%)  | Recall (%)  |
+| ------------------- | ----------|---------|---------|---------|
+| building            |  82.564  |  90.449  |  90.412  |  90.486  |
+| pervious_surface    |  54.149  |  70.255  |  70.723  |  69.794  |
+| impervious_surface  |  73.807  |  84.930  |  85.848  |  84.032  |
+| bare_soil           |  59.013  |  74.224  |  78.795  |  70.154  |
+| water               |  87.216  |  93.172  |  91.692  |  94.699  |
+| coniferous          |  61.591  |  76.231  |  79.304  |  73.387  |
+| deciduous           |  72.225  |  83.873  |  81.728  |  86.133  |
+| brushwood           |  31.187  |  47.546  |  57.056  |  40.754  |
+| vineyard            |  76.105  |  86.432  |  84.264  |  88.714  |
+| herbaceous          |  51.340  |  67.847  |  70.134  |  65.705  |
+| agricultural_land   |  57.558  |  73.063  |  67.249  |  79.978  |
+| plowed_land         |  45.840  |  62.863  |  61.094  |  64.737  |
+| **average**         |  **62.716**    |  **75.907**  |  **76.525**  |  **75.714**  |
 
 
 
@@ -286,9 +272,9 @@ The following illustration gives the resulting confusion matrix :
 
 <div style="position: relative; text-align: center;">
     <p style="margin: 0;">Normalized Confusion Matrix (precision)</p>
-    <img src="figs/FLAIR-INC_RVBIE_resnet34_unet_15cl_norm_cm-precision.png" alt="drawing" style="width: 70%; display: block; margin: 0 auto;"/>
+    <img src="FLAIR-INC_rgbie_12cl_resnet34-unet_confmat_norm-precision.png" alt="drawing" style="width: 70%; display: block; margin: 0 auto;"/>
     <p style="margin: 0;">Normalized Confusion Matrix (recall)</p>
-    <img src="figs/FLAIR-INC_RVBIE_resnet34_unet_15cl_norm_cm-recall.png" alt="drawing" style="width: 70%; display: block; margin: 0 auto;"/>
+    <img src="FLAIR-INC_rgbie_12cl_resnet34-unet_confmat_norm-recall.png" alt="drawing" style="width: 70%; display: block; margin: 0 auto;"/>
 </div>