license: etalab-2.0
tags:
- segmentation
- pytorch
- aerial imagery
- landcover
- IGN
model-index:
- name: FLAIR-INC_RVBIE_unetresnet34_15cl_norm
results:
- task:
type: semantic-segmentation
dataset:
name: IGNF/FLAIR#1-TEST
type: earth-observation-dataset
metrics:
- name: mIoU
type: mIoU
value: 54.7168
- name: Overall Accuracy
type: OA
value: 76.3711
- name: Fscore
type: Fscore
value: 67.6063
- name: Precision
type: Precision
value: 69.3481
- name: Recall
type: Recall
value: 67.6565
- name: IoU Buildings
type: IoU
value: 82.6313
- name: IoU Pervious surface
type: IoU
value: 53.2351
- name: IoU Impervious surface
type: IoU
value: 74.1742
- name: IoU Bare soil
type: IoU
value: 60.3958
- name: IoU Water
type: IoU
value: 87.5887
- name: IoU Coniferous
type: IoU
value: 46.3504
- name: IoU Deciduous
type: IoU
value: 67.4473
- name: IoU Brushwood
type: IoU
value: 30.2346
- name: IoU Vineyard
type: IoU
value: 82.9251
- name: IoU Herbaceous vegetation
type: IoU
value: 55.0283
- name: IoU Agricultural land
type: IoU
value: 52.0145
- name: IoU Plowed land
type: IoU
value: 40.8387
- name: IoU Swimming pool
type: IoU
value: 48.4433
- name: IoU Greenhouse
type: IoU
value: 39.4447
pipeline_tag: image-segmentation
FLAIR model collection
The FLAIR models is a collection of semantic segmentation models initially developed to classify land cover on very high resolution aerial ortho-images (BD ORTHO®). The distributed pre-trained models differ in their :
- input modalities : RVB (true colours), RVBI (true colours + infrared), RVBIE (true colours + infrared + elevation)
- model architecture : U-Net with a Resnet-34 encoder, Deeplab
- target class nomenclature : 12 or 15 land cover classes
- dataset for training : FLAIR dataset or the increased version of this dataset FLAIR-INC.
FLAIR FLAIR-INC_RVBIE_resnet34_unet_15cl_norm model
The general characteristics of this specific model FLAIR-INC_RVBIE_resnet34_unet_15cl_norm are :
- RVBIE images (true colours + infrared + elevation)
- U-Net with a Resnet-34 encoder
- 15 class nomenclature [building,pervious_surface,impervious_surface,bare_soil,water,coniferous,deciduous,brushwood,vineyard,herbaceous,agricultural_land,plowed_land,swimming pool,snow,greenhouse]
Model Informations
- Repository: https://github.com/IGNF/FLAIR-1-AI-Challenge
- Paper [optional]: https://arxiv.org/pdf/2211.12979.pdf
- Developed by: IGN
- Compute infrastructure:
- software: python, pytorch-lightning
- hardware: GENCI, XXX
- License: : Apache 2.0
Uses
The model has been trained with
Radiometry of input images : The input images are distributed in 8-bit encoding format per channel. or traning the model, input normalization was performed (see section Traing Details). It is recommended that the user apply the same type of input normalization while inferring the model.
Multi-domain model : The FLAIR-INC dataset that was used for training is composed of 75 radiometric domains. In the case of aerial images, domain shifts are due : the date of the aerial survey (april to november), spatial domain (equivalent to a french department administrative division) and downstream radimetric processing. By construction the model is robust to theses shifts, and can be applied to any images of the (BD ORTHO® product).
Land Cover classes of prediction : The orginial class nomenclature of the FLAIR Dataset is made up of 19 classes(See the FLAIR dataset page for details). However 3 classes corresponding to uncertain labelisation (Mixed (16), Ligneous (17) and Other (19)) and 1 class with very poor label quantity (Clear cut (15)) were deasctivated during training. As a result, the logits produced by the model are of size 19x1, but class 15,16,17 and 19 should appear at 0 in the logits. And labels 15,16,17 and 19 never predicted in the argmax.
Bias, Risks, and Limitations
Using the model on input images with other spatial resolution : The FLAIR-INC_RVBIE_resnet34_unet_15cl_norm model has been trained with fixed scale conditions. All patches used for training are derived from aerial images of 0.2 meters spatial resolution. No data augmentation method concerning scale change was used during training. The user should pay attention that generalization issues can occur while applying this model to images that have different spatial resolutions.
Using the model for other remote sensing sensors : The FLAIR-INC_RVBIE_resnet34_unet_15cl_norm model has been trained with aerial images of the (BD ORTHO® product) that encopass very specific radiometric image processing. Using the model on other type of aerial images or satellite images may imply the use of transfer learning or domain adaptation techniques.
Using the model on other spatial areas : The FLAIR-INC_RVBIE_resnet34_unet_15cl_norm model has been trained on patches reprensenting the French Metropolitan territory. The user should be aware that applying the model to other type of landscapes may imply a drop in model metrics.
{{ bias_risks_limitations | default("[More Information Needed]", true)}}
Recommendations
{{ bias_recommendations | default("Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.", true)}}
How to Get Started with the Model
Use the code below to get started with the model.
{{ get_started_code | default("[More Information Needed]", true)}}
Training Details
Training Data
218 400 patchs of 512 x 512 pixels were used to train the model. The train/validation split was performed patchwise to obtain a 80% / 20% distribution. Spatial independancy between patches is guaranted : neighbouring patches are assigned to the same partition :
- Train set : 174 700 patches
- Validation set : 43 700 patchs
{{ training_data | default("[More Information Needed]", true)}}
Training Procedure
Preprocessing [optional]
For traning the model, input normalization was performed so as the input dataset has a mean of 0 and a standart deviation of 1. For this model here are the statistics of the TRAIN+VALIDATION partition. It is recommended that the user apply the same type of input normalization. Input normalization was performed
| Modalities | Mean (Train + Validation) | Std (Train + Validation) |
|---|---|---|
| Red Channel (R) | 105.08 | 52.17 |
| Green Channel (V) | 110.87 | 45.38 |
| Blue Channel (B) | 101.82 | 44.00 |
| Infrared Channel (I) | 106.38 | 39.69 |
| Elevation Channel (E) | 53.26 | 79.30 |
{{ preprocessing | default("[More Information Needed]", true)}}
Training Hyperparameters
- Training regime: {{ training_regime | default("[More Information Needed]", true)}}
- Model architecture: Unet (implementation from the Segmentation Models Pytorch library
- Encoder : Resnet-34 pre-trained with ImageNet
- Augmentation :
- VerticalFlip(p=0.5)
- HorizontalFlip(p=0.5)
- RandomRotate90(p=0.5)
- Seed: 2022
- Batch size: 10
- Optimizer : SGD
- Learning rate : 0.02
- Class Weights :
- 1: [1, building]
- 2: [1, pervious surface]
- 3: [1, impervious surface]
- 4: [1, bare soil]
- 5: [1, water]
- 6: [1, coniferous]
- 7: [1, deciduous]
- 8: [1, brushwood]
- 9: [1, vineyard]
- 10: [1,herbaceous vegetation]
- 11: [1, agricultural land]
- 12: [1, plowed land]
- 13: [1, swimming_pool]
- 14: [1, snow]
- 15: [0, clear cut]
- 16: [0, mixed]
- 17: [0, ligneous]
- 18: [1, greenhouse]
- 19: [0, other]
Speeds, Sizes, Times [optional]
{{ speeds_sizes_times | default("[More Information Needed]", true)}}
Evaluation
Testing Data, Factors & Metrics
Testing Data
{{ testing_data | default("[More Information Needed]", true)}}
Metrics
{{ testing_metrics | default("[More Information Needed]", true)}}
Results
{{ results | default("[More Information Needed]", true)}}
Summary
{{ results_summary | default("", true) }}
Technical Specifications [optional]
Model Architecture and Objective
{{ model_specs | default("[More Information Needed]", true)}}
Citation [optional]
BibTeX:
{{ citation_bibtex | default("[More Information Needed]", true)}}
APA:
{{ citation_apa | default("[More Information Needed]", true)}}