README.md

---
license: mit
---

# Model: PhilEO Bench

A novel evaluation framework for EO Foundation Models.

## Model Details

### Model Description

The PhilEO Bench evaluation framework comprises of a testbed that can be used to test any EO Foundation Model. The three downstream tasks are building density estimation, road segmentation, and land cover classification.

- **Developed by:** ESA, Phi-lab
- **Model type:** Evaluation Framework
- **License:** MIT

The aim of Foundation Models is to improve the performance on several diverse downstream tasks. However, these models are often evaluated on a range of datasets with different characteristics (size, resolution, locations, satellite sources, and capture dates). There is also a focus on evaluating classification downstream tasks, while omitting image-to-image downstream tasks (such as segmentation). Therefore, it is challenging to fairly compare the performance of these burgeoning EO FMs and draw meaningful conclusions. To evaluate FMs, we propose the PhilEO Bench, an evaluation framework with the aim of providing a flexible, consistent, and fair benchmark for EO Sentinel-2 FMs. 

## Uses

The PhilEO Bench is used to evaluate EO Foundation Models. - We introduce a new flexible evaluation framework focused on generating comparable, fair, and reproducible results.

### Model Sources

The basic links for the model are:

- **Paper:** https://arxiv.org/pdf/2401.04464.pdf 
- **Code:** http://github.com/ESA-PhiLab/PhilEO-Bench 
- **Project Website:** http://phileo-bench.github.io 
- **Repository:** http://huggingface.co/ESA-philab/PhilEO-Bench 
- **arXiv:** https://arxiv.org/abs/2401.04464 
- **Pre-trained models:** http://huggingface.co/ESA-philab/PhilEO-Bench/tree/main/pretrained_philab_models 
- **Data in:** http://huggingface.co/datasets/ESA-philab/PhilEO-downstream 

## Citation

Casper Fibaek, Luke Camilleri, Andreas Luyts, Nikolaos Dionelis, and Bertrand Le Saux, “PhilEO Bench: Evaluating Geo-Spatial Foundation Models,” arXiv:2401.04464, 2024.