earthnet-minicuber 0.1.3
EarthNet Minicuber


# EarthNet Minicuber

*A Python library for creating EarthNet-style minicubes.*


<a href='https://pypi.python.org/pypi/earthnet-minicuber'>
    <img src='https://img.shields.io/pypi/v/earthnet-minicuber.svg' alt='PyPI' />
</a>
<a href="https://opensource.org/licenses/MIT" target="_blank">
    <img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="License">
</a>
<a href="https://twitter.com/vitusbenson" target="_blank">
    <img src="https://img.shields.io/twitter/follow/vitusbenson?style=social" alt="Twitter">
</a>


**GitHub**: [https://github.com/earthnet2021/earthnet-minicuber](https://github.com/earthnet2021/earthnet-minicuber)

**PyPI**: [https://pypi.org/project/earthnet-minicuber/](https://pypi.org/project/earthnet-minicuber/)



This package creates minicubes from cloud storage using STAC catalogues. A minicube usually contains a satellite image time series of Sentinel 2 imagery alongside other complementary information, all re-gridded to a common grid. This package implements a cloud mask based on deep learning, which allows for analysis-ready Sentinel 2 imagery.

It is currently under development, thus do expect bugs and please report them!


## Tutorial

1. Loading the package
```Python
import earthnet_minicuber as emc
```

2. Creating a dictionary with specifications of the desired minicube
```Python
specs = {
    "lon_lat": (43.598946, 3.087414), # center pixel
    "xy_shape": (256, 256), # width, height of cutout around center pixel
    "resolution": 10, # in meters.. will use this on a local UTM grid..
    "time_interval": "2021-07-01/2021-07-31",
    "providers": [
        {
            "name": "s2",
            "kwargs": {"bands": ["B02", "B03", "B04", "B8A"], "best_orbit_filter": True, "five_daily_filter": False, "brdf_correction": True, "cloud_mask": True, "aws_bucket": "planetary_computer"}
        },
        {
            "name": "s1",
            "kwargs": {"bands": ["vv", "vh"], "speckle_filter": True, "speckle_filter_kwargs": {"type": "lee", "size": 9}, "aws_bucket": "planetary_computer"} 
        },
        {
            "name": "ndviclim",
            "kwargs": {"bands": ["mean", "std"]}
        },
        {
            "name": "cop",
            "kwargs": {}
        },
        {
            "name": "esawc",
            "kwargs": {"bands": ["lc"], "aws_bucket": "planetary_computer"}
        }
        ]
}
```

3. Downloading the minicube
```Python
mc = emc.load_minicube(specs, compute = True)
```

4. Plotting cloud-masked Sentinel 2 RGB imagery
```Python
emc.plot_rgb(mc)
```

See `notebooks/example.ipynb` for a more detailed usage example.



## Data Providers

The minicuber is centered around the concept of data providers, which wrap a data source and handle data loading of that source. The `emc.Minicuber` class then manages these data providers, by telling them the spatio-temporal range for which data needs to be loaded and afterwards re-gridding all data to a common reference frame (UTM grid).

### Sentinel 2

The Sentinel 2 provider loads and processes Copernicus Sentinel 2 imagery.

Kwargs:
- `bands`: choose any subset from `["B01", "B02", "B03", "B04", "B05", "B06", "B07", "B08", "B8A", "B09", "B11", "B12", "WVP", "AOT", "SCL"]`.
- `aws_bucket`: We currently support data loading from three cloud buckets: Microsoft Planetary Computer (`"planetary_computer"`), Element84 AWS bucket (`element84`) and DigitalEarthAfrica AWS bucket (`dea`). We recommend using the Microsoft planetary computer with the keyword argument `aws_bucket = "planetary_computer"`.
- `best_orbit_filter`: Sentinel 2 has a regular overpass frequency of 5 days. However, sometimes it can be smaller due to off-nadir captures. Such captures change the viewing angle of the scene. If `True`, this filter finds the best orbit and then only returns imagery from a regular 5-daily cycle.
- `five_daily_filter`: If `True` returns a regular 5-daily cycle starting with the first date in `full_time_interval`. It has no effect, if `best_orbit_filter` is used.
- `brdf_correction`: If `True`, does BRDF correction based on the Sentinel 2 Metadata (illumination angles).
- `cloud_mask`: If `True`, creates a cloud and cloud shadow mask based on deep learning. It automatically finds the best available cloud mask for the requested `bands`.
- `cloud_mask_rescale_factor`: If using cloud mask and a lower resolution than 10m, set this rescaling factor to the multiple of 10m that you are requesting. E.g. if `resolution = 20`, set `cloud_mask_rescale_factor = 2`.
- `correct_processing_baseline`: If `True` (default): corrects the shift of +1000 that exists in Sentinel 2 data with processing baseline >= 4.0


## Installation

Prerequisites (We use an Anaconda environment):

```
conda create -n minicuber python=3.10 gdal cartopy -c conda-forge
conda deactivate
conda activate minicuber
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install scipy matplotlib seaborn netCDF4 xarray zarr dask shapely pillow pandas s3fs fsspec boto3 psycopg2 pystac-client stackstac planetary-computer rasterio[s3] rioxarray odc-algo segmentation-models-pytorch folium ipykernel ipywidgets sen2nbar
```

Install this package with PyPI:
```
pip install earthnet-minicuber
```

or install this package in developing mode with
```
git clone https://github.com/earthnet2021/earthnet-minicuber.git
cd earthnet-minicuber
pip install -e .
```

or directly with
```
pip install git+https://github.com/earthnet2021/earthnet-minicuber.git
```

## Similar Packages

This package is build on top of [stackstac](https://stackstac.readthedocs.io/en/latest/), which allows accessing data stored in cloud-optimized geotiffs with xarray.

Similar to this package, [cubo](https://github.com/davemlz/cubo) provides a high-level interface to stackstac.


## Acknowledgement

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101004188 ([DeepCube Horizon 2020](https://deepcube-h2020.eu/ "DeepCube Horizon 2020")). We are grateful to David Montero Loaiza for providing the [sen2nbar](https://github.com/ESDS-Leipzig/sen2nbar) package used for the Sentinel 2 BRDF correction. We are grateful to César Aybar and the [CloudSEN12](https://cloudsen12.github.io/) team, their work forms the basis for the cloud mask implemented in earthnet-minicuber.