s2etrac 1.1.0
SENTINEL-2 EXTRACTION TOOL FOR RADIOMETRIC ANALYSIS AND CALIBRATION
Requires Python: >=3.10,<3.11

# s2etrac

## Table of Contents

- [s2etrac](#s2etrac)
  - [Table of Contents](#table-of-contents)
  - [Description](#description)
  - [Terminology](#terminology)
  - [Installation](#installation)
    - [With Docker](#with-docker)
      - [Using GitLab Container Registry](#using-gitlab-container-registry)
      - [Locally](#locally)
    - [Without Docker](#without-docker)
      - [Requirements](#requirements)
      - [Procedure](#procedure)
      - [Troubleshooting](#troubleshooting)
        - [`Failed to unlock the collection!` when running `poetry install`](#failed-to-unlock-the-collection-when-running-poetry-install)
  - [Usage](#usage)
    - [With Docker](#with-docker-1)
    - [Without Docker](#without-docker-1)
    - [Command-Line Interface Help](#command-line-interface-help)
    - [Troubleshooting](#troubleshooting-1)
      - [`unrecognized engine cfgrib`](#unrecognized-engine-cfgrib)
  - [Documentation](#documentation)
  - [Jupyter Notebooks](#jupyter-notebooks)
  - [S2ETRAC Architecture - Standalone version](#s2etrac-architecture---standalone-version)
    - [Data Model](#data-model)
    - [High-Level Command Line Interface (Inputs and Outputs)](#high-level-command-line-interface-inputs-and-outputs)
    - [The Processor](#the-processor)
    - [Reproject, Clip and Merge](#reproject-clip-and-merge)
    - [Source Code Tree](#source-code-tree)
  - [Development](#development)
    - [Integration in VSCode](#integration-in-vscode)
      - [Troubleshooting](#troubleshooting-2)
        - [`Pytest Discovery Error` in VSCode](#pytest-discovery-error-in-vscode)
    - [Debugging](#debugging)
    - [Testing](#testing)
      - [Coverage](#coverage)
    - [Code Quality](#code-quality)
      - [Format the code with `black`](#format-the-code-with-black)
      - [Lint the code with `pylint`](#lint-the-code-with-pylint)
      - [Lint the code with `flake8`](#lint-the-code-with-flake8)
      - [Verify types with `mypy`](#verify-types-with-mypy)
      - [Prune unused imports with `pautoflake`](#prune-unused-imports-with-pautoflake)
      - [Sort unused imports with `isort`](#sort-unused-imports-with-isort)
    - [Bump version](#bump-version)
    - [Performance](#performance)
      - [Using `scalene`](#using-scalene)
  - [Implementation Notes](#implementation-notes)
    - [Dimension Names](#dimension-names)
    - [Products are rejected when spreaded over multiple DataStrips](#products-are-rejected-when-spreaded-over-multiple-datastrips)
    - [Handling of new mask names by S2ETRAC](#handling-of-new-mask-names-by-s2etrac)
      - [Thresholds (Extraction Parameters in the ATBD)](#thresholds-extraction-parameters-in-the-atbd)
      - [Variable names in output NetCDF](#variable-names-in-output-netcdf)
    - [Rejection of full product if one band is invalid](#rejection-of-full-product-if-one-band-is-invalid)
  - [Links](#links)
    - [On "PixelIsArea" or "PixelIsPoint"](#on-pixelisarea-or-pixelispoint)
  - [Misc](#misc)
    - [Emojis](#emojis)

## Description

![Raster resolutions per band](./docs/source/_static/assets/diagrams/s2etrac-raster-size-comparison.svg)
![Raster resolutions per band](./_static/assets/diagrams/s2etrac-raster-size-comparison.svg)

## Terminology

## Installation

### With Docker

#### Using GitLab Container Registry

💡 The image will be downloadable from the [GitLab Registry](https://gitlab.acri-cwa.fr/opt-mpc/s2etrac/s2etrac-standalone/container_registry/1894).

Login to the registry:

    docker login gitlabreg.acri-cwa.fr:443/opt-mpc/s2etrac/s2etrac-standalone/

Pull the latest image from the registry:

    docker pull gitlabreg.acri-cwa.fr:443/opt-mpc/s2etrac/s2etrac-standalone/s2etrac-standalone:latest

#### Locally

Locally, you can build the image using `scripts/build_image.sh`, which does:

```bash
docker build --tag s2etrac-image:${VERSION} .
```

### Without Docker

#### Requirements

`s2etrac` requires Python 3.10 and [Poetry](https://python-poetry.org/) to be installed and ran.

Poetry lists the other necessary requirements for the project.

#### Procedure

```bash
# Prerequisite:
# > This library also requires the eccodes library to be able to parse the GRIB ECMWF DataArray containing the meteorological data associated with each tile. You can install it on Ubuntu with the following command:
sudo apt-get install libeccodes-dev

# From project's root:
pip install poetry
poetry config --local virtualenvs.create true
poetry config --local virtualenvs.in-project true
poetry config --list
poetry install
poetry run python ./src/s2etrac/jobs/entrypoint.py --version
poetry run python ./src/s2etrac/jobs/entrypoint.py --help
```

#### Troubleshooting

##### `Failed to unlock the collection!` when running `poetry install`

See [issue https://github.com/python-poetry/poetry/issues/1917](https://github.com/python-poetry/poetry/issues/1917#issuecomment-1380429197)

Try:

```bash
PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring poetry install
```

## Usage

Use Case example:

I want to run S2ETRAC on the 'Libya 3 small' Calibration Site, which is of type 'desert'. Look for SAFE archives with a Relative Orbit Number of 7 and a Datatake Sensing Start Time of 2023-04-27. The SAFE Archives are contained in `${FOLDER_CONTAINING_SAFE_ARCHIVES}` and I want to output the result of the processing into `${OUTPUT_DIRECTORY}`.

### With Docker

The input and output folders must be mapped to folders on the host's filesystem.

Then, it can be ran with `docker run`. All parameters passed to docker will be passed to the entrypoint in `s2etrac`. Example:

```bash
# Change these variables according to your needs.

# The input and output folders must be mapped.
# The input folder contains SAFE archives.
# The output contain the products of S2ETRAC.

# Docker-related variables
CONT_NAME=s2etrac-container
IMAGE_NAME=gitlabreg.acri-cwa.fr:443/opt-mpc/s2etrac/s2etrac-standalone/s2etrac-standalone:latest

# Docker-related variables (volume mappings)
HOST_INPUT_FOLDER=/mount/dmz28/dev/projects/s2etrac/S2_L1/Dome_1/
CONT_INPUT_FOLDER=/input/
HOST_OUTPUT_PATH=/mount/dmz28/dev/projects/s2etrac/output_test/
CONT_OUTPUT_PATH=/output/

# If you brought changes to the configuration on your local file-system and want to test it
# without rebuilding the image:
HOST_CONFIG_PATH=/mount/dmz28/dev/projects/s2etrac/configuration_override/default
CONT_CONFIG_PATH=/resources/configuration/default/

# Static configuration (to enable or disable logging of memory usage)
HOST_STATIC_CONFIG_PATH=
CONT_STATIC_CONFIG_PATH=/resources/configuration/static/

# S2ETRAC-related variables (command-line options)
S2ETRAC_SITE_NAME=Dome_1
S2ETRAC_SITE_TYPE=dome
S2ETRAC_RELATIVE_ORBIT_NUMBER=101
S2ETRAC_DATE=20230304
S2ETRAC_ADDITIONAL_OPTIONS='--feature_flag output_to_json True'

# Print version only
docker run \
    --rm \
    --volume "${HOST_INPUT_FOLDER}:${CONT_INPUT_FOLDER}" \
    --volume "${HOST_OUTPUT_PATH}:${CONT_OUTPUT_PATH}" \
    --name "${CONT_NAME}" \
    --memory=128g --cpus=8 \
    "${IMAGE_NAME}" \
    --version

# Do a full run
docker run \
    --rm \
    --volume "${HOST_INPUT_FOLDER}:${CONT_INPUT_FOLDER}" \
    --volume "${HOST_OUTPUT_PATH}:${CONT_OUTPUT_PATH}" \
    --volume "${HOST_CONFIG_PATH}:${CONT_CONFIG_PATH}" \
    --name "${CONT_NAME}" \
    --memory=128g --cpus=8 \
    "${IMAGE_NAME}" \
    --input_safe_lookup_folder_path "${CONT_INPUT_FOLDER}"  \
    --output_dir_path "${CONT_OUTPUT_PATH}" \
    --site_name "${S2ETRAC_SITE_NAME}" \
    --site_type "${S2ETRAC_SITE_TYPE}" \
    --relative_orbit_number "${S2ETRAC_RELATIVE_ORBIT_NUMBER}" \
    --date "${S2ETRAC_DATE}" \
    ${S2ETRAC_ADDITIONAL_OPTIONS}
```

### Without Docker

You can run the script using the venv created by poetry:

```bash
poetry run python ./src/s2etrac/jobs/entrypoint.py \
    --input_safe_lookup_folder_path "${YOUR_FOLDER_CONTAINING_SAFE_ARCHIVES}" \
    --output_dir_path "${OUTPUT_DIRECTORY}"\
    --site_name 'Libya 3 small' \
    --site_type 'desert' \
    --relative_orbit_number 7 \
    --date 20230427
```

### Command-Line Interface Help

Here is a copy of the help of the CLI:

```
$ poetry run python ./src/s2etrac/jobs/entrypoint.py --help
Usage: entrypoint.py [OPTIONS]

  S2ETRAC Entrypoint

Options:
  --version                       Show the version and exit.
  --dry_run                       Validate input and exit.
  -n, --site_name TEXT            Site Name. Example: 'Libya 3'  [required]
  -t, --site_type [desert|dome|ocean|cloud|instrumented]
                                  Type of Site  [required]
  -s, --input_safe_path PATH      Path(s) to the input SAFE archives
  -c, --input_configuration_path PATH
                                  Input path to the application's
                                  configuration  [default:
                                  resources/configuration/default]
  -o, --output_dir_path PATH      Path to the output directory in which the
                                  NetCDF, Quicklook and Trace will be written
                                  [default: generated/entrypoint]
  -b, --band_name [B01|B02|B03|B04|B05|B06|B07|B08|B8A|B09|B10|B11|B12]
                                  Bands to include in the analysis  [default:
                                  B01, B02, B03, B04, B05, B06, B07, B08, B8A,
                                  B09, B10, B11, B12]
  -e, --feature_flag <TEXT BOOLEAN>...
                                  Boolean flags among {'msk_qualit',
                                  'debug_logs', 'output_to_netcdf',
                                  'dome_optimization_split_preprocessing',
                                  'output_to_json', 'cams', 'dask_chunk',
                                  'ecmwf', 'debug_inspect_xds',
                                  'granule_metadata', 'quicklook',
                                  'clip_all_touched', 'debug_downscale',
                                  'intermediate_quicklooks', 'clip', 'generate
                                  _performance_report_after_processing',
                                  'img_data', 'msk_detfoo', 'use_dask_client',
                                  'use_integer_channel_coordinates',
                                  'detailed_name_for_output_dir',
                                  'msk_classi', 'calibrate',
                                  'continue_if_product_is_rejected',
                                  'datastrip_metadata'}. Example: 'debug_logs
                                  True'
  -f, --input_safe_lookup_folder_path PATH
                                  Path to the folder containing SAFE archives
  -d, --date [%Y%m%d]             Datatake sensing start time. To use in
                                  conjunction with the
                                  input_safe_lookup_folder_path option. If
                                  none is provided, tries to find the latest
                                  SAFE archives.
  -r, --relative_orbit_number INTEGER RANGE
                                  Relative Orbit Number used to look for SAFE
                                  archives.  [1<=x<=143]
  --help                          Show this message and exit.
```

### Troubleshooting

#### `unrecognized engine cfgrib`

If the following Exception occurs when running S2ETRAC:

```
Exception has occurred: ValueError
unrecognized engine cfgrib must be one of: ['netcdf4', 'scipy', 'rasterio', 'store']
```

It is likely that the `eecodes` lib is not installed on your machine. You can install it, eg via

```bash
apt-get install libeccodes-dev
```

Or, you can still perform a run, but without the Meteorological Data. To do so, set the feature flags `ecmwf` and `cams` to False.

Exemple:

```bash
--feature_flag ecmwf False --feature_flag cams False
```

## Documentation

The documentation can be built and run locally:

Generate the API Reference:

```bash
poetry run sphinx-apidoc --force --separate -o docs/source/api/ src/s2etrac/
```

Run the documentation:

```bash
poetry run sphinx-autobuild -b html docs/source/ docs/build/html
```

**The documentation is accessible at [http://127.0.0.1:8000](http://127.0.0.1:8000)**

Note: in future versions, the documentation might be also published on a private URL.

## Jupyter Notebooks

Static results of Jupyter Notebooks related to the project are integrated to the documentation under [Notebooks](notebooks.html).

You can also run them yourself. Install the [Jupyter](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter) extension for VSCode.

Notebooks that must be clean and integrated into the documentation are placed under `docs/source/notebooks/presentation/`. They should always be guaranteed to work when on the main development branch.

Notebooks used to test code are located into `docs/source/notebooks/drafts/`. They are not guaranteed to work and serve as _versioned drafts_. If you want to create any notebook for testing, please put it inside this folder.

## S2ETRAC Architecture - Standalone version

The project development is split in two parts:

- The first part, the "Service" aspect of S2Etrac, is all the development required to download Sentinel-2 products ;
- The second part, the "Standalone" aspect of S2Etrac, consists of a tool that can be called both manually from the command-line, and automatically by the "Service" part.

**This presentations focuses on the "Standalone" part.**

### Data Model

![Inputs and Outputs](./docs/source/_static/assets/diagrams/s2etrac-datamodel.svg)
![Inputs and Outputs](./_static/assets/diagrams/s2etrac-datamodel.svg)

### High-Level Command Line Interface (Inputs and Outputs)

![High-Level CLI](./docs/source/_static/assets/diagrams/s2etrac-io.svg)
![High-Level CLI](./_static/assets/diagrams/s2etrac-io.svg)

### The Processor

![S2ETRAC Processor](./docs/source/_static/assets/diagrams/s2etrac-proc.svg)
![S2ETRAC Processor](./_static/assets/diagrams/s2etrac-proc.svg)

### Reproject, Clip and Merge

![Site Creation](./docs/source/_static/assets/diagrams/s2etrac-site-creation.svg)
![Site Creation](./_static/assets/diagrams/s2etrac-site-creation.svg)

### Source Code Tree

```bash
$ tree -I __pycache__ -I  __init__.py src/s2etrac/
src/s2etrac/
├── configuration
│   ├── extraction.py
│   ├── intersection.py
│   ├── managers.py
│   ├── models.py
│   └── version.py
├── core
│   ├── exceptions.py
│   ├── intersection.py
│   ├── models.py
│   ├── preprocessing
│   │   ├── common_preprocessing.py
│   │   ├── dome
│   │   │   └── dome_preprocessor.py
│   │   ├── epsg.py
│   │   ├── geo_information.py
│   │   ├── preprocessor.py
│   │   ├── quicklook_generation.py
│   │   └── raster_operations.py
│   ├── selection.py
│   ├── statistics.py
│   └── types.py
├── debug_utils.py
├── interface
│   ├── inbound
│   │   ├── execute.py
│   │   ├── load.py
│   │   ├── s2_l1c_safe
│   │   │   ├── product_datastrip_metadata.py
│   │   │   ├── product_granule_metadata.py
│   │   │   ├── product_metadata.py
│   │   │   ├── product_meteorological_information.py
│   │   │   ├── product.py
│   │   │   └── product_raster_data.py
│   │   └── validation
│   │       ├── lookup.py
│   │       └── validation.py
│   └── outbound
│       ├── data_collection.py
│       ├── formatting.py
│       └── quicklook.py
├── jobs
│   ├── entrypoint.py
│   ├── kml_acquisition_plans_to_geojson.py
│   ├── kml_tiling_grid_to_geojson.py
│   ├── load_netcdf.py
│   ├── performance_report.py
│   ├── sites_to_acquisition_plans.py
│   └── sites_to_tile_ids.py
├── logging.py
├── performance
│   ├── garbage_collection.py
│   ├── performance_monitoring.py
│   └── reporting.py
└── visualization
    ├── cartopy_functions.py
    └── main.py
```

## Development

### Integration in VSCode

Open the command palette: <kbd>Ctrl</kbd>+<kbd>Maj</kbd>+<kbd>P</kbd>

Choose `Python: Select Interpreter`
Choose the one in the `.venv` folder in project's root

#### Troubleshooting

##### `Pytest Discovery Error` in VSCode

See [StackOverflow: VSCode pytest test discovery fails](https://stackoverflow.com/questions/55837922/vscode-pytest-test-discovery-fails)

Run `poetry run pytest --collect-only`, there are probably import errors.

### Debugging

The code can be debugged using VSCode. You can store run&debug configurations inside `.vscode/launch.json`. Some personal configurations are already present and can be used as a template for yours.

ℹ️ The `justMyCode` parameter turn on or off the debugging of the libraries used in the `s2etrac` code.

### Testing

```bash
poetry run pytest
```

#### Coverage

```bash
poetry run pytest --cov=src # Coverage check, slower
```

```bash
poetry run pytest --cov=src --cov-report html
```

### Code Quality

On a regular basis, run the `scripts/clean_python_code.sh` script. It executes all the tools that improve the code quality in-place:

- pruning unused imports with `pautoflake`
- sorting imports with `isort`
- format code with `black`
- format docstrings with `docformatter`

#### Format the code with `black`

```bash
poetry run black src
```

#### Lint the code with `pylint`

```bash
poetry run pylint src
```

(pylint lite: `poetry run pylint --disable=fixme,duplicate-code,line-too-long src`)

#### Lint the code with `flake8`

```bash
poetry run flake8 src
```

#### Verify types with `mypy`

```bash
poetry run mypy src
```

#### Prune unused imports with `pautoflake`

```bash
poetry run pautoflake src
```

#### Sort unused imports with `isort`

```bash
poetry run isort src
```

### Bump version

The [poetry version](https://python-poetry.org/docs/cli/#version) subcommand is used to bump the version. No manual change of `pyproject.toml` is required.

The project tries to follows semantic versioning in the measure of the possible.

Currently, the `major` version plan is the following:

- `0` during initial development
- `1` when releasing DESERT and DOME processing
- `2` when releasing OCEAN, CLOUD and INSTRUMENTED processing

➡️ To bump: `poetry version major`

The `minor` version should be increased on a regular basis, preferentially for each milestone. This rule has not been followed consistently for now

- `0.1.0` -> Milestone #1
- `0.2.0` -> Milestone #2
- `0.3.0` -> Milestone #2

➡️ To bump: `poetry version minor`

The `patch` version should be used for smaller non-breaking changes. Since the project is not a library but an integration project, the relevance of the `patch` version is limited.

➡️ To bump: `poetry version patch`.

Then, for you local `s2etrac` installation to be updated, run `poetry install`.

Finally, update the CI (Continuous Integration) configuration by running `scripts/bump_gitlab_ci.sh`. The script reads the version of `s2etrac` that is installed on your machine and updated the GitLab configuration file accordingly

### Performance

#### Using `scalene`

Full HTML

```bash
poetry run python -m scalene --html --outfile scalene.html --profile-all ./src/s2etrac/jobs/entrypoint.py --site_name Libya\ 4\ small --site_type desert --relative_orbit_number 7 --input_safe_lookup_folder_path /home/eschalk/dev/data/Sentinel-2/S2_Test --date 20230616 --feature_flag quicklook True --feature_flag datastrip_metadata False --feature_flag continue_if_product_is_rejected True --feature_flag output_to_json True
```

Full JSON, no profile all

```bash
poetry run python -m scalene --json --outfile scalene.jsonl ./src/s2etrac/jobs/entrypoint.py --site_name Libya\ 4\ small --site_type desert --relative_orbit_number 7 --input_safe_lookup_folder_path /home/eschalk/dev/data/Sentinel-2/S2_Test --date 20230616 --feature_flag quicklook True --feature_flag datastrip_metadata False --feature_flag continue_if_product_is_rejected True --feature_flag output_to_json True
```

Ascii

```bash
poetry run python -m scalene --outfile scalene.html --profile-all ./src/s2etrac/jobs/entrypoint.py --site_name Libya\ 4\ small --site_type desert --relative_orbit_number 7 --input_safe_lookup_folder_path /home/eschalk/dev/data/Sentinel-2/S2_Test --date 20230616 --feature_flag quicklook True --feature_flag datastrip_metadata False --feature_flag continue_if_product_is_rejected True --feature_flag output_to_json True
```

## Implementation Notes

### Dimension Names

In the ATBD, dimensions are referred to their cardinality: `n_rec`, `n_det`, `n_chan`.

Here is the mapping to their dimension names in the Output NetCDF:

```py
"n_rec" -> "record",
"n_det" -> "detector",
"n_chan" -> "channel",
```

### Products are rejected when spreaded over multiple DataStrips

In the current implementation, when an acquisition is spread over multiple datastrips, which translates by having a set of `SAFE` Product URIs being identical except for their suffix, the "Product Discriminator Time", the product is not processed by S2ETRAC.

`Mauritania 2` is a site known to be concerned by this issue.

See GitHub Issue: [Handle tiles spreaded over multiple datastrips #98](https://github.com/ARGANS/s2etrac/issues/98)

### Handling of new mask names by S2ETRAC

#### Thresholds (Extraction Parameters in the ATBD)

```python
"OPAQUE" -> "CLD_max",
"CIRRUS" -> "CLD_max",
"ANC_LOST" -> "TECQUA_max",
"ANC_DEG" -> "TECQUA_max",
"MSI_LOST" -> "TECQUA_max",
"MSI_DEG" -> "TECQUA_max",
"QT_DEFECTIVE_PIXELS" -> "DEFECT_max",
"QT_NODATA_PIXELS" -> "NODATA_PIX_max",
"QT_PARTIALLY_CORRECTED_PIXELS" -> "NODATA_COR_max",
"QT_SATURATED_PIXELS_L1A" -> "SATURA_max",
```

#### Variable names in output NetCDF

The masks mentioned in the ATBD are referred by their outdate names. So, a mapping has been made to match the new names, using the Sentinel-2 documentation as a source.

The mapping is the following, from old name to set of new names.

```python
"n_opaque_cloud" -> {"n_msk_classi_opaque"},
"n_cirrus_cloud" ->  {"n_msk_classi_cirrus"},
"n_tecqua" -> {
    "n_msk_qualit_anc_lost",
    "n_msk_qualit_anc_deg",
    "n_msk_qualit_msi_lost",
    "n_msk_qualit_msi_deg",
},
"n_defect" -> {"n_msk_qualit_qt_defective_pixels"},
"n_nodata_pix" -> {"n_msk_qualit_qt_nodata_pixels"},
"n_nodata_cor" ->  {"n_msk_qualit_qt_partially_corrected_pixels"},
"n_satura" -> {"n_msk_qualit_qt_saturated_pixels_l1a"},
```

Note that the `n_tecqua` mask now maps to 4 masks.

See [Level-1C Masks](https://sentinels.copernicus.eu/web/sentinel/technical-guides/sentinel-2-msi/level-1c/masks)

### Rejection of full product if one band is invalid

Currently, the processing is interrupted when any single band is failing to match the criteria defined in the "Data Selection" paragraphs of the ATBD. Indeed, the initial requirement was that _"a product is rejected if it meets at least one of these criteria"_, the criterias being thresholds of invalid pixels _"in at least one spectral band"_

It is is planned in a future release to change this requirement, by continuing the processing on the subset of bands matching the criterias, instead of rejecting the full product.

## Links

- [Data Products](https://sentinels.copernicus.eu/web/sentinel/missions/sentinel-2/data-products)
- [Product Types](https://sentinels.copernicus.eu/web/sentinel/user-guides/sentinel-2-msi/product-types)
- [Product Types: Level 1C](https://sentinels.copernicus.eu/web/sentinel/user-guides/sentinel-2-msi/product-types/level-1c)
- [Level-1C Masks](https://sentinels.copernicus.eu/web/sentinel/technical-guides/sentinel-2-msi/level-1c/masks)
- [Cloud Masks](https://sentinel.esa.int/web/sentinel/technical-guides/sentinel-2-msi/level-1c/cloud-masks)
- [ADFS](https://sentinel.esa.int/web/sentinel/technical-guides/sentinel-2-msi/level-1/adfs)
- [Naming Convention](https://sentinels.copernicus.eu/web/sentinel/user-guides/sentinel-2-msi/naming-convention)
- [SAFE Specification](https://sentinels.copernicus.eu/web/sentinel/user-guides/sentinel-1-sar/data-formats/safe-specification)
- [Acquisition Plans](https://sentinels.copernicus.eu/web/sentinel/missions/sentinel-2/acquisition-plans)

### On "PixelIsArea" or "PixelIsPoint"

In the `rioxarray` documentation, it is stated that when a file is opened, the `x` and `y` coordinates are generated automatically, and **are shifted to the center of each pixel**. This is so that each point in the xarray grid represents the **center** of an area covered by a pixel. So, pixels are converted to a "grid".

See [`open_dataset` - rioxarray](https://corteva.github.io/rioxarray/html/rioxarray.html)

See [2.5 Coordinate Systems in GeoTIFF](http://web.archive.org/web/20160326194152/http://remotesensing.org/geotiff/spec/geotiff2.5.html#2.5.2)

The following data is coming from an example SAFE Archive.

If we look at the MTD_TL.xml (Granule Metadata), we can see a tag `Tile_Geocoding`. For all resolutions, ULX and ULY are respectively identical. This is because they represent the upper left, that is invariant for all resolutions. If we compare them to the upperleft

```xml
    <Tile_Geocoding metadataLevel="Brief">
      <HORIZONTAL_CS_NAME>WGS84 / UTM zone 33S</HORIZONTAL_CS_NAME>
      <HORIZONTAL_CS_CODE>EPSG:32733</HORIZONTAL_CS_CODE>
      <Size resolution="10">
        <NROWS>10980</NROWS>
        <NCOLS>10980</NCOLS>
      </Size>
      <Size resolution="20">
        <NROWS>5490</NROWS>
        <NCOLS>5490</NCOLS>
      </Size>
      <Size resolution="60">
        <NROWS>1830</NROWS>
        <NCOLS>1830</NCOLS>
      </Size>
      <Geoposition resolution="10">
        <ULX>499980</ULX>
        <ULY>7400020</ULY>
        <XDIM>10</XDIM>
        <YDIM>-10</YDIM>
      </Geoposition>
      <Geoposition resolution="20">
        <ULX>499980</ULX>
        <ULY>7400020</ULY>
        <XDIM>20</XDIM>
        <YDIM>-20</YDIM>
      </Geoposition>
      <Geoposition resolution="60">
        <ULX>499980</ULX>
        <ULY>7400020</ULY>
        <XDIM>60</XDIM>
        <YDIM>-60</YDIM>
      </Geoposition>
    </Tile_Geocoding>
```

First, we can see that the 'GeoTransform' attribute of the `spatial_ref` special coord of the DataArray read by rioxarray contains similar information to the `Tile_Geocoding` tag from the Granule Metadata. Note that the X axis is increasing (+) whereas the Y axis is decreasing (-)

```python
print(json.dumps(({resolution: f"{product.raster_data['img_data'].spatial_ref.attrs['GeoTransform']}" for products in products_split_by_resolution for resolution, product in products.items() }), indent=4, sort_keys=True))
{
    "10": "600000.0 10.0 0.0 4900020.0 0.0 -10.0",
    "20": "600000.0 20.0 0.0 4900020.0 0.0 -20.0",
    "60": "600000.0 60.0 0.0 4900020.0 0.0 -60.0"
}
```

Let's compare to the corners of the rasters for each resolution that were read by `rioxarray`:

```python
print(json.dumps(({resolution: f"{product.raster_data['img_data'].x[0].item()}" for products in products_split_by_resolution for resolution, product in products.items() }), indent=4, sort_keys=True))
{
    "10": "600005.0",
    "20": "600010.0",
    "60": "600030.0"
}

print(json.dumps(({resolution: f"{product.raster_data['img_data'].y[0].item()}" for products in products_split_by_resolution for resolution, product in products.items() }), indent=4, sort_keys=True))
{
    "10": "4900015.0",
    "20": "4900010.0",
    "60": "4899990.0"
}
```

However, we can see the "shift" that was operated by `rioxarray` when loading the coordinates. Summary:

|                              | X (+)  | Y (-)   |
| ---------------------------- | ------ | ------- |
| "PixelIsPoint" Corner        | 600000 | 4900020 |
| "PixelIsArea", resolution=10 | 600005 | 4900015 |
| "PixelIsArea", resolution=20 | 600010 | 4900010 |
| "PixelIsArea", resolution=60 | 600030 | 4899990 |

We can see that the `[0, 0]` coordinate in "PixelIsArea" is:

- `ULX + resolution // 2` for the increasing X-axis,
- `ULY - resolution // 2` for the decreasing Y-axis.

All DataArrays (raster data, geo data, meteo data) are to be considered using the "PixelIsArea" convention when loaded by `rioxarray`.

## Misc

### Emojis

| Emoji | Site Type    |
| ----- | ------------ |
| 🏜️    | Desert       |
| ❄️    | Dome         |
| 🌊    | Ocean        |
| ☁️    | Cloud        |
| 🔭    | Instrumented |