Repository: CS-SI/eodag

Issue:
Earth Search: Missing ProductPath in some downloaded product
Bug linked to #423. 
Sorry to reopen again this issue, but I have encountered new products where it fails 😅
I am recreating a new issue to be more precise.

**Describe the bug**

Some products are missing a ProductPath in their properties, leading to failure to find the manifest file
`FileNotFoundError: No manifest.safe could be found in D:\_EODOWNLOAD\OUTPUT\GRAND_EST\L2A\S2B_MSIL2A_20211229T130249_N0301_R095_T24MUV_20211229T151639`
![2022-03-16_09h17_21](https://user-images.githubusercontent.com/67311115/158546345-4ce55c5e-d57d-4c8e-afff-25029c837625.png)

This comes from the fact that the ProductPath is missing and its url is replaced by the product location in `get_bucket_name_and_prefix` (in `aws.py`). This leads to a wrong name and bucket:

```python
print(
    self.get_bucket_name_and_prefix(
        product, product.properties["productPath"]
    )
)
    
('v0', '/collections/sentinel-s2-l2a/items/S2A_32TLT_20211016_0_L2A')
```

**Code To Reproduce**
```py
import os
import tempfile

from eodag import EODataAccessGateway, setup_logging
from eodag.config import override_config_from_file

from eodownload import dir_utils
from eodownload.dir_utils import get_data_directory

OUTPUT = r"D:\"

os.environ["EODAG__EARTH_SEARCH__DOWNLOAD__EXTRACT"] = "false"
os.environ["EODAG__EARTH_SEARCH__DOWNLOAD__OUTPUTS_PREFIX"] = OUTPUT
os.environ["EODAG__EARTH_SEARCH__AUTH__CREDENTIALS__AWS_ACCESS_KEY_ID"] = "*****"
os.environ["EODAG__EARTH_SEARCH__AUTH__CREDENTIALS__AWS_SECRET_ACCESS_KEY"] = "*****"
os.environ["EODAG__EARTH_SEARCH__AUTH__CREDENTIALS__AWS_PROFILE"] = "*****"

if __name__ == "__main__":
    setup_logging(2)  # DEBUG level

    with tempfile.TemporaryDirectory() as tmp:
        # Create gateway
        dag = EODataAccessGateway()
        dag.set_preferred_provider("earth_search")

        # Query all
        products = dag.search_all(
            start="2021-12-28",
            end="2021-12-30",
            **{
                "productType": "S2_MSI_L2A",
                "geom": "POINT (-40.30881686219417 -5.018666560087038)"
            }
        )

        # Download all
        dag.download_all(products)
```

```
2022-06-30 15:46:48,821 eodag.config                     [INFO    ] Loading user configuration from: C:\Users\rbraun\.config\eodag\eodag.yml
2022-06-30 15:46:48,830 eodag.config                     [INFO    ] aws_s3_sentinel2_l1c: unknown provider found in user conf, trying to use provided configuration
2022-06-30 15:46:48,830 eodag.config                     [WARNING ] aws_s3_sentinel2_l1c skipped: could not be loaded from user configuration
2022-06-30 15:46:48,965 eodag.core                       [INFO    ] usgs: provider needing auth for search has been pruned because no crendentials could be found
2022-06-30 15:46:48,965 eodag.core                       [INFO    ] aws_eos: provider needing auth for search has been pruned because no crendentials could be found
2022-06-30 15:46:49,114 eodag.core                       [INFO    ] Locations configuration loaded from C:\Users\rbraun\.config\eodag\locations.yml
2022-06-30 15:46:49,763 eodag.core                       [INFO    ] Searching product type 'S2_MSI_L2A' on provider: earth_search
2022-06-30 15:46:49,763 eodag.core                       [INFO    ] Iterate search over multiple pages: page #1
2022-06-30 15:46:49,856 eodag.plugins.search.qssearch    [INFO    ] Sending search request: https://earth-search.aws.element84.com/v0/search
2022-06-30 15:46:50,392 eodag.core                       [INFO    ] Found 1 result(s) on provider 'earth_search'
2022-06-30 15:46:50,392 eodag.core                       [INFO    ] Downloading 1 products
Downloaded products:   0%|          | 0/1 [00:00<?, ?product/s]
0.00B [00:00, ?B/s]
S2B_24MUV_20211229_1_L2A: 0.00B [00:00, ?B/s]2022-06-30 15:46:50,394 eodag.plugins.download.base      [INFO    ] Download url: https://earth-search.aws.element84.com/v0/collections/sentinel-s2-l2a/items/S2B_24MUV_20211229_1_L2A
2022-06-30 15:46:50,394 eodag.plugins.download.aws       [ERROR   ] complementary_url_key productPath is missing in S2B_24MUV_20211229_1_L2A properties
S2B_24MUV_20211229_1_L2A: 100%|██████████| 407M/407M [00:27<00:00, 8.57MB/s]2022-06-30 15:47:22,499 eodag.plugins.download.aws       
[ERROR   ] Could not finalize SAFE product from downloaded data
Traceback (most recent call last):
  File "C:\Users\rbraun\Anaconda3\envs\eodownload\lib\site-packages\eodag\plugins\download\aws.py", line 657, in finalize_s2_safe_product
    f"No manifest.safe could be found in {product_path}"
FileNotFoundError: No manifest.safe could be found in D:\_EODOWNLOAD\OUTPUT\GRAND_EST\L2A\S2B_MSIL2A_20211229T130249_N0301_R095_T24MUV_20211229T151639
2022-06-30 15:47:22,500 eodag.plugins.download.base      [WARNING ] A problem occurred during download of product: EOProduct(id=S2B_24MUV_20211229_1_L2A, provider=earth_search). Skipping it
S2B_24MUV_20211229_1_L2A: 100%|██████████| 407M/407M [00:27<00:00, 14.6MB/s]
2022-06-30 15:47:22,503 eodag.plugins.download.base      [INFO    ] [Retry #1, 0/1 D/L] Waiting 87s until next download try (retry every 2' for 20')
```

**Output**
Compete output obtained with maximal verbosity.

**Environment:**
 - Python version: Python 3.7.12
 - EODAG version: eodag (Earth Observation Data Access Gateway): version 2.5.1


Discussion:
github-actions[bot]: <strong> </strong>

| File | Coverage |   |
| - | :-: | :-: |
| **All files** | `80%` | :white_check_mark: |

<strong> </strong>

| File | Coverage |   |
| - | :-: | :-: |
| **All files** | `79%` | :white_check_mark: |

_Minimum allowed coverage is `70%`_

<p align="right">Generated by :monkey: cobertura-action against 3daa8ff24c4f12d0f64328f8d266f223eb4f0f1f </p>
github-actions[bot]: ## Unit Test Results
    2 files  ±0      2 suites  ±0   10m 19s [:stopwatch:](https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.20/README.md#the-symbols "duration of all tests") -46s
253 tests ±0  251 [:heavy_check_mark:](https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.20/README.md#the-symbols "passed tests") ±0  2 [:zzz:](https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.20/README.md#the-symbols "skipped / disabled tests") ±0  0 [:x:](https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.20/README.md#the-symbols "failed tests") ±0 
506 runs  ±0  502 [:heavy_check_mark:](https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.20/README.md#the-symbols "passed tests") ±0  4 [:zzz:](https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.20/README.md#the-symbols "skipped / disabled tests") ±0  0 [:x:](https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.20/README.md#the-symbols "failed tests") ±0 

Results for commit 59ba44f9. ± Comparison against base commit 54a51822.

[test-results]:data:application/gzip;base64,H4sIAAhAxGIC/02M0Q7DEBRAf6XxvAdcpPYzC0oia2tRnpb9+0yxPp5zkvNGzq/2QPeJ3iZ0ZJ8GLDmq5MNeUBBZREmpRg6dHkc2piryV0//aotTOOXXIvAQNsYQm4l5/z05Fg36kmM6zHlknS/DytefCdvmUwHEpVaMOak0CK1nIjDWQIXGEsDZmVK7MKDKoM8Xu2tuoQUBAAA=


Patch:
diff --git a/CHANGES.rst b/CHANGES.rst
index 964bb480..5762d991 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -1,6 +1,11 @@
 Release history
 ---------------
 
+2.5.2 (2022-07-05)
+++++++++++++++++++
+
+* Fixes missing ``productPath`` property for some ``earth_search`` products (:pull:`480`)
+
 2.5.1 (2022-06-27)
 ++++++++++++++++++
 
diff --git a/eodag/resources/providers.yml b/eodag/resources/providers.yml
index 1bd6eb37..d9080757 100644
--- a/eodag/resources/providers.yml
+++ b/eodag/resources/providers.yml
@@ -1725,7 +1725,7 @@
         platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
         polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
         productPath: |
-          $.properties."sentinel:product_id".`sub(/([S2AB]{3})_MSIL1C_([0-9]{4})([0-9]{2})([0-9]{2})(T.*)/, products/\\2/\\3/\\4/\\1_MSIL1C_\\2\\3\\4\\5)`.`sub(/\/0+/, \/)`
+          $.properties."sentinel:product_id".`sub(/([S2AB]{3})_MSIL1C_([0-9]{4})([0-9]{2})([0-9]{2})(T.*)/, products!\\2!\\3!\\4!\\1_MSIL1C_\\2\\3\\4\\5)`.`sub(/!0*/, /)`
         tilePath: |
           $.assets.info.href.`sub(/.*/sentinel-s2-l1c\/(tiles\/.*)\/tileInfo\.json/, \\1)`
     S2_MSI_L2A:
@@ -1735,7 +1735,7 @@
         platformSerialIdentifier: '$.id.`split(_, 0, -1)`'
         polarizationMode: '$.id.`sub(/.{14}([A-Z]{2}).*/, \\1)`'
         productPath: |
-          $.properties."sentinel:product_id".`sub(/([S2AB]{3})_MSIL2A_([0-9]{4})([0-9]{2})([0-9]{2})(T.*)/, products/\\2/\\3/\\4/\\1_MSIL2A_\\2\\3\\4\\5)`.`sub(/\/0+/, \/)`
+          $.properties."sentinel:product_id".`sub(/([S2AB]{3})_MSIL2A_([0-9]{4})([0-9]{2})([0-9]{2})(T.*)/, products!\\2!\\3!\\4!\\1_MSIL2A_\\2\\3\\4\\5)`.`sub(/!0*/, /)`
         tilePath: |
           $.assets.info.href.`sub(/.*/sentinel-s2-l2a\/(tiles\/.*)\/tileInfo\.json/, \\1)`
     L8_OLI_TIRS_C1L1:
diff --git a/pyproject.toml b/pyproject.toml
index f002f4b6..730b048d 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -3,4 +3,4 @@ requires = ["setuptools>=45", "setuptools_scm[toml]>=6.2,<7"]
 build-backend = "setuptools.build_meta"
 
 [tool.setuptools_scm]
-fallback_version = "2.5.2.dev0"
+fallback_version = "2.5.3.dev0"


Test patch:
diff --git a/tests/units/test_search_plugins.py b/tests/units/test_search_plugins.py
index 636c92e3..f8e0f7d7 100644
--- a/tests/units/test_search_plugins.py
+++ b/tests/units/test_search_plugins.py
@@ -327,6 +327,13 @@ class TestSearchPluginStacSearch(BaseSearchPluginTest):
                             "sentinel:product_id": "S2B_MSIL1C_20200910T012345_N0209_R008_T31TCJ_20200910T123456",
                         },
                     },
+                    {
+                        "id": "bar",
+                        "geometry": geojson_geometry,
+                        "properties": {
+                            "sentinel:product_id": "S2B_MSIL1C_20201010T012345_N0209_R008_T31TCJ_20201010T123456",
+                        },
+                    },
                 ]
             },
         ]
@@ -344,3 +351,7 @@ class TestSearchPluginStacSearch(BaseSearchPluginTest):
             products[1].properties["productPath"],
             "products/2020/9/10/S2B_MSIL1C_20200910T012345_N0209_R008_T31TCJ_20200910T123456",
         )
+        self.assertEqual(
+            products[2].properties["productPath"],
+            "products/2020/10/10/S2B_MSIL1C_20201010T012345_N0209_R008_T31TCJ_20201010T123456",
+        )