Repository: CS-SI/eodag
Pull request #639 (2023-02-09 17:12:30)

fix: restore creodias product types discovery
Following #623, restores creodias product types discovery using creodias STAC API

github-actions[bot]: ## Unit Test Results
    2 files  ±0      2 suites  ±0   5m 6s [:stopwatch:](https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.20/README.md#the-symbols "duration of all tests") -6s
359 tests ±0  356 [:heavy_check_mark:](https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.20/README.md#the-symbols "passed tests") ±0  3 [:zzz:](https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.20/README.md#the-symbols "skipped / disabled tests") ±0  0 [:x:](https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.20/README.md#the-symbols "failed tests") ±0 
718 runs  ±0  712 [:heavy_check_mark:](https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.20/README.md#the-symbols "passed tests") ±0  6 [:zzz:](https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.20/README.md#the-symbols "skipped / disabled tests") ±0  0 [:x:](https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.20/README.md#the-symbols "failed tests") ±0 

Results for commit ff5752ac. ± Comparison against base commit bc21b08f.

[test-results]:data:application/gzip;base64,H4sIAOcq5WMC/03MSw7CIBSF4a00jB0AlUtxM4YiNyG2xfAYGfcu1IId/t9JzpugW2wkt4FfBhKzSz0eOejk/FZypFCgTKmOo1Ct7jEbsxP86elelTqgdksB2sGG4MMhIW/1U7LpiHYpGe/ye4TWp8O9z3/Gr6tLJQiikIJrY+XEBABSa8Q4M8CZconsikrZupPPFyrggHEFAQAA


Patch:
diff --git a/eodag/resources/providers.yml b/eodag/resources/providers.yml
index dea45452..88d0217e 100644
--- a/eodag/resources/providers.yml
+++ b/eodag/resources/providers.yml
@@ -897,6 +897,7 @@
         wrapper: '{}'
         operations:
           and:
+            - "Collection/Name eq '{collection}'"
             - "OData.CSC.Intersects(area=geography'{geometry#to_ewkt}')"
             - "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq '{productType}')"
             - "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'platformShortName' and att/OData.CSC.StringAttribute/Value eq '{platform}')"
@@ -922,6 +923,23 @@
                 -  "Attributes/OData.CSC.StringAttribute/any(att:att/Name eq '{metadata}' and att/OData.CSC.StringAttribute/Value eq '{{{metadata}}}')"
       metadata_path: '$.Attributes.*'
     common_metadata_mapping_path: '$'
+    discover_product_types:
+      fetch_url: 'https://datahub.creodias.eu/stac/collections'
+      result_type: json
+      results_entry: 'collections[*]'
+      generic_product_type_id: '$.id'
+      generic_product_type_parsable_properties:
+        collection: '$.id'
+      generic_product_type_parsable_metadata:
+        abstract: '$.description'
+        instrument: '{$.summaries.instruments#csv_list}'
+        platform: '{$.summaries.constellation#csv_list}'
+        platformSerialIdentifier: '{$.summaries.platform#csv_list}'
+        processingLevel: '$.summaries."processing:level"'
+        keywords: '{$.keywords#csv_list}'
+        license: '$.license'
+        title: '$.title'
+        missionStartDate: '$.extent.temporal.interval[0][0]'
     per_product_metadata_query: false
     metadata_pre_mapping:
       metadata_path: '$.Attributes'
@@ -997,6 +1015,9 @@
       downloadLink: 'https://datahub.creodias.eu/odata/v1/Products({uid})/$value'
       # storageStatus: must be one of ONLINE, STAGING, OFFLINE
       storageStatus: '{$.Online#get_group_name((?P<ONLINE>True)|(?P<OFFLINE>False))}'
+      collection:
+        - null
+        - $.null
       # Additional metadata provided by the providers but that don't appear in the reference spec
       productIdentifier: '$.S3Path'
   download: !plugin
@@ -1017,109 +1038,109 @@
   products:
     # S2
     S2_MSI_L1C:
-      platform: SENTINEL-2
+      collection: SENTINEL-2
       productType: S2MSI1C
     S2_MSI_L2A:
-      platform: SENTINEL-2
+      collection: SENTINEL-2
       productType: S2MSI2A
     # S1
     S1_SAR_RAW:
       productType: RAW
-      platform: SENTINEL-1
+      collection: SENTINEL-1
       metadata_mapping:
         cloudCover: '$.null'
     S1_SAR_GRD:
       productType: GRD
-      platform: SENTINEL-1
+      collection: SENTINEL-1
       metadata_mapping:
         cloudCover: '$.null'
     S1_SAR_SLC:
       productType: SLC
-      platform: SENTINEL-1
+      collection: SENTINEL-1
       metadata_mapping:
         cloudCover: '$.null'
     S1_SAR_OCN:
       productType: OCN
-      platform: SENTINEL-1
+      collection: SENTINEL-1
       metadata_mapping:
         cloudCover: '$.null'
     # S3 SRAL
     S3_SRA:
       productType: SR_1_SRA___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
       metadata_mapping:
         cloudCover: '$.null'
     S3_SRA_A:
       productType: SR_1_SRA_A_
-      platform: SENTINEL-3
+      collection: SENTINEL-3
       metadata_mapping:
         cloudCover: '$.null'
     S3_SRA_BS:
       productType: SR_1_SRA_BS
-      platform: SENTINEL-3
+      collection: SENTINEL-3
       metadata_mapping:
         cloudCover: '$.null'
     S3_LAN:
       productType: SR_2_LAN___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
       metadata_mapping:
         cloudCover: '$.null'
     S3_WAT:
       productType: SR_2_WAT___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
       metadata_mapping:
         cloudCover: '$.null'
     # S3 OLCI
     S3_EFR:
       productType: OL_1_EFR___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     S3_ERR:
       productType: OL_1_ERR___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     S3_OLCI_L2LRR:
       productType: OL_2_LRR___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     S3_OLCI_L2LFR:
       productType: OL_2_LFR___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     S3_OLCI_L2WRR:
       productType: OL_2_WRR___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     S3_OLCI_L2WFR:
       productType: OL_2_WFR___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     # S3 SLSTR
     S3_SLSTR_L1RBT:
       productType: SL_1_RBT___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     S3_SLSTR_L2LST:
       productType: SL_2_LST___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     S3_SLSTR_L2WST:
       productType: SL_2_WST___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     S3_SLSTR_L2AOD:
       productType: SL_2_AOD___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     S3_SLSTR_L2FRP:
       productType: SL_2_FRP___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     # S3 SY
     S3_SY_AOD:
       productType: SY_2_AOD___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     S3_SY_SYN:
       productType: SY_2_SYN___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     S3_SY_V10:
       productType: SY_2_V10___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     S3_SY_VG1:
       productType: SY_2_VG1___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     S3_SY_VGP:
       productType: SY_2_VGP___
-      platform: SENTINEL-3
+      collection: SENTINEL-3
     GENERIC_PRODUCT_TYPE:
       productType: '{productType}'
       collection: '{collection}'


Test patch:
diff --git a/tests/test_end_to_end.py b/tests/test_end_to_end.py
index 23de8247..14111227 100644
--- a/tests/test_end_to_end.py
+++ b/tests/test_end_to_end.py
@@ -28,6 +28,8 @@ import unittest
 from pathlib import Path
 from tempfile import TemporaryDirectory
 
+import pytest
+
 from eodag.api.product.metadata_mapping import ONLINE_STATUS
 from tests import TEST_RESOURCES_PATH
 from tests.context import (
@@ -186,6 +188,7 @@ METEOBLUE_SEARCH_ARGS = [
 ]
 
 
+@pytest.mark.enable_socket
 class EndToEndBase(unittest.TestCase):
     def execute_search(
         self,
@@ -530,14 +533,14 @@ class TestEODagEndToEnd(EndToEndBase):
         provider = "creodias"
         ext_product_types_conf = self.eodag.discover_product_types(provider=provider)
         self.assertEqual(
-            "Sentinel1",
-            ext_product_types_conf[provider]["providers_config"]["Sentinel1"][
+            "SENTINEL-1",
+            ext_product_types_conf[provider]["providers_config"]["SENTINEL-1"][
                 "collection"
             ],
         )
         self.assertEqual(
-            "Sentinel-1",
-            ext_product_types_conf[provider]["product_types_config"]["Sentinel1"][
+            "SENTINEL-1",
+            ext_product_types_conf[provider]["product_types_config"]["SENTINEL-1"][
                 "title"
             ],
         )