Repository: CS-SI/eodag
Pull request #1301 (2024-08-19 07:48:43)

fix(providers): handle cop_marine in situ historical data
`cop_marine` provider fixes:
- if no date is given in the id of the product, the date given for the dataset the product belongs to is now used
- a mapping has been added to retrieve other information from the id (such as platform, analysis type) and to add it to the properties of the product


github-actions[bot]: ## Test Results
    4 files  ± 0      4 suites  ±0   6m 10s :stopwatch: +7s
  562 tests + 1    379 :white_check_mark: + 3   2 :zzz: ±0  50 :x:  - 2  131 :fire: ± 0 
2 107 runs  +58  1 710 :white_check_mark: +49  98 :zzz: ±0  63 :x:  - 2  236 :fire: +11 

For more details on these failures and errors, see [this check](https://github.com/CS-SI/eodag/runs/28931566905).

Results for commit 89945c43. ± Comparison against base commit 19486454.

[test-results]:data:application/gzip;base64,H4sIANX5wmYC/0WMSw6DMAwFr4Ky7sL5x71MRQxIUflUgayq3r2mJbCc8fO8xZDGfhX3xtwasZa0ndCV3G5pmRm1BxZ82vajdarSYy1EvwFe6plerK7N0KZxfzsbjz7nJbOSWrLLZd6zSoI/qGall3CqfxZDFUfW6SpqVWnHipZpShujCIjGktExOIzK62BjB5YIUfdBEQDEaJBAfL41Q0aJDgEAAA==

github-actions[bot]: ![badge](https://raw.githubusercontent.com/CS-SI/eodag/_xml_coverage_reports/data/1287-handle-cop_marine-in-situ-historical-data/./badge.svg)


## Code Coverage (Ubuntu)

<details>


```
Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         312      70  77.56%   551-571, 655-714, 816-867, 871
config.py                                      367      32  91.28%   82-84, 93, 101, 105-107, 175, 186, 452-454, 514-517, 560-561, 570-571, 676, 733-744, 746, 751
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    772     179  76.81%   615, 635, 679-682, 720, 760, 764, 798, 801-820, 847, 851-856, 882, 947-951, 973, 985, 995, 1041, 1065, 1179, 1262-1274, 1310, 1312, 1338-1340, 1344-1387, 1445-1513, 1536-1609, 1623-1630, 1684-1694, 1702, 1716, 1725, 1771, 1839, 1868, 1914-1915, 1944, 1956, 1975-1981, 1993-2002, 2063-2064, 2076-2078, 2232, 2236-2239, 2250-2252, 2284
api/search_result.py                            51       4  92.16%   73, 82, 89, 103
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          48      16  66.67%   66-81, 147, 155, 158-162
api/product/_product.py                        187      20  89.30%   70-72, 237-238, 313, 342, 399, 413-416, 429, 453-456, 499-505
api/product/metadata_mapping.py                675      88  86.96%   129-131, 226, 258-259, 305-306, 316-328, 330, 341, 347-359, 404-405, 432-434, 442, 456, 463-466, 489, 497-498, 574-575, 599-600, 606-609, 624-625, 774, 820, 947-948, 991-996, 1123, 1137-1157, 1177, 1182, 1186, 1289, 1311, 1325, 1338-1357, 1396, 1448, 1480-1484, 1499
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      6       1  83.33%   38
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 21       2  90.48%   48, 55
plugins/manager.py                             130      21  83.85%   104-109, 127-131, 159, 197, 219, 223, 242-247, 281-282, 308-309
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           91      63  30.77%   88, 99-125, 134-153, 165-242, 257, 269
plugins/apis/usgs.py                           180     121  32.78%   103-112, 132, 146, 157-161, 179-203, 211-246, 261-270, 283-439, 454
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              19       0  100.00%
plugins/authentication/base.py                  17       2  88.24%   34, 47
plugins/authentication/generic.py               14       2  85.71%   40, 50
plugins/authentication/header.py                19       2  89.47%   106-107
plugins/authentication/keycloak.py              48       4  91.67%   134, 160-165
plugins/authentication/oauth.py                 13       7  46.15%   32-34, 38-41
plugins/authentication/openid_connect.py       187      17  90.91%   119, 133-158, 166, 320-323, 349
plugins/authentication/qsauth.py                34       1  97.06%   83
plugins/authentication/sas_auth.py              47       1  97.87%   76
plugins/authentication/token.py                 89      16  82.02%   79, 107, 109, 132-145, 201-205
plugins/authentication/token_exchange.py        36      20  44.44%   74-80, 92-122
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       1  90.00%   40
plugins/crunch/filter_date.py                   59      14  76.27%   49-54, 66, 75, 84, 87, 99-101, 110-112, 119
plugins/crunch/filter_latest_intersect.py       47       8  82.98%   51-52, 68, 77-80, 82, 89-92
plugins/crunch/filter_latest_tpl_name.py        31       2  93.55%   49, 83
plugins/crunch/filter_overlap.py                66      18  72.73%   28-30, 68-71, 78-81, 87, 95, 106-122
plugins/crunch/filter_property.py               30       7  76.67%   54-59, 62-63, 79-83
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        489     163  66.67%   265, 278, 345-348, 362-366, 408-410, 414, 446-447, 453-457, 486, 518, 522, 529, 559-567, 571, 603-611, 618-620, 654-728, 746-807, 818-823, 835-848, 873, 888-890, 893, 903-911, 919-932, 942-973, 980-992, 1030, 1056, 1101-1103, 1323
plugins/download/base.py                       253      97  61.66%   136, 164, 192-195, 216, 231-233, 296-297, 314-320, 351-355, 361-362, 404, 407-421, 433, 437, 501-505, 524-532, 535-536, 544-586, 623-684
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       534     421  21.16%   151-210, 221-247, 270-537, 555, 558-561, 585-601, 623, 633-665, 670-678, 694, 708, 717-728, 757-821, 828-839, 851-881, 903-904, 912, 918, 931-936, 942, 945, 960-965, 978-979, 1008-1120, 1138-1240, 1246-1262, 1271-1336
plugins/download/s3rest.py                     116      97  16.38%   85-86, 112-334
plugins/search/__init__.py                      22       0  100.00%
plugins/search/base.py                         128      20  84.38%   102, 106, 117, 225, 240-245, 255, 274-275, 289, 331-332, 352, 355-363
plugins/search/build_search_result.py          181      27  85.08%   95, 118, 136-137, 143, 154, 189, 208, 215, 288-291, 320, 373-390, 450, 453, 463, 480, 508, 510
plugins/search/cop_marine.py                   234      49  79.06%   55, 63-65, 75-76, 81, 86-87, 103, 105, 108, 143-145, 157-158, 204, 208, 212, 225, 236-237, 245, 273, 277, 292, 296, 300, 304, 308-312, 318-321, 324-338, 355, 404-408, 413, 425
plugins/search/creodias_s3.py                   55       3  94.55%   53, 71, 105
plugins/search/csw.py                          105      81  22.86%   58-59, 63-64, 72-120, 126-139, 147-179, 197-238
plugins/search/data_request_search.py          200      67  66.50%   90-93, 109, 120, 124-125, 136, 141, 146, 153, 166-169, 223-224, 228, 238-244, 249, 275-278, 286-297, 314, 316, 323-324, 326-327, 345-349, 382, 392, 403, 416, 422-437, 442
plugins/search/qssearch.py                     682     110  83.87%   379, 388, 392-398, 406-407, 513-525, 569, 585, 595, 614-629, 666-669, 740-741, 789, 808, 815, 827, 884, 905, 908-909, 918-919, 928-929, 938-939, 966, 1037-1042, 1046-1055, 1089, 1111, 1126, 1171, 1205-1232, 1238, 1294-1298, 1358, 1361, 1367-1368, 1389, 1416-1428, 1435, 1467-1469, 1479-1485, 1515, 1538, 1553, 1575, 1658-1661, 1666-1669, 1696-1697, 1712
plugins/search/static_stac_search.py            72      10  86.11%   98-125, 141, 154
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33       7  78.79%   35-37, 53-55, 59, 68
rest/config.py                                  26       0  100.00%
rest/constants.py                                7       0  100.00%
rest/core.py                                   234      40  82.91%   110-115, 127-130, 269, 355, 456, 644, 653, 659-667, 671-672, 679-727
rest/server.py                                 283      53  81.27%   108, 131-133, 246-248, 304-305, 317-333, 425-430, 458, 623-630, 659, 703-704, 727, 798-800, 817-822, 851, 853, 857-858, 862-863
rest/stac.py                                   473     121  74.42%   318, 340, 392-395, 422-449, 454-460, 480-482, 505, 537-538, 566, 579, 620-660, 682-698, 772-785, 792, 846-847, 908, 973-975, 1179, 1187-1199, 1210-1232, 1243-1288, 1447-1448
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     179       5  97.21%   232-236, 289, 292, 360
rest/types/queryables.py                        56       1  98.21%   164
rest/types/stac_search.py                      126       7  94.44%   129, 175, 190-192, 200, 204
rest/utils/__init__.py                          93      16  82.80%   106-109, 128-130, 154, 167, 182, 192-206
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       3  86.96%   48, 60, 62
types/__init__.py                              114      14  87.72%   53, 70, 129-132, 199, 213-222, 232, 253, 266
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              500      57  88.60%   85, 90, 194-195, 204-231, 234, 248, 328-332, 406-410, 429-431, 510, 525, 561-562, 868, 926-929, 937-938, 976-977, 1074-1077, 1090-1091, 1098-1109, 1148, 1381-1383
utils/constraints.py                           119      37  68.91%   89-98, 139, 144, 148, 159, 182-184, 194, 208-224, 233-244
utils/exceptions.py                             35       1  97.14%   93
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                28       1  96.43%   41
utils/notebook.py                               44      25  43.18%   25-29, 36-41, 58-62, 69-78, 83-87
utils/repr.py                                   30       8  73.33%   36, 38, 42, 76, 94-101
utils/requests.py                               55      11  80.00%   64, 88, 90, 92, 94, 96, 112, 120-122, 130
utils/rest.py                                   36      20  44.44%   44-63, 69-80
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 137, 153-156, 203-212, 222-252
TOTAL                                         9645    2447  74.63%
```

</details>


### Diff against develop

```
Filename                                    Stmts    Miss  Cover
----------------------------------------  -------  ------  -------
config.py                                       0      -6  +1.63%
api/core.py                                     0     -23  +2.98%
plugins/authentication/keycloak.py              0     -10  +20.84%
plugins/authentication/openid_connect.py        0      -9  +4.81%
plugins/download/aws.py                         0    -134  +27.41%
plugins/download/base.py                        0     +18  -7.11%
plugins/download/creodias_s3.py                 0      -1  +5.88%
plugins/download/s3rest.py                      0     +73  -62.93%
plugins/search/cop_marine.py                  +37      +2  +2.92%
utils/notebook.py                               0      +2  -4.55%
TOTAL                                         +37     -88  +1.01%
```


Results for commit: 89945c43b869b27385bd05cc993e82c000bb49c0


_Minimum allowed coverage is `70%`_


:recycle: This comment has been updated with latest results


<!-- Sticky Pull Request Commentartifacts/unit-test-results-python3.12-ubuntu-latest/coverage.xml -->

Patch:
diff --git a/eodag/plugins/search/cop_marine.py b/eodag/plugins/search/cop_marine.py
index 9b67b526..99afaab0 100644
--- a/eodag/plugins/search/cop_marine.py
+++ b/eodag/plugins/search/cop_marine.py
@@ -67,6 +67,21 @@ def _get_date_from_yyyymmdd(date_str: str, item_key: str) -> Optional[datetime]:
         return date
 
 
+def _get_dates_from_dataset_data(
+    dataset_item: Dict[str, Any]
+) -> Optional[Dict[str, str]]:
+    dates = {}
+    if "start_datetime" in dataset_item["properties"]:
+        dates["start"] = dataset_item["properties"]["start_datetime"]
+        dates["end"] = dataset_item["properties"]["end_datetime"]
+    elif "datetime" in dataset_item["properties"]:
+        dates["start"] = dataset_item["properties"]["datetime"]
+        dates["end"] = dataset_item["properties"]["datetime"]
+    else:
+        return None
+    return dates
+
+
 def _get_s3_client(endpoint_url: str) -> S3Client:
     s3_session = boto3.Session()
     return s3_session.client(
@@ -180,20 +195,16 @@ class CopMarineSearch(StaticStacSearch):
             "dataset": dataset_item["id"],
         }
         if use_dataset_dates:
-            if "start_datetime" in dataset_item:
-                properties["startTimeFromAscendingNode"] = dataset_item[
-                    "start_datetime"
-                ]
-                properties["completionTimeFromAscendingNode"] = dataset_item[
-                    "end_datetime"
-                ]
-            elif "datetime" in dataset_item:
-                properties["startTimeFromAscendingNode"] = dataset_item["datetime"]
-                properties["completionTimeFromAscendingNode"] = dataset_item["datetime"]
+            dates = _get_dates_from_dataset_data(dataset_item)
+            if not dates:
+                return None
+            properties["startTimeFromAscendingNode"] = dates["start"]
+            properties["completionTimeFromAscendingNode"] = dates["end"]
         else:
-            item_dates = re.findall(r"\d{8}", item_key)
+            item_dates = re.findall(r"(\d{4})(0[1-9]|1[0-2])([0-3]\d)", item_id)
             if not item_dates:
-                item_dates = re.findall(r"\d{6}", item_key)
+                item_dates = re.findall(r"_(\d{4})(0[1-9]|1[0-2])", item_id)
+            item_dates = ["".join(row) for row in item_dates]
             item_start = _get_date_from_yyyymmdd(item_dates[0], item_key)
             if not item_start:  # identified pattern was not a valid datetime
                 return None
@@ -209,11 +220,26 @@ class CopMarineSearch(StaticStacSearch):
             ).strftime("%Y-%m-%dT%H:%M:%SZ")
 
         for key, value in collection_dict["properties"].items():
-            if key not in ["id", "title", "start_datetime", "end_datetime"]:
+            if key not in ["id", "title", "start_datetime", "end_datetime", "datetime"]:
                 properties[key] = value
         for key, value in dataset_item["properties"].items():
-            if key not in ["id", "title", "start_datetime", "end_datetime"]:
+            if key not in ["id", "title", "start_datetime", "end_datetime", "datetime"]:
                 properties[key] = value
+
+        code_mapping = self.config.products.get(product_type, {}).get(
+            "code_mapping", None
+        )
+        if code_mapping:
+            id_parts = item_id.split("_")
+            if len(id_parts) > code_mapping["index"]:
+                code = id_parts[code_mapping["index"]]
+                if "pattern" not in code_mapping:
+                    properties[code_mapping["param"]] = code
+                elif re.findall(code_mapping["pattern"], code):
+                    properties[code_mapping["param"]] = re.findall(
+                        code_mapping["pattern"], code
+                    )[0]
+
         _check_int_values_properties(properties)
 
         properties["thumbnail"] = collection_dict["assets"]["thumbnail"]["href"]
@@ -348,16 +374,54 @@ class CopMarineSearch(StaticStacSearch):
 
                 for obj in s3_objects["Contents"]:
                     item_key = obj["Key"]
+                    item_id = item_key.split("/")[-1].split(".")[0]
                     # filter according to date(s) in item id
-                    item_dates = re.findall(r"\d{8}", item_key)
+                    item_dates = re.findall(r"(\d{4})(0[1-9]|1[0-2])([0-3]\d)", item_id)
                     if not item_dates:
-                        item_dates = re.findall(r"\d{6}", item_key)
-                    item_start = _get_date_from_yyyymmdd(item_dates[0], item_key)
-                    if not item_start:  # identified pattern was not a valid datetime
+                        item_dates = re.findall(r"_(\d{4})(0[1-9]|1[0-2])", item_id)
+                    item_dates = [
+                        "".join(row) for row in item_dates
+                    ]  # join tuples returned by findall
+                    item_start = None
+                    item_end = None
+                    use_dataset_dates = False
+                    if item_dates:
+                        item_start = _get_date_from_yyyymmdd(item_dates[0], item_key)
+                        if len(item_dates) > 2:  # start, end and created_at timestamps
+                            item_end = _get_date_from_yyyymmdd(item_dates[1], item_key)
+                    if not item_start:
+                        # no valid datetime given in id
+                        use_dataset_dates = True
+                        dates = _get_dates_from_dataset_data(dataset_item)
+                        if dates:
+                            item_start_str = dates["start"].replace("Z", "+0000")
+                            item_end_str = dates["end"].replace("Z", "+0000")
+                            try:
+                                item_start = datetime.strptime(
+                                    item_start_str, "%Y-%m-%dT%H:%M:%S.%f%z"
+                                )
+                                item_end = datetime.strptime(
+                                    item_end_str, "%Y-%m-%dT%H:%M:%S.%f%z"
+                                )
+                            except ValueError:
+                                item_start = datetime.strptime(
+                                    item_start_str, "%Y-%m-%dT%H:%M:%S%z"
+                                )
+                                item_end = datetime.strptime(
+                                    item_end_str, "%Y-%m-%dT%H:%M:%S%z"
+                                )
+                    if not item_start:
+                        # no valid datetime in id and dataset data
                         continue
                     if item_start > end_date:
                         stop_search = True
-                    if not item_dates or (start_date <= item_start <= end_date):
+                    if (
+                        (start_date <= item_start <= end_date)
+                        or (item_end and start_date <= item_end <= end_date)
+                        or (
+                            item_end and item_start < start_date and item_end > end_date
+                        )
+                    ):
                         num_total += 1
                         if num_total < start_index:
                             continue
@@ -368,6 +432,7 @@ class CopMarineSearch(StaticStacSearch):
                                 endpoint_url + "/" + bucket,
                                 dataset_item,
                                 collection_dict,
+                                use_dataset_dates,
                             )
                             if product:
                                 products.append(product)
diff --git a/eodag/resources/product_types.yml b/eodag/resources/product_types.yml
index b6f181fe..b54de976 100644
--- a/eodag/resources/product_types.yml
+++ b/eodag/resources/product_types.yml
@@ -5518,7 +5518,7 @@ MO_OCEANCOLOUR_GLO_BGC_L3_MY_009_107:
   platformSerialIdentifier:
   processingLevel: Level 3
   keywords: CMEMS,Mercator,ocean,global,L3,bio-geo-chemical,BGC,chlorophyll,phytoplankton,reflectance
-  sensorType:
+  sensorType: multi
   license: proprietary
   title: Global Ocean Colour Plankton and Reflectances MY L3 daily observations
   missionStartDate: "1997-09-04T00:00:00Z"
@@ -5622,7 +5622,7 @@ MO_OCEANCOLOUR_GLO_BGC_L4_MY_009_108:
   platformSerialIdentifier:
   processingLevel: L4
   keywords: CMEMS,Mercator,ocean,global,colour,L4,bio-geo-chemical,BGC,chlorophyll,MY,multi-years,monthly
-  sensorType:
+  sensorType: multi
   license: proprietary
   title: Global Ocean Colour Plankton MY L4 monthly observations
   missionStartDate: "1997-09-01T00:00:00Z"
diff --git a/eodag/resources/providers.yml b/eodag/resources/providers.yml
index 0730e000..add36bfd 100644
--- a/eodag/resources/providers.yml
+++ b/eodag/resources/providers.yml
@@ -6909,6 +6909,29 @@
   roles:
     - host
   url: https://data.eumetsat.int
+  anchor_sentinel: &sentinel_params
+    orbitDirection:
+      - orbitdir
+      - '$.properties.acquisitionInformation[0].acquisitionParameters.orbitDirection'
+    relativeOrbitNumber:
+      - relorbit
+      - '$.properties.acquisitionInformation[0].acquisitionParameters.relativeOrbitNumber'
+    timeliness:
+      - timeliness
+      - '$.properties.productInformation.timeliness'
+    cycleNumber:
+      - cycle
+      - '$.properties.acquisitionInformation[0].acquisitionParameters.cycleNumber'
+  anchor_orbit_zone_tile: &orbit_zone_tile
+    orbitNumber:
+      - orbit
+      - '$.properties.acquisitionInformation[0].acquisitionParameters.orbitNumber'
+    utmZone:
+      - zone
+      - '$.null'
+    tileIdentifier:
+      - t6
+      - '$.null'
   search: !plugin
     type: QueryStringSearch
     api_endpoint: 'https://api.eumetsat.int/data/search-products/1.0.0/os'
@@ -6970,7 +6993,6 @@
         - sat
         - '$.properties.acquisitionInformation[0].platform.platformShortName'
       instrument: '$.properties.acquisitionInformation[0].instrument.instrumentShortName'
-
       # INSPIRE obligated OpenSearch Parameters for Collection Search (Table 4)
       title:
         - title
@@ -6978,19 +7000,11 @@
       publicationDate:
         - publication
         - '$.null'
-
       # OpenSearch Parameters for Product Search (Table 5)
       parentIdentifier:
         - pi
         - '$.properties.parentIdentifier'
-      orbitNumber:
-        - orbit
-        - '$.properties.acquisitionInformation[0].acquisitionParameters.orbitNumber'
-      orbitDirection:
-        - orbitdir
-        - '$.properties.acquisitionInformation[0].acquisitionParameters.orbitDirection'
       modificationDate: '$.properties.updated'
-
       # OpenSearch Parameters for Acquistion Parameters Search (Table 6)
       startTimeFromAscendingNode:
         - dtstart
@@ -7002,12 +7016,6 @@
       id:
         - id
         - $.properties.identifier
-      utmZone:
-        - zone
-        - '$.null'
-      tileIdentifier:
-        - t6
-        - '$.null'
       # The geographic extent of the product
       geometry:
         - 'geo={geometry#to_rounded_wkt}'
@@ -7021,25 +7029,11 @@
       # storageStatus set to ONLINE for consistency between providers
       storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
       assets: '{$.properties.links.sip-entries#assets_list_to_dict}'
-
       # Additional metadata provided by the providers but that don't appear in the reference spec
-      timeliness:
-        - timeliness
-        - '$.properties.productInformation.timeliness'
-      relativeOrbitNumber:
-        - relorbit
-        - '$.properties.acquisitionInformation[0].acquisitionParameters.relativeOrbitNumber'
-      cycleNumber:
-        - cycle
-        - '$.properties.acquisitionInformation[0].acquisitionParameters.cycleNumber'
-      fire:
-        - fire
-        - '$.properties.extraInformation.fireDetected'
       size:
         - size
         - '$.properties.productInformation.size'
       type: '$.null'
-
       # set duplicate metadata due to metadata discovery to null
       acquisitionInformation: '$.null'
       productInformation: '$.null'
@@ -7049,113 +7043,223 @@
     S3_SRA:
       productType: SR_1_SRA___
       parentIdentifier: EO:EUM:DAT:0406
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_SRA_A:
       productType: SR_1_SRA_A_
       parentIdentifier: EO:EUM:DAT:0413
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_SRA_1A_BC004:
       productType: SR_1_SRA_A_
       parentIdentifier: EO:EUM:DAT:0583
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_SRA_1A_BC005:
       productType: SR_1_SRA_A_
       parentIdentifier: EO:EUM:DAT:0836
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_SRA_1B_BC004:
       productType: SR_1_SRA___
       parentIdentifier: EO:EUM:DAT:0584
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_SRA_1B_BC005:
       productType: SR_1_SRA___
       parentIdentifier: EO:EUM:DAT:0833
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_SRA_BS:
       productType: SR_1_SRA_BS
       parentIdentifier: EO:EUM:DAT:0414
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_SRA_BS_BC004:
       productType: SR_1_SRA_BS
       parentIdentifier: EO:EUM:DAT:0585
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_SRA_BS_BC005:
       productType: SR_1_SRA_BS
       parentIdentifier: EO:EUM:DAT:0835
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_WAT:
       productType: SR_2_WAT___
       parentIdentifier: EO:EUM:DAT:0415
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_WAT_BC004:
       productType: SR_2_WAT___
       parentIdentifier: EO:EUM:DAT:0586
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_WAT_BC005:
       productType: SR_2_WAT___
       parentIdentifier: EO:EUM:DAT:0834
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     # S3 OLCI
     S3_EFR:
       productType: OL_1_EFR___
       parentIdentifier: EO:EUM:DAT:0409
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_EFR_BC002:
       productType: OL_1_EFR___
       parentIdentifier: EO:EUM:DAT:0577
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_ERR:
       productType: OL_1_ERR___
       parentIdentifier: EO:EUM:DAT:0410
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_ERR_BC002:
       productType: OL_1_ERR___
       parentIdentifier: EO:EUM:DAT:0578
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_OLCI_L2WRR:
       productType: OL_2_WRR___
       parentIdentifier: EO:EUM:DAT:0408
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_OLCI_L2WRR_BC003:
       productType: OL_2_WRR___
       parentIdentifier: EO:EUM:DAT:0557
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_OLCI_L2WFR:
       productType: OL_2_WFR___
       parentIdentifier: EO:EUM:DAT:0407
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_OLCI_L2WFR_BC003:
       productType: OL_2_WFR___
       parentIdentifier: EO:EUM:DAT:0556
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     # S3 SLSTR
     S3_SLSTR_L1RBT:
       productType: SL_1_RBT___
       parentIdentifier: EO:EUM:DAT:0411
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_SLSTR_L1RBT_BC003:
       productType: SL_1_RBT___
       parentIdentifier: EO:EUM:DAT:0581
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_SLSTR_L1RBT_BC004:
       productType: SL_1_RBT___
       parentIdentifier: EO:EUM:DAT:0615
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_SLSTR_L2WST:
       productType: SL_2_WST___
       parentIdentifier: EO:EUM:DAT:0412
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_SLSTR_L2WST_BC003:
       productType: SL_2_WST___
       parentIdentifier: EO:EUM:DAT:0582
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_SLSTR_L2AOD:
       productType: SL_2_AOD___
       parentIdentifier: EO:EUM:DAT:0416
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
     S3_SLSTR_L2FRP:
       productType: SL_2_FRP___
       parentIdentifier: EO:EUM:DAT:0417
+      metadata_mapping:
+        <<: *orbit_zone_tile
+        <<: *sentinel_params
+        fire:
+          - fire
+          - '$.properties.extraInformation.fireDetected'
     # METOP
     METOP_AMSU_L1:
       parentIdentifier: EO:EUM:DAT:METOP:AMSUL1
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_OSI_104:
       parentIdentifier: EO:EUM:DAT:METOP:OSI-104
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_OSI_150A:
       parentIdentifier: EO:EUM:DAT:METOP:OSI-150-A
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_OSI_150B:
       parentIdentifier: EO:EUM:DAT:METOP:OSI-150-B
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_ASCSZF1B:
       parentIdentifier: EO:EUM:DAT:METOP:ASCSZF1B
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_ASCSZR1B:
       parentIdentifier: EO:EUM:DAT:METOP:ASCSZR1B
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_ASCSZO1B:
       parentIdentifier: EO:EUM:DAT:METOP:ASCSZO1B
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_ASCSZFR02:
       parentIdentifier: EO:EUM:CM:METOP:ASCSZFR02
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_ASCSZOR02:
       parentIdentifier: EO:EUM:CM:METOP:ASCSZOR02
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_ASCSZRR02:
       parentIdentifier: EO:EUM:CM:METOP:ASCSZRR02
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_AVHRRL1:
       parentIdentifier: EO:EUM:DAT:METOP:AVHRRL1
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_SOMO12:
       parentIdentifier: EO:EUM:DAT:METOP:SOMO12
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_SOMO25:
       parentIdentifier: EO:EUM:DAT:METOP:SOMO25
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_AVHRRGACR02:
       parentIdentifier: EO:EUM:DAT:0558
     METOP_LSA_002:
@@ -7164,18 +7268,32 @@
       parentIdentifier: EO:EUM:DAT:METOP:GLB-SST-NC
     METOP_GOMEL1:
       parentIdentifier: EO:EUM:DAT:METOP:GOMEL1
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_GOMEL1R03:
       parentIdentifier: EO:EUM:DAT:0533
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_IASTHR011:
       parentIdentifier: EO:EUM:DAT:0576
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_IASSND02:
       parentIdentifier: EO:EUM:DAT:METOP:IASSND02
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_IASIL1C_ALL:
       parentIdentifier: EO:EUM:DAT:METOP:IASIL1C-ALL
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_MHSL1:
       parentIdentifier: EO:EUM:DAT:METOP:MHSL1
+      metadata_mapping:
+        <<: *orbit_zone_tile
     METOP_HIRSL1:
       parentIdentifier: EO:EUM:DAT:MULT:HIRSL1
+      metadata_mapping:
+        <<: *orbit_zone_tile
     GENERIC_PRODUCT_TYPE:
       productType: '{productType}'
       parentIdentifier: '{parentIdentifier}'
@@ -7249,10 +7367,19 @@
       productType: GLOBAL_MULTIYEAR_PHY_ENS_001_031
     MO_INSITU_GLO_PHY_UV_DISCRETE_NRT_013_048:
       productType: INSITU_GLO_PHY_UV_DISCRETE_NRT_013_048
+      code_mapping:
+        param: platformSerialIdentifier
+        index: 3
     MO_INSITU_GLO_PHY_TS_OA_NRT_013_002:
       productType: INSITU_GLO_PHY_TS_OA_NRT_013_002
+      code_mapping:
+        param: analysis_name
+        index: 1
     MO_INSITU_GLO_PHY_TS_OA_MY_013_052:
       productType: INSITU_GLO_PHY_TS_OA_MY_013_052
+      code_mapping:
+        param: analysis_name
+        index: 1
     MO_MULTIOBS_GLO_BIO_BGC_3D_REP_015_010:
       productType: MULTIOBS_GLO_BIO_BGC_3D_REP_015_010
     MO_MULTIOBS_GLO_BIO_CARBON_SURFACE_REP_015_008:
@@ -7279,6 +7406,10 @@
       productType: SEALEVEL_GLO_PHY_MDT_008_063
     MO_SST_GLO_SST_L3S_NRT_OBSERVATIONS_010_010:
       productType: SST_GLO_SST_L3S_NRT_OBSERVATIONS_010_010
+      code_mapping:
+        param: sensorType
+        index: 3
+        pattern: '[A-Z]{3}'
     MO_SST_GLO_SST_L4_NRT_OBSERVATIONS_010_001:
       productType: SST_GLO_SST_L4_NRT_OBSERVATIONS_010_001
     MO_SST_GLO_SST_L4_REP_OBSERVATIONS_010_011:
@@ -7287,8 +7418,15 @@
       productType: SST_GLO_SST_L4_REP_OBSERVATIONS_010_024
     MO_WAVE_GLO_WAV_L3_SPC_NRT_OBSERVATIONS_014_002:
       productType: WAVE_GLO_WAV_L3_SPC_NRT_OBSERVATIONS_014_002
+      code_mapping:
+        param: platformSerialIdentifier
+        index: 0
+        pattern: 's1a|s1b|cfo'
     MO_WAVE_GLO_PHY_SWH_L3_NRT_014_001:
       productType: WAVE_GLO_PHY_SWH_L3_NRT_014_001
+      code_mapping:
+        param: platformSerialIdentifier
+        index: 4
     MO_WAVE_GLO_PHY_SWH_L4_NRT_014_003:
       productType: WAVE_GLO_PHY_SWH_L4_NRT_014_003
     MO_WIND_GLO_PHY_CLIMATE_L4_MY_012_003:
@@ -7305,12 +7443,28 @@
       productType: OCEANCOLOUR_GLO_BGC_L3_MY_009_107
     MO_OCEANCOLOUR_GLO_BGC_L3_NRT_009_101:
       productType: OCEANCOLOUR_GLO_BGC_L3_NRT_009_101
+      code_mapping:
+        param: sensorType
+        index: 6
+        pattern: 'olci|gapfree-multi|multi-climatology|multi'
     MO_OCEANCOLOUR_GLO_BGC_L3_MY_009_103:
       productType: OCEANCOLOUR_GLO_BGC_L3_MY_009_103
+      code_mapping:
+        param: sensorType
+        index: 6
+        pattern: 'olci|gapfree-multi|multi-climatology|multi'
     MO_OCEANCOLOUR_GLO_BGC_L4_NRT_009_102:
       productType: OCEANCOLOUR_GLO_BGC_L4_NRT_009_102
+      code_mapping:
+        param: sensorType
+        index: 6
+        pattern: 'olci|gapfree-multi|multi-climatology|multi'
     MO_OCEANCOLOUR_GLO_BGC_L4_MY_009_104:
       productType: OCEANCOLOUR_GLO_BGC_L4_MY_009_104
+      code_mapping:
+        param: sensorType
+        index: 6
+        pattern: 'olci|gapfree-multi|multi-climatology|multi'
     MO_OCEANCOLOUR_GLO_BGC_L4_MY_009_108:
       productType: OCEANCOLOUR_GLO_BGC_L4_MY_009_108
     GENERIC_PRODUCT_TYPE:
diff --git a/eodag/rest/core.py b/eodag/rest/core.py
index d4f22785..c509594c 100644
--- a/eodag/rest/core.py
+++ b/eodag/rest/core.py
@@ -33,6 +33,7 @@ from requests.models import Response as RequestsResponse
 import eodag
 from eodag import EOProduct
 from eodag.api.product.metadata_mapping import (
+    DEFAULT_METADATA_MAPPING,
     NOT_AVAILABLE,
     OFFLINE_STATUS,
     ONLINE_STATUS,
@@ -172,7 +173,6 @@ def search_stac_items(
         raise ValidationError(format_pydantic_error(e)) from e
 
     catalog_url = re.sub("/items.*", "", request.state.url)
-
     catalog = StacCatalog(
         url=(
             catalog_url
@@ -206,6 +206,16 @@ def search_stac_items(
             **catalog.search_args,
             **eodag_args.model_dump(exclude_none=True),
         }
+        # remove provider prefixes
+        stac_extensions = stac_config["extensions"]
+        keys_to_update = {}
+        for key in criteria:
+            if ":" in key and key.split(":")[0] not in stac_extensions:
+                new_key = key.split(":")[1]
+                keys_to_update[key] = new_key
+        for key, new_key in keys_to_update.items():
+            criteria[new_key] = criteria[key]
+            criteria.pop(key)
 
         search_results = eodag_api.search(count=True, **criteria)
         total = search_results.number_matched or 0
@@ -605,8 +615,18 @@ async def get_queryables(
         stac_queryables: Dict[str, StacQueryableProperty] = deepcopy(
             StacQueryables.default_properties
         )
+        # get stac default properties to set prefixes
+        stac_item_properties = list(stac_config["item"]["properties"].values())
+        stac_item_properties.extend(list(stac_queryables.keys()))
+        ignore = stac_config["metadata_ignore"]
+        stac_item_properties.extend(ignore)
+        default_mapping = DEFAULT_METADATA_MAPPING.keys()
         for param, queryable in python_queryables.items():
-            stac_param = EODAGSearch.to_stac(param)
+            if param in default_mapping and not any(
+                param in str(prop) for prop in stac_item_properties
+            ):
+                param = f"oseo:{param}"
+            stac_param = EODAGSearch.to_stac(param, stac_item_properties, provider)
             # only keep "datetime" queryable for dates
             if stac_param in stac_queryables or stac_param in (
                 "start_datetime",
diff --git a/eodag/rest/server.py b/eodag/rest/server.py
index a5085330..4bbc3143 100755
--- a/eodag/rest/server.py
+++ b/eodag/rest/server.py
@@ -36,6 +36,7 @@ from typing import (
 
 from fastapi import APIRouter as FastAPIRouter
 from fastapi import FastAPI, HTTPException, Request
+from fastapi.concurrency import run_in_threadpool
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.openapi.utils import get_openapi
 from fastapi.responses import ORJSONResponse
@@ -864,10 +865,12 @@ async def post_search(request: Request) -> ORJSONResponse:
 
     logger.debug("Body: %s", search_request.model_dump(exclude_none=True))
 
-    response = search_stac_items(
-        request=request,
-        search_request=search_request,
+    response = await run_in_threadpool(
+        search_stac_items,
+        request,
+        search_request,
     )
+
     return ORJSONResponse(content=response, media_type="application/json")
 
 
diff --git a/eodag/rest/types/eodag_search.py b/eodag/rest/types/eodag_search.py
index 38089cbf..3d546961 100644
--- a/eodag/rest/types/eodag_search.py
+++ b/eodag/rest/types/eodag_search.py
@@ -367,9 +367,21 @@ class EODAGSearch(BaseModel):
         return cls._to_eodag_map.get(value, value)
 
     @classmethod
-    def to_stac(cls, field_name: str) -> str:
+    def to_stac(
+        cls,
+        field_name: str,
+        stac_item_properties: List[str] = None,
+        provider: str = None,
+    ) -> str:
         """Get the alias of a field in a Pydantic model"""
         field = cls.model_fields.get(field_name)
         if field is not None and field.alias is not None:
             return field.alias
+        if (
+            provider
+            and ":" not in field_name
+            and stac_item_properties
+            and field_name not in stac_item_properties
+        ):
+            return f"{provider}:{field_name}"
         return field_name


Test patch:
diff --git a/tests/units/test_http_server.py b/tests/units/test_http_server.py
index 638a375a..a5fef56e 100644
--- a/tests/units/test_http_server.py
+++ b/tests/units/test_http_server.py
@@ -1017,6 +1017,41 @@ class RequestTestCase(unittest.TestCase):
             ],
         )
 
+    @mock.patch("eodag.rest.core.eodag_api.search", autospec=True)
+    def test_provider_prefix_post_search(self, mock_search):
+        """provider prefixes should be removed from query parameters"""
+        post_data = {
+            "collections": ["ERA5_SL"],
+            "provider": "cop_cds",
+            "query": {
+                "cop_cds:month": {"eq": "10"},
+                "cop_cds:year": {"eq": "2010"},
+                "cop_cds:day": {"eq": "10"},
+            },
+        }
+        mock_search.return_value = SearchResult.from_geojson(
+            {"features": [], "type": "FeatureCollection"}
+        )
+        self.app.request(
+            method="POST",
+            url="search",
+            json=post_data,
+            follow_redirects=True,
+            headers={"Content-Type": "application/json"},
+        )
+        expected_search_kwargs = dict(
+            productType="ERA5_SL",
+            page=1,
+            items_per_page=DEFAULT_ITEMS_PER_PAGE,
+            month="10",
+            year="2010",
+            day="10",
+            raise_errors=True,
+            count=True,
+            provider="cop_cds",
+        )
+        mock_search.assert_called_once_with(**expected_search_kwargs)
+
     def test_search_response_contains_pagination_info(self):
         """Responses to valid search requests must return a geojson with pagination info in properties"""
         response = self._request_valid(f"search?collections={self.tested_product_type}")
@@ -1730,11 +1765,14 @@ class RequestTestCase(unittest.TestCase):
         # queryables properties not shared by all constraints must be removed
         not_shared_properties = ["leadtime_hour", "type"]
         provider_queryables_from_constraints_file = [
-            properties
+            f"cop_cds:{properties}"
             for properties in provider_queryables_from_constraints_file
             if properties not in not_shared_properties
         ]
-        default_provider_stac_properties = ["api_product_type", "format"]
+        default_provider_stac_properties = [
+            "cop_cds:api_product_type",
+            "cop_cds:format",
+        ]
 
         res = self._request_valid(
             "collections/ERA5_SL/queryables?provider=cop_cds",
diff --git a/tests/units/test_search_plugins.py b/tests/units/test_search_plugins.py
index 477d319a..53fccc6b 100644
--- a/tests/units/test_search_plugins.py
+++ b/tests/units/test_search_plugins.py
@@ -2694,6 +2694,16 @@ class TestSearchPluginCopMarineSearch(BaseSearchPluginTest):
                 },
             ]
         }
+        self.list_objects_response3 = {
+            "Contents": [
+                {
+                    "Key": "native/PRODUCT_A/dataset-number-two/item_15325642_fizncnqijei.nc"
+                },
+                {
+                    "Key": "native/PRODUCT_A/dataset-number-two/item_846282_niznjvnqkrf.nc"
+                },
+            ]
+        }
         self.s3 = boto3.client(
             "s3",
             config=botocore.config.Config(
@@ -2771,6 +2781,86 @@ class TestSearchPluginCopMarineSearch(BaseSearchPluginTest):
                 products_dataset2[0].properties["completionTimeFromAscendingNode"],
             )
 
+    @mock.patch("eodag.plugins.search.cop_marine.requests.get")
+    def test_plugins_search_cop_marine_query_no_dates_in_id(self, mock_requests_get):
+        mock_requests_get.return_value.json.side_effect = [
+            self.product_data,
+            self.dataset1_data,
+            self.dataset2_data,
+        ]
+
+        search_plugin = self.get_search_plugin("PRODUCT_A", self.provider)
+        search_plugin.config.products = {
+            "PRODUCT_A": {
+                "productType": "PRODUCT_A",
+                "code_mapping": {"param": "platformSerialIdentifier", "index": 1},
+            }
+        }
+
+        with mock.patch("eodag.plugins.search.cop_marine._get_s3_client") as s3_stub:
+            s3_stub.return_value = self.s3
+            stubber = Stubber(self.s3)
+            stubber.add_response(
+                "list_objects",
+                self.list_objects_response1,
+                {"Bucket": "bucket1", "Prefix": "native/PRODUCT_A/dataset-number-one"},
+            )
+            stubber.add_response(
+                "list_objects",
+                self.list_objects_response3,
+                {"Bucket": "bucket1", "Prefix": "native/PRODUCT_A/dataset-number-two"},
+            )
+            stubber.add_response(
+                "list_objects",
+                {},
+                {
+                    "Bucket": "bucket1",
+                    "Prefix": "native/PRODUCT_A/dataset-number-two",
+                    "Marker": "native/PRODUCT_A/dataset-number-two/item_846282_niznjvnqkrf.nc",
+                },
+            )
+            stubber.activate()
+            result, num_total = search_plugin.query(
+                productType="PRODUCT_A",
+                startTimeFromAscendingNode="1969-01-01T01:00:00Z",
+                completionTimeFromAscendingNode="1970-02-01T01:00:00Z",
+            )
+            mock_requests_get.assert_has_calls(
+                calls=[
+                    call(
+                        "https://stac.marine.copernicus.eu/metadata/PRODUCT_A/product.stac.json"
+                    ),
+                    call().json(),
+                    call(
+                        "https://stac.marine.copernicus.eu/metadata/PRODUCT_A/dataset-number-one/dataset.stac.json"
+                    ),
+                    call().json(),
+                    call(
+                        "https://stac.marine.copernicus.eu/metadata/PRODUCT_A/dataset-number-two/dataset.stac.json"
+                    ),
+                    call().json(),
+                ]
+            )
+            self.assertEqual(2, num_total)
+            products_dataset2 = [
+                product
+                for product in result
+                if product.properties["dataset"] == "dataset-number-two"
+            ]
+            self.assertEqual(2, len(products_dataset2))
+            self.assertEqual(
+                "1970-01-01T00:00:00.000000Z",
+                products_dataset2[0].properties["startTimeFromAscendingNode"],
+            )
+            self.assertEqual(
+                "1970-01-01T00:00:00.000000Z",
+                products_dataset2[0].properties["completionTimeFromAscendingNode"],
+            )
+            self.assertEqual(
+                "15325642",
+                products_dataset2[0].properties["platformSerialIdentifier"],
+            )
+
     @mock.patch("eodag.plugins.search.cop_marine.requests.get")
     def test_plugins_search_cop_marine_query_with_id(self, mock_requests_get):
         mock_requests_get.return_value.json.side_effect = [