Repository: ESMValGroup/ESMValCore

Issue:
The `extract_location` preprocessor doesn't work at the Met Office :(
**Describe the bug**

A new ESMValTool user at the Met Office was following the [quick start guide](https://esmvalgroup.github.io/ESMValTool_Tutorial/01-quickstart/index.html) and using our new ESMValTool v2.8.0 environment, but the recipe raised the exception:
```
geopy.exc.GeocoderUnavailable: HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443):
Max retries exceeded with url: /search?q=Amsterdam&format=json&limit=1 
(Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: 
self-signed certificate in certificate chain (_ssl.c:997)')))
```
The recipe works fine with our ESMValTool v2.7.0 environment.

I have been investigating this for a while; here are the key points:

+ I did not see any problems when running this recipe on JASMIN.
+ In ESMValTool v2.8.0, the [esmvaltool/recipes/examples/recipe_python.yml](https://github.com/ESMValGroup/ESMValTool/compare/v2.7.0...v2.8.0#diff-9ede07c7320e4272382fe41064146c3f8583e358e1cd14e5aaa67a7884c8047d) recipe was updated to use the `extract_location` preprocessor.
+ The `extract_location` preprocessor is (as far as I can tell) the only function in the entire code base that uses `geopy`.
+ I can reproduce the problem by running the code [within the `extract_location` function](https://github.com/ESMValGroup/ESMValCore/blob/9406b7eb7828a8d1a1328697158e4e7805b0a3c6/esmvalcore/preprocessor/_regrid.py#L363) in an interactive Python session:
```
% python
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:08:06) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from geopy.geocoders import Nominatim
>>> geolocator = Nominatim(user_agent="test_emma")
>>> geolocation = geolocator.geocode("Amsterdam")
Traceback (most recent call last):
  [...]
geopy.exc.GeocoderUnavailable: HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443):
Max retries exceeded with url: /search?q=Amsterdam&format=json&limit=1
(Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:
self-signed certificate in certificate chain (_ssl.c:997)')))
```
+ The problem ultimately is the fault of `geopy`, since  [`requests.Session().trust_env` is hard-coded in `geopy` to be equal to `False`](https://github.com/geopy/geopy/blob/2.3.0/geopy/adapters.py#L397), which prevents our SSL certificates from being used (see [Requests Documentation: trust_env](https://docs.python-requests.org/en/latest/api/#requests.Session.trust_env) for slightly more information). 
+ This can be overridden as follows:
```
% python
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:08:06) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from geopy.geocoders import Nominatim
>>> import ssl
>>> ssl_context = ssl.create_default_context()
>>> geolocator = Nominatim(user_agent="test_emma", ssl_context=ssl_context)
>>> geolocation = geolocator.geocode("Amsterdam")
>>> print(geolocation)
Amsterdam, Noord-Holland, Nederland
>>> 
```
Would it be possible to implement this in ESMValCore, please? I'm not sure whether [ssl.create_default_context()](https://docs.python.org/3/library/ssl.html#ssl.create_default_context) would cause issues for others (I can't think of a reason why it would?), but I would be happy to create a PR for testing purposes?

Discussion:
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/ESMValGroup/ESMValCore/pull/2023?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ESMValGroup) Report
> Merging [#2023](https://app.codecov.io/gh/ESMValGroup/ESMValCore/pull/2023?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ESMValGroup) (24a70f6) into [main](https://app.codecov.io/gh/ESMValGroup/ESMValCore/commit/664f7c909f4ab6ce4d03c6c4e6d431d6ba70b154?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ESMValGroup) (664f7c9) will **increase** coverage by `0.00%`.
> The diff coverage is `100.00%`.

```diff
@@           Coverage Diff           @@
##             main    #2023   +/-   ##
=======================================
  Coverage   92.78%   92.78%           
=======================================
  Files         236      236           
  Lines       12455    12457    +2     
=======================================
+ Hits        11556    11558    +2     
  Misses        899      899           
```


| [Impacted Files](https://app.codecov.io/gh/ESMValGroup/ESMValCore/pull/2023?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ESMValGroup) | Coverage Δ | |
|---|---|---|
| [esmvalcore/preprocessor/\_regrid.py](https://app.codecov.io/gh/ESMValGroup/ESMValCore/pull/2023?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ESMValGroup#diff-ZXNtdmFsY29yZS9wcmVwcm9jZXNzb3IvX3JlZ3JpZC5weQ==) | `98.11% <100.00%> (+0.01%)` | :arrow_up: |



Patch:
diff --git a/esmvalcore/preprocessor/_regrid.py b/esmvalcore/preprocessor/_regrid.py
index 302e0c86b..a14abfbe6 100644
--- a/esmvalcore/preprocessor/_regrid.py
+++ b/esmvalcore/preprocessor/_regrid.py
@@ -5,6 +5,7 @@ import inspect
 import logging
 import os
 import re
+import ssl
 from copy import deepcopy
 from decimal import Decimal
 from pathlib import Path
@@ -360,15 +361,26 @@ def extract_location(cube, location, scheme):
     if scheme is None:
         raise ValueError("Interpolation scheme needs to be specified."
                          " Use either 'linear' or 'nearest'.")
-    geolocator = Nominatim(user_agent='esmvalcore')
+    try:
+        # Try to use the default SSL context, see
+        # https://github.com/ESMValGroup/ESMValCore/issues/2012 for more
+        # information.
+        ssl_context = ssl.create_default_context()
+        geolocator = Nominatim(user_agent='esmvalcore',
+                               ssl_context=ssl_context)
+    except ssl.SSLError:
+        logger.warning(
+            "ssl.create_default_context() encountered a problem, not using it."
+        )
+        geolocator = Nominatim(user_agent='esmvalcore')
     geolocation = geolocator.geocode(location)
     if geolocation is None:
         raise ValueError(f'Requested location {location} can not be found.')
     logger.info("Extracting data for %s (%s °N, %s °E)", geolocation,
                 geolocation.latitude, geolocation.longitude)
 
-    return extract_point(cube, geolocation.latitude,
-                         geolocation.longitude, scheme)
+    return extract_point(cube, geolocation.latitude, geolocation.longitude,
+                         scheme)
 
 
 def extract_point(cube, latitude, longitude, scheme):
@@ -998,13 +1010,11 @@ def get_cmor_levels(cmor_table, coordinate):
     """
     if cmor_table not in CMOR_TABLES:
         raise ValueError(
-            f"Level definition cmor_table '{cmor_table}' not available"
-        )
+            f"Level definition cmor_table '{cmor_table}' not available")
 
     if coordinate not in CMOR_TABLES[cmor_table].coords:
         raise ValueError(
-            f'Coordinate {coordinate} not available for {cmor_table}'
-        )
+            f'Coordinate {coordinate} not available for {cmor_table}')
 
     cmor = CMOR_TABLES[cmor_table].coords[coordinate]
 
@@ -1015,8 +1025,7 @@ def get_cmor_levels(cmor_table, coordinate):
 
     raise ValueError(
         f'Coordinate {coordinate} in {cmor_table} does not have requested '
-        f'values'
-    )
+        f'values')
 
 
 def get_reference_levels(dataset):


Test patch:
diff --git a/tests/integration/preprocessor/_regrid/test_extract_location.py b/tests/integration/preprocessor/_regrid/test_extract_location.py
index 5e63db83f..15c1ffb41 100644
--- a/tests/integration/preprocessor/_regrid/test_extract_location.py
+++ b/tests/integration/preprocessor/_regrid/test_extract_location.py
@@ -1,4 +1,6 @@
 """Integration tests for :func:`esmvalcore.preprocessor.extract_location."""
+import ssl
+from unittest.mock import patch
 
 import iris
 import iris.fileformats
@@ -14,17 +16,13 @@ def mocked_geopy_geocoders_nominatim(mocker):
     """Mock :class:`geopy.geocoders.Nominatim`.
 
     See https://github.com/ESMValGroup/ESMValCore/issues/1982.
-
     """
     mocked_nominatim = mocker.patch(
-        'esmvalcore.preprocessor._regrid.Nominatim', autospec=True
-    )
-    geolocation_penacaballera = mocker.Mock(
-        latitude=40.3442754, longitude=-5.8606859
-    )
+        'esmvalcore.preprocessor._regrid.Nominatim', autospec=True)
+    geolocation_penacaballera = mocker.Mock(latitude=40.3442754,
+                                            longitude=-5.8606859)
     mocked_nominatim.return_value.geocode.side_effect = (
-        lambda x: geolocation_penacaballera if x == 'Peñacaballera' else None
-    )
+        lambda x: geolocation_penacaballera if x == 'Peñacaballera' else None)
 
 
 @pytest.fixture
@@ -96,9 +94,7 @@ def test_no_location_parameter(test_cube):
     """Test if no location supplied."""
     msg = "Location needs to be specified."
     with pytest.raises(ValueError, match=msg):
-        extract_location(test_cube,
-                         scheme='nearest',
-                         location=None)
+        extract_location(test_cube, scheme='nearest', location=None)
 
 
 def test_no_scheme_parameter(test_cube):
@@ -108,3 +104,14 @@ def test_no_scheme_parameter(test_cube):
         extract_location(test_cube,
                          scheme=None,
                          location='Calvitero,Candelario')
+
+
+@patch("esmvalcore.preprocessor._regrid.ssl.create_default_context")
+def test_create_default_ssl_context_raises_exception(mock_create, test_cube):
+    """Test the original way 'extract_location' worked before adding the
+    default SSL context, see
+    https://github.com/ESMValGroup/ESMValCore/issues/2012 for more
+    information."""
+    mock_create.side_effect = ssl.SSLSyscallError
+    extract_location(test_cube, scheme="nearest", location="Peñacaballera")
+    mock_create.assert_called_once()