Repository: opendatacube/datacube-core

Issue:
Non-EPSG WKT projections are slower to create than standard EPSG projections
### Expected behaviour
`CRS` object creation should have near-constant execution times for valid input strings.

### Actual behaviour
`CRS` objects created using a valid WKT string that does not have an equivalent EPSG representation are created ~5x slower due to calling and failing `to_epsg` before attempting the WKT string representation. While the execution time is only different by ~25ms, this code snippet is run for each individual band that's loaded so larger load calls can see significant slowdowns (0.5s difference for a 20 band single MODIS tile for our deployment, a yearly timeseries takes 12mins vs 7mins with the change mentioned below).


### Steps to reproduce the behaviour

This snippet can be used to reproduce the behavior:
```Python
from datacube.utils.geometry import CRS
from pyproj import CRS as _CRS

crs = _CRS.from_user_input(
    'PROJCS["unnamed",GEOGCS["Unknown datum based upon the custom spheroid",DATUM["Not specified (based on custom spheroid)",SPHEROID["Custom spheroid",6371007.181,0]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433]],PROJECTION["Sinusoidal"],PARAMETER["longitude_of_center",0],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["Meter",1],AXIS["Easting",EAST],AXIS["Northing",NORTH]]'
)
CRS(crs)
```

By updating the [_guess_crs_string](https://github.com/opendatacube/datacube-core/blob/f19a816f1a8f59041cffe5e53a100c6426e11096/datacube/utils/geometry/_base.py#L134) function to move the `to_wkt` check above the `to_epsg` check you can verify the execution time differences. On my local machine the runtimes before/after the update are:
![image](https://user-images.githubusercontent.com/16520854/148820220-c3ac33e1-b9bb-49cd-b93e-385a4b4256c9.png)
 
Projections with an EPSG code remain the same:
![image](https://user-images.githubusercontent.com/16520854/148826365-8a5cfe49-3a3e-41a6-af57-2a9879b59a6f.png)


### Environment information

* Which ``datacube --version`` are you using? `datacube==1.8.6`, `pyproj==3.1.0`
* What datacube deployment/environment are you running against? Local environment. 


> **Note:** Stale issues will be automatically closed after a period of six months with no activity. 
> To ensure critical issues are not closed, tag them with the Github `pinned` tag .



Discussion:
Is the `to_epsg` check necessary? I think all EPSG representations have an equivalent WKT string and most common Python CRS interfaces (pyproj, rasterio, ODC) provide both `to_epsg` and `to_wkt`, so WKT should cover both cases. 

cc @snowman2 
Also, the EPSG code is included in the WKT form at the end. For example `ID["EPSG",3211]]` in WKT2 and `AUTHORITY["EPSG","3211"]]` in WKT1 when calling `to_wkt()`:

```python
>>> import rasterio.crs
>>> import pyproj.crs
>>> rasterio.crs.CRS.from_epsg(3211).to_wkt()
'PROJCS["WGS 84 / SCAR IMW SQ39-40",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]],PROJECTION["Lambert_Conformal_Conic_2SP"],PARAMETER["latitude_of_origin",-90],PARAMETER["central_meridian",54],PARAMETER["standard_parallel_1",-64.6666666666667],PARAMETER["standard_parallel_2",-67.3333333333333],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","3211"]]'
>>> pyproj.crs.CRS.from_epsg(3211).to_wkt()
'PROJCRS["WGS 84 / SCAR IMW SQ39-40",BASEGEOGCRS["WGS 84",ENSEMBLE["World Geodetic System 1984 ensemble",MEMBER["World Geodetic System 1984 (Transit)"],MEMBER["World Geodetic System 1984 (G730)"],MEMBER["World Geodetic System 1984 (G873)"],MEMBER["World Geodetic System 1984 (G1150)"],MEMBER["World Geodetic System 1984 (G1674)"],MEMBER["World Geodetic System 1984 (G1762)"],MEMBER["World Geodetic System 1984 (G2139)"],ELLIPSOID["WGS 84",6378137,298.257223563,LENGTHUNIT["metre",1]],ENSEMBLEACCURACY[2.0]],PRIMEM["Greenwich",0,ANGLEUNIT["degree",0.0174532925199433]],ID["EPSG",4326]],CONVERSION["SCAR IMW SQ39-40",METHOD["Lambert Conic Conformal (2SP)",ID["EPSG",9802]],PARAMETER["Latitude of false origin",-90,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8821]],PARAMETER["Longitude of false origin",54,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8822]],PARAMETER["Latitude of 1st standard parallel",-64.6666666666667,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8823]],PARAMETER["Latitude of 2nd standard parallel",-67.3333333333333,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8824]],PARAMETER["Easting at false origin",0,LENGTHUNIT["metre",1],ID["EPSG",8826]],PARAMETER["Northing at false origin",0,LENGTHUNIT["metre",1],ID["EPSG",8827]]],CS[Cartesian,2],AXIS["(E)",east,ORDER[1],LENGTHUNIT["metre",1]],AXIS["(N)",north,ORDER[2],LENGTHUNIT["metre",1]],USAGE[SCOPE["Topographic mapping (medium scale)."],AREA["Antarctica - 64°S to 68°S and 48°E to 60°E."],BBOX[-68,48,-64,60]],ID["EPSG",3211]]'
```
@alfredoahds thanks for the analysis, let me summarize my understanding of the problem:

1. Extracting EPSG code from a CRS that doesn't have one is slow
2. When given non-string CRS object as input Datacube prefers `.to_epsg()` representation over `.to_wkt()` representation as an input into cached CRS object construction.
3. But `.to_epsg()` is expensive when input CRS is native CRS object from `pyproj`

Proposed solution is to swap the order of `.to_epsg` and `.to_wkt` checks in code below:

https://github.com/opendatacube/datacube-core/blob/f19a816f1a8f59041cffe5e53a100c6426e11096/datacube/utils/geometry/_base.py#L144-L149

This should work, I don't see it causing much problems.

But I think even better option is to short-circuit CRS object construction when input is a `pyproj.CRS` already.

https://github.com/opendatacube/datacube-core/blob/f19a816f1a8f59041cffe5e53a100c6426e11096/datacube/utils/geometry/_base.py#L162-L171

If in the code above badly named `crs_str` object is of the type `pyproj.CRS` then we should not be round-tripping it via string representation back into a `pyproj.CRS`.

And for your current scenario you can maybe work-around by using string representation of the CRS as input, as this one will use cached constructor and won't be calling `.to_epsg` all the time.

By the way this part of the datacube code is moving to a [stand-alone library](https://github.com/opendatacube/odc-geo/tree/cleanz). It will take sometime though...
Yep, that's correct. Short circuiting would likely be a better solution, but that function is called with a few different types depending on the call location - We create the DatasetType/Dataset model instances with a string representation of the CRS but the CRS init either receives a string (our input to the DatasetType/Dataset instances), `rasterio.crs.CRS` (pulled from the files we're loading), or `datacube.utils.geometry._base.CRS` (roundtripped version of the previous two). 

We've got a short term solution of just monkey-patching the reordered `.to_epsg` and `.to_wkt` function in, but if that content is moving to a separate package (eventually) it might be easier to just swap the epsg/wkt checks now and refactor the larger module when it's moved. Thoughts? 
I can't see this causing any problems, it really only changes what string is used for caching of `pyproj.CRS` objects for CRS's that DO have epsg. Raise a PR for the change and if tests pass we'll merge it, not sure when this will be released though...

Resolving patch:
diff --git a/datacube/utils/geometry/_base.py b/datacube/utils/geometry/_base.py
--- a/datacube/utils/geometry/_base.py
+++ b/datacube/utils/geometry/_base.py
@@ -141,12 +141,13 @@ def _guess_crs_str(crs_spec: Any) -> Optional[str]:
     if isinstance(crs_spec, dict):
         crs_spec = _CRS.from_dict(crs_spec)
 
+    if hasattr(crs_spec, 'to_wkt'):
+        return crs_spec.to_wkt()
     if hasattr(crs_spec, 'to_epsg'):
         epsg = crs_spec.to_epsg()
         if epsg is not None:
             return 'EPSG:{}'.format(crs_spec.to_epsg())
-    if hasattr(crs_spec, 'to_wkt'):
-        return crs_spec.to_wkt()
+
     return None
 
 


Test patch:
diff --git a/tests/test_geometry.py b/tests/test_geometry.py
--- a/tests/test_geometry.py
+++ b/tests/test_geometry.py
@@ -949,7 +949,7 @@ def test_crs():
 
     crs2 = CRS(crs)
     assert crs2 == crs
-    assert crs.proj is crs2.proj
+    assert crs.proj == crs2.proj
 
     assert epsg4326.valid_region == geometry.box(-180, -90, 180, 90, epsg4326)
     assert epsg3857.valid_region.crs == epsg4326
@@ -1475,7 +1475,7 @@ def test_crs_hash():
 
 
 def test_base_internals():
-    assert _guess_crs_str(CRS("epsg:3577")) == 'EPSG:3577'
+    assert _guess_crs_str(CRS("epsg:3577")) == epsg3577.to_wkt()
     no_epsg_crs = CRS(SAMPLE_WKT_WITHOUT_AUTHORITY)
     assert _guess_crs_str(no_epsg_crs) == no_epsg_crs.to_wkt()