Repository: SciTools/cartopy

Issue:
PyProj transformers often reinstantiated
Profiling an animation I've been fiddling with, I noticed about 10 seconds (~15% of the total run time) was spent in `_safe_pj_transform`:

```
         24631906 function calls (23998894 primitive calls) in 62.484 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
...
     4660    0.025    0.000   11.467    0.002 geoaxes.py:117(transform_non_affine)
     4660    0.407    0.000   11.441    0.002 crs.py:336(transform_points)
...
     4660    0.016    0.000   10.998    0.002 crs.py:42(_safe_pj_transform)
     4660    0.025    0.000   10.859    0.002 transformer.py:471(from_crs)
     4660    0.033    0.000   10.813    0.002 transformer.py:298(__init__)
     4660    0.009    0.000   10.764    0.002 transformer.py:91(__call__)
     4660   10.724    0.002   10.746    0.002 {pyproj._transformer.from_crs}
```

Constantly creating new `Transformer` instances seems to be quite expensive. A quick hack improves things drastically:

```
sage: import functools
sage: ccrs.Transformer.from_crs = functools.cache(ccrs.Transformer.from_crs)
sage: cProfile.run('ani.save("/tmp/test.gif")', sort=pstats.SortKey.CUMULATIVE)
         24871283 function calls (24218669 primitive calls) in 51.364 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
...
     4660    0.020    0.000    0.322    0.000 geoaxes.py:117(transform_non_affine)
...
     4660    0.147    0.000    0.301    0.000 crs.py:336(transform_points)
...
     4660    0.019    0.000    0.123    0.000 crs.py:42(_safe_pj_transform)
...
        1    0.000    0.000    0.003    0.003 transformer.py:471(from_crs)
        1    0.000    0.000    0.003    0.003 transformer.py:298(__init__)
        1    0.000    0.000    0.003    0.003 transformer.py:91(__call__)
...
        1    0.003    0.003    0.003    0.003 {pyproj._transformer.from_crs}
```
Is there any chance Cartopy could take care of caching transformers in a similar manner?


Discussion:
Yes, see also #1895.
@Bob131, a PR with that proposed fix would be most welcome!

We added in the cache inside the Cython routine to catch the segment transforms, but it looks like I forgot about the Transformer creation in the transform_points area of the code. See here for the other cache that we currently have:
https://github.com/SciTools/cartopy/blob/72216fd3d68aaf3fb693327ff88baa032a0170db/lib/cartopy/trace.pyx#L579-L580
So I did prepare a PR, but the CLA put me off somewhat; I might leave it to someone else (sorry!).

I will mention a couple of things, though:
- It turns out this pathological behaviour is specifically related to inline labels on the gridliner.
- The only caller of `_safe_pj_transform` is `CRS.transform_points`, so instead of using `lru_cache` with an arbitrary size it seemed like a better idea to stash transformers in a `WeakKeyDictionary` on the `CRS`.
- I wasn't able to properly run the test suite anyway; the matplotlib tests run on my machine look like the baseline images pre 2f6e5a08, so almost every test fails.

Resolving patch:
diff --git a/lib/cartopy/crs.py b/lib/cartopy/crs.py
--- a/lib/cartopy/crs.py
+++ b/lib/cartopy/crs.py
@@ -12,6 +12,7 @@
 
 from abc import ABCMeta
 from collections import OrderedDict
+from functools import lru_cache
 import io
 import json
 import math
@@ -39,8 +40,14 @@
 WGS84_SEMIMINOR_AXIS = 6356752.3142
 
 
+# Cache the transformer creation method
+@lru_cache()
+def _get_transformer_from_crs(src_crs, tgt_crs):
+    return Transformer.from_crs(src_crs, tgt_crs, always_xy=True)
+
+
 def _safe_pj_transform(src_crs, tgt_crs, x, y, z=None, trap=True):
-    transformer = Transformer.from_crs(src_crs, tgt_crs, always_xy=True)
+    transformer = _get_transformer_from_crs(src_crs, tgt_crs)
     transformed_coords = transformer.transform(x, y, z, errcheck=trap)
     if z is None:
         xx, yy = transformed_coords
@@ -1639,6 +1646,9 @@ def __init__(self, central_longitude=0.0,
 
         super().__init__(proj4_params, globe=globe)
 
+        # Need to have x/y limits defined for the initial hash which
+        # gets used within transform_points for caching
+        self._x_limits = self._y_limits = None
         # Calculate limits.
         minlon, maxlon = self._determine_longitude_bounds(central_longitude)
         limits = self.transform_points(self.as_geodetic(),


Test patch: