Repository: geopandas/geopandas

Issue:
Check whether crs is the same when transforming
Currently, the `to_crs` method will always do a transformation, even if the requested crs is the same as the current one.  This could be improved with a simple dictionary check to skip if the crs is equal.  A more involved check would need to determine whether coordinate systems are equivalent even if the PROJ.4 parameters are not equal.

An example of different crs mappings that refer to the same projection is

```
crs1 = {'datum': 'NAD83', 'ellps': 'GRS80', 'proj': 'utm', 'units': 'm', 'zone': 18, 'no_defs': True}
crs2 = {'init': 'epsg:26918', 'no_defs': True}
```

which both represent [UTM zone 18N](http://spatialreference.org/ref/epsg/26918).



Discussion:
What if we took some N number of geo items from the geoseries on setting crs and tried to convert them. If they stayed the same, we'd believe that it was the same projection. No projections are exactly the same in any one particular area, right?

Relating to the discussion on #64, I'm not sure if  `ogr` or `pyproj` would 'pass through' in cases where the crs are the same; I think they would do the conversion naively. This is why it might be good to do more involved checks as suggested by @kjordahl when crs are to be compared (in `to_crs`, and any operations on two GeoSeries' [like intersect]). Is there any way to convert from Proj to WKT or something like that for comparison? Taking N number of geo items is risky: we might happen to grab a large geometry that is expensive to transform, or a geometry that doesn't change much (one close to the equator in the case of a Mercator projection)?

I'll look around, maybe we can just pick a canonical crs format and translate anything else to that crs format.

I'm pretty sure that `pyproj` would pass it naively and I suspect the same is true for `ogr`.  I'd be interested in how other geographic transformation handle this.  You might try just transforming the bounding box corners, but I can still imagine cases where that wouldn't work.  Getting the `crs` in a canonical form would be better.  I don't think WKT is going to be unique either, is it?

All of this is a special case, though.  I'll do the dict comparison for now, and we can think about handling the general case in the future.

Actually, the WKT output from `osr` looks promising:

```
>>> from osgeo import osr
>>> srs = osr.SpatialReference()
>>> srs.ImportFromProj4(to_string(crs))
>>> srs.GetAttrValue('projcs')
'UTM Zone 18, Northern Hemisphere'
>>> srs.ExportToWkt()
PROJCS["UTM Zone 18, Northern Hemisphere",GEOGCS["NAD83",DATUM["North_American_D
atum_1983",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]],T
OWGS84[0,0,0,0,0,0,0],AUTHORITY["EPSG","6269"]],PRIMEM["Greenwich",0,AUTHORITY["
EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9108"]],AUTHOR
ITY["EPSG","4269"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_ori
gin",0],PARAMETER["central_meridian",-75],PARAMETER["scale_factor",0.9996],PARAM
ETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["Meter",1]]
```

Which matches for my example above.

@kjordahl If we go with `osr.SpatialReference` as our internal crs representation, then we get `IsSame`, which tests: "Do these two spatial references describe the same system ?"

@cfarmer Ah, that's perfect for the comparison.  I'd prefer keeping the dict mapping as the standard stored representation, though, and abstract out comparison functions that use `osr` to do it.

@kjordahl That sounds reasonable. If we're using `osr` internally, perhaps we can revisit my faster `to_crs` implementation at some later stage ;-)

Wow, `IsSame()` doesn't even coerce 0 and 1 results to a `bool` in the python API.  I would love to see someone do for `gdal` and `osr` what Fiona has done for `ogr`.

@cfarmer Indeed, I haven't forgotten it ;-)

@kjordahl Funny you should say that: https://github.com/sgillies/rasterio. 

(GDAL's C API is proven and quite performant and good for C programming, but its Python bindings are a disaster.)

Couldn't PROJ.4 dicts ala pyproj (and Fiona) be canonical? They're like the GeoJSON of coordinate reference systems :)

Cool @sgillies, gonna play with that one right away! I'd be happy to use proj4 dicts as the canonical crs format, but I think there's just too much variation in crs definitions and (more importantly?) (mis)usage. At some point, conversion for comparison is probably required...

@sgillies Excellent!

I do prefer the crs mappings as our standard, I think we can make it more robust with validation and comparison when necessary.

My vote would be for to_crs() to always transform because functions that are sometimes no-ops can be unpredictable. 

Analogy: a unicode object's upper() method always (near as i can tell from looking in the source) cases upward and doesn't test whether the object is already upper case.

This makes sense, but checking would still be required for things like
intersections of GeoSeries' with potentially different crs. If they are
different, automatically projecting to the first GeoSeries' crs would be a
nice feature.

On Tue, Nov 12, 2013 at 1:41 PM, Sean Gillies notifications@github.comwrote:

> My vote would be for to_crs() to always transform because functions that
> are sometimes no-ops can be unpredictable.
> 
> Analogy: a unicode object's upper() method always (near as i can tell from
> looking in the source) cases upward and doesn't test whether the object is
> already upper case.
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/kjordahl/geopandas/issues/65#issuecomment-28320599
> .

## 

Dr. Carson J. Q. Farmer
Associate Director, CARSI http://www.carsilab.org/
Assistant Professor
Department of Geography
Hunter College - CUNY
695 Park Avenue
New York, NY, 10065
Carson.Farmer@gmail.com
@CarsonFarmer
www.CarsonFarmer.com

@sgillies Transformation could be expensive for a large object, and I can easily imagine a pattern in which a user will, say, plot a number of objects sequentially, and put them all through `to_crs` to ensure that they are in the same reference system.  I'm fine with deferring the comparison for equivalent reference systems for now, but it seems worth doing the simple dict comparison at least.

I'm not sure `upper()` is a fair comparison, that's just a table lookup in C that would probably take as long to check as it does to execute.  Numpy has lots of examples of functions that return the same object (or a copy) when appropriate (e.g. `np.atleast_2d`, `np.ndarray.astype`).  `to_crs()` with the same crs would be a no-op if `inplace == True`, otherwise we'd return a copy.

On the other hand, maybe the equivalent is a user multiplying two large matrices A and B.  You're probably not going to spend time checking first whether one of them is the identity matrix.

If we want to force set if user specifies, I suggest we use the following
hierarchy:
- to_crs takes a force=True kwarg that causes a projection even if we think
  it's the same crs.
- Setting crs property only causes projection if it's different
- internal projections for comparison (eg I pass a set of polygons that
  have a different crs than a set of points) should not force.

So it's basically just external calls to_crs always project.

We could reduce the complexity by disallowing direct setting of crs, but
it's also a nice Pythonic way to expose the API.

I'm a bit weary of doing anything beyond naively allowing setting the
crs... i.e. when setting, don't do _any_ projecting at all...
My reasoning: Many users working with geographic data separate 'setting'
crs from 'changing' crs. For instance, in QGIS, you can specify what crs a
layer _should_ be in, and you can also specify what crs a layer should be
projected _to_... I'm not quite sure if this is the best way or not, but we
should think about this, and perhaps stick with naively allowing someone to
directly set the crs without doing anything more clever than validation...

Having said that, if we _do_ want setting to work as Jeff suggests (which
I'm starting to think would be cool), then my above workflow would still be
possible with clever use of None:

df = GeoDataFrame.from_file(...) # get crs information from shapefile
df.crs = None # Don't do any projecting (obviously). This removes any
projection info, which I think should be allowed
df.crs = "test" # This isn't a valid crs specification _and_ it isn't None:
raise a ValueError
df.crs = {'init': 'epsg:26918', 'no_defs': True} # Again, don't do any
projecting (current crs is None)
df.crs = {'datum': 'NAD83', 'ellps': 'GRS80', 'proj': 'utm', 'units': 'm',
'zone': 18, 'no_defs': True} # Still no projecting because they are
equivalent
df.crs = {'init': 'epsg:4326', 'no_defs': True} # Finally project because
they are different

In this case, it might be worth having a setter which is accessible, and
takes a keyword project (or something):

def set_crs(self, crs, project=True):
if not validate(crs):
raise Something
if project: self.to_crs(crs)
else:
self.crs = crs
def get_crs(self): return self.crs crs = property(get_crs, set_crs)

That way, if I want to set the crs directly without any projecting, I can:
df.set_crs({'init': 'epsg:26918', 'no_defs': True}, project=False) # I'm
telling the GeoDataFrame that it is in EPSG 26918, whether it thinks it is
or not...

On Tue, Nov 12, 2013 at 2:48 PM, Jeff Tratner notifications@github.comwrote:

> If we want to force set if user specifies, I suggest we use the following
> hierarchy:
> - to_crs takes a force=True kwarg that causes a projection even if we think
>   it's the same crs.
> - Setting crs property only causes projection if it's different
> - internal projections for comparison (eg I pass a set of polygons that
>   have a different crs than a set of points) should not force.
> 
> So it's basically just external calls to_crs always project.
> 
> We could reduce the complexity by disallowing direct setting of crs, but
> it's also a nice Pythonic way to expose the API.
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/kjordahl/geopandas/issues/65#issuecomment-28326564
> .

## 

Dr. Carson J. Q. Farmer
Associate Director, CARSI http://www.carsilab.org/
Assistant Professor
Department of Geography
Hunter College - CUNY
695 Park Avenue
New York, NY, 10065
Carson.Farmer@gmail.com
@CarsonFarmer
www.CarsonFarmer.com

From a naive user's 'oh no where did I make an error??' perspective, it's
much much better to raise on bad input sooner rather than later. If there
are no incompatible crs, then we don't need to project immediately but we
do need to pass it to whatever will validate the crs so we can raise
quickly.

What concerns me about not projecting immediately is that you end up with
data in an inconsistent / misleading state. So say I have some geometry
that I haven't projected yet and I decide to write it to disk, unless we
track whether the data has been projected, you now are referring to a
totally different geometry. It's just simpler to reason about if the crs
always matches the current data.

Can you offer an example where you couldn't just set the crs right when you
care about projecting? I don't understand why you can't just wait until
you're doing something where the crs matters before setting. An example
would be instructive for me.
On Nov 12, 2013 3:15 PM, "Carson Farmer" notifications@github.com wrote:

I'm a bit weary of doing anything beyond naively allowing setting the
crs... i.e. when setting, don't do _any_ projecting at all...
My reasoning: Many users working with geographic data separate 'setting'
crs from 'changing' crs. For instance, in QGIS, you can specify what crs a
layer _should_ be in, and you can also specify what crs a layer should be
projected _to_... I'm not quite sure if this is the best way or not, but we
should think about this, and perhaps stick with naively allowing someone to
directly set the crs without doing anything more clever than validation...

Having said that, if we _do_ want setting to work as Jeff suggests (which
I'm starting to think would be cool), then my above workflow would still be
possible with clever use of None:

df = GeoDataFrame.from_file(...) # get crs information from shapefile
df.crs = None # Don't do any projecting (obviously). This removes any
projection info, which I think should be allowed
df.crs = "test" # This isn't a valid crs specification _and_ it isn't None:
raise a ValueError
df.crs = {'init': 'epsg:26918', 'no_defs': True} # Again, don't do any
projecting (current crs is None)
df.crs = {'datum': 'NAD83', 'ellps': 'GRS80', 'proj': 'utm', 'units': 'm',
'zone': 18, 'no_defs': True} # Still no projecting because they are
equivalent
df.crs = {'init': 'epsg:4326', 'no_defs': True} # Finally project because
they are different

In this case, it might be worth having a setter which is accessible, and
takes a keyword project (or something):

def set_crs(self, crs, project=True):
if not validate(crs):
raise Something
if project: self.to_crs(crs)
else:
self.crs = crs
def get_crs(self): return self.crs crs = property(get_crs, set_crs)

That way, if I want to set the crs directly without any projecting, I can:
df.set_crs({'init': 'epsg:26918', 'no_defs': True}, project=False) # I'm
telling the GeoDataFrame that it is in EPSG 26918, whether it thinks it is
or not...

On Tue, Nov 12, 2013 at 2:48 PM, Jeff Tratner notifications@github.comwrote:

> If we want to force set if user specifies, I suggest we use the following
> hierarchy:
> - to_crs takes a force=True kwarg that causes a projection even if we
>   think
>   it's the same crs.
> - Setting crs property only causes projection if it's different
> - internal projections for comparison (eg I pass a set of polygons that
>   have a different crs than a set of points) should not force.
> 
> So it's basically just external calls to_crs always project.
> 
> We could reduce the complexity by disallowing direct setting of crs, but
> it's also a nice Pythonic way to expose the API.
> 
> —
> Reply to this email directly or view it on GitHub<
> https://github.com/kjordahl/geopandas/issues/65#issuecomment-28326564>
> .

## 

Dr. Carson J. Q. Farmer
Associate Director, CARSI http://www.carsilab.org/
Assistant Professor
Department of Geography
Hunter College - CUNY
695 Park Avenue
New York, NY, 10065
Carson.Farmer@gmail.com
@CarsonFarmer
www.CarsonFarmer.com

—

Reply to this email directly or view it on
GitHubhttps://github.com/kjordahl/geopandas/issues/65#issuecomment-28328840
.

I don't like the idea of doing the transformation on setting the crs property either.

It seems to me that setting the `crs` property should validate, but not reproject.  We could validate that a `pyproj` dict mapping or PROJ.4 string can generate a proper `Proj` instance without raising an error, for example (and store it as a dict).

@jtratner I'm not sure I understand the argument about an inconsistent state.  It's always going to be possible to set the wrong metadata, the best we can do is make sure that it's valid metadata.  Often we may get data without an explicit crs and have to adapt it manually (e.g. lat/lon columns in a table).

Okay, sounds good.

All I mean is that if you serialize your geometry and claim it has some
projection, but it hasn't actually been projected, that could get confusing.

I'm with @kjordahl, but I think @jtratner's mention of inconsistent states lies here: I'm not suggesting projection is deferred or put off, I'm suggesting it never happens in the case of direct setting. In other words, if I set the crs using something like `df.set_crs(new_crs, project=False)` (or the current way of direct setting), then crs validation is done right away, and if its invalid, raise an exception, otherwise, set the crs property but _don't_ _re_project at all, even if they're different. This is based on the idea that I know better than some naive geometry what its projection is ;-)

I get what you mean.  So:

```
gdf.crs = some_crs
```

means "The geometry of this geodataframe was _actually_ some_crs"

whereas

```
gdf = gdf.to_crs(some_crs)
```

means "change/project the crs of the geometry to some_crs"

@jtratner Exactly.

Probably a good analogy would be PostGIS. `ST_SetSRID` just changes the meta-data associated with the geometry but does no projecting. `ST_Transform` projects from the stored projection to the specified one.

For GeoPandas, `ST_SetSRID` -> setting crs property. `ST_Transform` -> `to_crs`. The analogy isn't perfect because in PostGIS, each geometry has its own crs.

Just like http://postgis.refractions.net/docs/ST_SetSRID.html, it should be very clear in the documentation that no projecting is done when you set the crs. I wouldn't agree with the idea of even providing the flag to project on setting the crs.

So why not ditch `to_crs` as a term and call it project instead? (especially if PostGIS calls it something different).  That way it's not ambiguous, setting crs is metadata, projecting is an action that could have non-trivial cost.

```
gdf.project(another_crs)
```

And then we figure out some keyword argument for "do this even if the crs's are equal"

Possibly because `project` is already a shapely method as well?
http://toblerity.org/shapely/manual.html#object.project

Though I never really liked this name for this exact reason...

While we're thinking about renaming things... perhaps `project` in the shapely sense could be renamed to `dist_along` instead?

Maybe `transform()` (if we don't expose `shapely.ops.transform`), `transform_crs()`? 

I'd prefer to keep a one-to-one mapping of the geom method names to the corresponding Shapely methods... and I believe all Shapely methods are named similarly to their corresponding GEOS functions, with minimal modification.

maybe `project_to_crs(crs)`? a little verbose, but makes it clear what input it's expecting.

I think pyproj will now skip transformations if crs are the same which make the check almost useless.
see https://pyproj4.github.io/pyproj/html/optimize_transformations.html

> I think pyproj will now skip transformations if crs are the same which make the check almost useless.
see https://pyproj4.github.io/pyproj/html/optimize_transformations.html

It does as of pyproj version 2.1.2, see [pyproj CHANGELOG](https://github.com/pyproj4/pyproj/blob/master/sphinx/history.rst).
I would recommend using pyproj.CRS.is_exact_same for skipping transformations. Even though pyproj does skip the transformation, you can skip even iteration over an empty operation with this check.
Indeed, it would still be beneficial to skip on the geopandas level, so we don't need to reconstruct the geometries from the unconverted coordinates. 


Resolving patch:
diff --git a/geopandas/geoseries.py b/geopandas/geoseries.py
--- a/geopandas/geoseries.py
+++ b/geopandas/geoseries.py
@@ -18,7 +18,8 @@
 from .array import GeometryDtype, from_shapely
 from .base import is_geometry_type
 
-_PYPROJ2 = LooseVersion(pyproj.__version__) >= LooseVersion("2.1.0")
+
+_PYPROJ_VERSION = LooseVersion(pyproj.__version__)
 
 
 _SERIES_WARNING_MSG = """\
@@ -389,22 +390,38 @@ def to_crs(self, crs=None, epsg=None):
         """
         from fiona.crs import from_epsg
 
+        if crs is None and epsg is None:
+            raise TypeError("Must set either crs or epsg for output.")
+
         if self.crs is None:
             raise ValueError(
                 "Cannot transform naive geometries.  "
                 "Please set a crs on the object first."
             )
+
         if crs is None:
             try:
                 crs = from_epsg(epsg)
-            except TypeError:
-                raise TypeError("Must set either crs or epsg for output.")
-        proj_in = pyproj.Proj(self.crs, preserve_units=True)
-        proj_out = pyproj.Proj(crs, preserve_units=True)
-        if _PYPROJ2:
-            transformer = pyproj.Transformer.from_proj(proj_in, proj_out)
+            except (TypeError, ValueError):
+                raise ValueError("Invalid epsg: {}".format(epsg))
+
+        # skip transformation if the input CRS and output CRS are the exact same
+        if _PYPROJ_VERSION >= LooseVersion("2.1.2") and pyproj.CRS.from_user_input(
+            self.crs
+        ).is_exact_same(pyproj.CRS.from_user_input(crs)):
+            return self
+
+        if _PYPROJ_VERSION >= LooseVersion("2.2.0"):
+            # if availale, use always_xy=True to preserve GIS axis order
+            transformer = pyproj.Transformer.from_crs(self.crs, crs, always_xy=True)
+            project = transformer.transform
+        elif _PYPROJ_VERSION >= LooseVersion("2.1.0"):
+            # use transformer for repeated transformations
+            transformer = pyproj.Transformer.from_crs(self.crs, crs)
             project = transformer.transform
         else:
+            proj_in = pyproj.Proj(self.crs, preserve_units=True)
+            proj_out = pyproj.Proj(crs, preserve_units=True)
             project = partial(pyproj.transform, proj_in, proj_out)
         result = self.apply(lambda geom: transform(project, geom))
         result.__class__ = GeoSeries


Test patch:
diff --git a/geopandas/tests/test_crs.py b/geopandas/tests/test_crs.py
--- a/geopandas/tests/test_crs.py
+++ b/geopandas/tests/test_crs.py
@@ -1,6 +1,11 @@
+from distutils.version import LooseVersion
+
 import numpy as np
 
+from shapely.geometry import Point
+
 from geopandas import GeoDataFrame, points_from_xy
+from geopandas.geoseries import _PYPROJ_VERSION
 
 from geopandas.testing import assert_geodataframe_equal
 import pytest
@@ -94,3 +99,22 @@ def test_transform2(epsg4326, epsg26918):
     # can't check for CRS equality, as the formats differ although representing
     # the same CRS
     assert_geodataframe_equal(df, utm, check_less_precise=True, check_crs=False)
+
+
+@pytest.mark.skipif(
+    _PYPROJ_VERSION < LooseVersion("2.2.0"),
+    reason="EPSG strings without +init= won't work on previous versions of pyproj.",
+)
+def test_crs_axis_order__always_xy():
+    df = GeoDataFrame(geometry=[Point(-1683723, 6689139)], crs="epsg:26918")
+    lonlat = df.to_crs("epsg:4326")
+    test_lonlat = GeoDataFrame(
+        geometry=[Point(-110.1399901, 55.1350011)], crs="epsg:4326"
+    )
+    assert_geodataframe_equal(lonlat, test_lonlat, check_less_precise=True)
+
+
+def test_skip_exact_same():
+    df = df_epsg26918()
+    utm = df.to_crs(df.crs)
+    assert_geodataframe_equal(df, utm, check_less_precise=True)