Repository: geopandas/geopandas

Issue:
Spatial joins
This is a sketch of an API for spatial joins in GeoPandas:

```
def sjoin(left_df, right_df, op='intersects', how='left', **kwargs)
    """ Spatial join of two GeoDataFrames.

    left_df, right_df are GeoDataFrames (or GeoSeries)
    op: binary predicate {'intersects', 'contains', 'dwithin'}
    how: the type of join {'left', 'right', 'inner', 'outer'}
    left_cols, right_cols: optional lists of columns to include from each side
    kwargs: passed to op method
```

References: PostGIS [spatial joins](http://revenant.ca/www/postgis/workshop/joins.html) and R's [over](http://cran.r-project.org/web/packages/sp/vignettes/over.pdf) function.



Discussion:
comment by @jwass: "Note that this work out only if the resulting GeoDataFrame could contain multiple geometry columns. If we don't allow multiple geom columns, I think the join API become much more complex."

To address the issue raised by @jwass , I think the join would effectively retain the geometries of `left_df` in the case of a left join. IOW the spatial join would never alter geometries, just use the binary predicate in the join clause. It would be analogous to this postgis query:

```
select L.attr, R.attr, L.geom  from L
left join R
on ST_Intersects(R.geom, L.geom)
```

The case of altered geometries might be more in line with what spatial overlays #112 aims to do?

One potential use case that is missing is the aggregation step that is common in these types of queries. This becomes an issue when a left geometry intersects with more than one right geometry. To extend the example above, you might want:

```
select sum(R.attr) as sum_right_attr, L.geom  from L
left join R
on ST_Intersects(R.geom, L.geom)
group by L.geom
```

This might translate to two steps in geopandas:

```
df = sjoin(left, right, op="intersection", how="left")
df.groupby(...)
```

Or we could extend `sjoin` to handle aggregation directly (upon writing it out, it smellsa little funky but we might as well explore our options, even bad ones!)

```
df = sjoin(left, right, op="intersection", how="left", 
               aggregate_column="attr", aggregate_func=sum)
```

@perrygeo My comment about the geometries should have had a caveat that it's only a problem if the user desires _both_ geometries to be retained. I agree that `sjoin` shouldn't modify the geometries.

Like this example:

```
SELECT ST_Distance(A.geom, B.geom)
FROM A JOIN B
WHERE ST_DWithin(A.geom, B.geom, 10)
```

or something like more complex than `ST_Distance`

Retaining the geometry from one of `left` or `right` wouldn't be a problem. Wanting both is the issue.

Also, we might want to join on a non-geometry column and then use the geometries on the joined dataframe, which we can't handle easily now... which is why I think we should figure out how to seamlessly allow multiple geom columns, but that's a whole different discussion :)

@perrygeo I'm a fan of the two step version of doing the `sjoin` then the `groupby`. `DataFrame.groupby`'s mechanics are really powerful and I think would be difficult to match in this interface.

Unless there's a performance benefit and specific algorithm that handles specifying the join and grouping simultaneously (?), I'd vote to leave them separate for now too.


Resolving patch:
diff --git a/geopandas/tools/__init__.py b/geopandas/tools/__init__.py
--- a/geopandas/tools/__init__.py
+++ b/geopandas/tools/__init__.py
@@ -1,5 +1,10 @@
 from __future__ import absolute_import
 
 from .overlay import overlay
+from .sjoin import sjoin
+
+__all__ = [
+    'overlay',
+    'sjoin',
+]
 
-__all__ = ['overlay']
diff --git a/geopandas/tools/sjoin.py b/geopandas/tools/sjoin.py
new file mode 100644
--- /dev/null
+++ b/geopandas/tools/sjoin.py
@@ -0,0 +1,62 @@
+import pandas as pd
+from geopandas import GeoDataFrame
+from .overlay import _uniquify
+
+def sjoin(left_df, right_df, how="left", op="intersects", use_sindex=True, **kwargs):
+    """Spatial join of two GeoDataFrames.
+
+    left_df, right_df are GeoDataFrames
+    how: type of join
+        left -> use keys from left_df; retain only left_df geometry column
+        right -> use keys from right_df; retain only right_df geometry column
+        inner -> use intersection of keys from both dfs; retain only left_df geometry column
+    op: binary predicate {'intersects', 'contains', 'within'}
+        see http://toblerity.org/shapely/manual.html#binary-predicates
+    use_sindex : Use the spatial index to speed up operation? Default is True
+    kwargs: passed to op method
+    """
+    allowed_hows = ['left', 'right', 'inner']
+
+    if how not in allowed_hows:
+        raise ValueError("`how` was \"%s\" but is expected to be in %s" % \
+            (how, allowed_hows))
+
+    if how == "right":
+        # right outer join just implemented as the inverse of left; swap names
+        left_df, right_df = right_df, left_df
+
+    collection = []
+    for i, feat in left_df.iterrows():
+        geom = feat.geometry
+
+        if hasattr(right_df, '_sindex') and right_df._sindex is not None and use_sindex:
+            candidates = [x.object for x in
+                           right_df._sindex.intersection(geom.bounds, objects=True)]
+        else:
+            candidates = [i for i, x in right_df.iterrows()]
+
+        feature_hits = 0
+        for cand_id in candidates:
+            candidate = right_df.ix[cand_id]
+            if getattr(geom, op)(candidate.geometry, **kwargs):
+                newseries = candidate.drop(right_df._geometry_column_name) 
+                newfeat = pd.concat([feat, newseries])
+                newfeat.index = _uniquify(newfeat.index)
+                collection.append(newfeat)
+                feature_hits += 1
+
+        # TODO Should we perform aggregation if feature_hit > 1?
+        # Advantage: single step and possible performance improvement
+        # Disadvantage: Pandas already has groupby so user can do this later
+
+        # If left does not spatially join with any right features,
+        # Fill in the right columns with NA
+        if how != 'inner' and feature_hits == 0:
+            empty = pd.Series(dict.fromkeys(right_df.columns, None))
+            empty.drop(right_df._geometry_column_name, inplace=True) 
+
+            newfeat = pd.concat([feat, empty])
+            newfeat.index = _uniquify(newfeat.index)
+            collection.append(newfeat)
+
+    return GeoDataFrame(collection, index=range(len(collection)))
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -85,7 +85,7 @@ def write_version_py(filename=None):
       author_email='kjordahl@enthought.com',
       url='http://geopandas.org',
       long_description=LONG_DESCRIPTION,
-      packages=['geopandas', 'geopandas.io'],
+      packages=['geopandas', 'geopandas.io', 'geopandas.tools'],
       install_requires=[
         'pandas', 'shapely', 'fiona', 'descartes', 'pyproj', 'rtree'],
 )


Test patch:
diff --git a/tests/test_plotting.py b/tests/test_plotting.py
--- a/tests/test_plotting.py
+++ b/tests/test_plotting.py
@@ -32,7 +32,7 @@ def tearDown(self):
         shutil.rmtree(self.tempdir)
         return
 
-    def _compare_images(self, ax, filename, tol=8):
+    def _compare_images(self, ax, filename, tol=10):
         """ Helper method to do the comparisons """
         assert isinstance(ax, Artist)
         if GENERATE_BASELINE:
diff --git a/tests/test_sjoin.py b/tests/test_sjoin.py
new file mode 100644
--- /dev/null
+++ b/tests/test_sjoin.py
@@ -0,0 +1,75 @@
+from __future__ import absolute_import
+import tempfile
+import shutil
+from shapely.geometry import Point
+from geopandas import GeoDataFrame, read_file
+from geopandas.tools import sjoin
+from .util import unittest, download_nybb
+
+
+class TestSpatialJoin(unittest.TestCase):
+
+    def setUp(self):
+        nybb_filename = download_nybb()
+        self.polydf = read_file('/nybb_14a_av/nybb.shp', vfs='zip://' + nybb_filename)
+        self.tempdir = tempfile.mkdtemp()
+        self.crs = {'init': 'epsg:4326'}
+        N = 20 
+        b = [int(x) for x in self.polydf.total_bounds]
+        self.pointdf = GeoDataFrame([
+            {'geometry' : Point(x, y), 'pointattr1': x + y, 'pointattr2': x - y}
+            for x, y in zip(range(b[0], b[2], int((b[2]-b[0])/N)),
+                            range(b[1], b[3], int((b[3]-b[1])/N)))], crs=self.crs)
+
+    def tearDown(self):
+        shutil.rmtree(self.tempdir)
+
+    def test_sjoin_left(self):
+        df = sjoin(self.pointdf, self.polydf)
+        self.assertEquals(df.shape, (21,7))
+        for i, row in df.iterrows():
+            self.assertEquals(row.geometry.type, 'Point')
+        self.assertTrue('pointattr1' in df.columns)
+        self.assertTrue('BoroCode' in df.columns)
+
+    def test_sjoin_right(self):
+        # the inverse of left
+        df = sjoin(self.pointdf, self.polydf, how="right")
+        df2 = sjoin(self.polydf, self.pointdf, how="left")
+        self.assertEquals(df.shape, (12, 7))
+        self.assertEquals(df.shape, df2.shape)
+        for i, row in df.iterrows():
+            self.assertEquals(row.geometry.type, 'MultiPolygon')
+        for i, row in df2.iterrows():
+            self.assertEquals(row.geometry.type, 'MultiPolygon')
+
+    def test_sjoin_inner(self):
+        df = sjoin(self.pointdf, self.polydf, how="inner")
+        self.assertEquals(df.shape, (11, 7))
+
+    def test_sjoin_op(self):
+        # points within polygons
+        df = sjoin(self.pointdf, self.polydf, how="left", op="within")
+        self.assertEquals(df.shape, (21,7))
+        self.assertAlmostEquals(df.ix[1]['Shape_Leng'], 330454.175933)
+
+        # points contain polygons? never happens so we should have nulls
+        df = sjoin(self.pointdf, self.polydf, how="left", op="contains")
+        self.assertEquals(df.shape, (21, 7))
+        self.assertEquals(df.ix[1]['Shape_Area'], None)
+
+    def test_sjoin_bad_op(self):
+        # AttributeError: 'Point' object has no attribute 'spandex'
+        self.assertRaises(AttributeError, sjoin,
+            self.pointdf, self.polydf, how="left", op="spandex")
+
+    def test_sjoin_duplicate_column_name(self):
+        pointdf2 = self.pointdf.rename(columns={'pointattr1': 'Shape_Area'})
+        df = sjoin(pointdf2, self.polydf, how="left")
+        self.assertTrue('Shape_Area' in df.columns)
+        self.assertTrue('Shape_Area_2' in df.columns)
+
+    @unittest.skip("Not implemented")
+    def test_sjoin_outer(self):
+        df = sjoin(self.pointdf, self.polydf, how="outer")
+        self.assertEquals(df.shape, (21,7))