Repository: geopandas/geopandas

Issue:
IndexError when conducting spatial join with empty polygons
Hey, not sure if this addressed elsewhere, but a search of past results suggests it's not. 

```python
import geopandas, numpy, shapely.geometry as geom

center = geom.Point((0,0))
circle = center.buffer(1)
empty = center.buffer(0)

empty_df = geopandas.GeoDataFrame(geometry=[circle, empty])

points = [geom.Point(x) for x in numpy.random.random(size=(10,2))]
points_df = geopandas.GeoDataFrame(geometry=points)

geopandas.sjoin(points_df, empty_df, op='intersects') # raises an IndexError but completes
```
I get a result from this join, but I also get an `IndexError`:

```
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
_ctypes/callbacks.c in 'calling callback function'()

~/anaconda3/envs/ana/lib/python3.7/site-packages/rtree/index.py in py_next_item(p_id, p_mins, p_maxs, p_dimension, p_data, p_length)
    708             # xmin, xmax, ymin, ymax, zmin, zmax
    709             for i in range(dimension):
--> 710                 mins[i] = coordinates[i*2]
    711                 maxs[i] = coordinates[(i*2)+1]
    712 

IndexError: list index out of range
```
Not sure how this `IndexError` relates to the result, but I didn't see another issue mentioning this, so I thought I'd raise it. 

rtree 0.8.3
geopandas 0.4.1
shapely 1.6.4.post1



Discussion:
Thanks for the report! I don't think that was already addressed (in any case, also on master getting this error). A bug that was fixed is the case when the *result* is empty.

Note that you actually don't have a classical "error". The error traceback is only printed, but there is still a GeoDataFrame returned (whether this is actually correct or not, that I didn't check yet)
I do think the result is correct, judging from a few checks. 

It might just be a solution to drop empty polygons and do the join only on non-empty geometries? 
I have the same problem but my resulting GeoDataFrame is NOT correct. @jorisvandenbossche Is there any way of catching these non-"classical" errors?
@julietsitron can you provide reproducible example of a situation where the result is not correct?
@martinfleis sure, what is the best way to share the data? I am trying to find a minimal example that reproduces the problem but the behavior is different when I grab subsets of my gdfs. To be clear, I am doing a spatial join between points and polygons. When my polygons have two rows with null geometries I get an IndexError and a resulting GeoDataFrame that is different from the result I get when performing the same spatial join with those two null rows removed from the Polygon gdf.
Ideally you could reduce it to a minimal example, either minimal enough so you can simply create the GeoDataFrame in code:

```
In [32]: from shapely.geometry import Point 

In [33]: df = geopandas.GeoDataFrame({'a': [1, 2], 'geometry': [Point(1, 1), Point(2, 2)]}) 
```

or if you could reduce your original data, you can also export a geojson like representation, and recreate the GeoDataFrames from that:

```
In [35]: geojson_dict  = df.__geo_interface__    

In [36]: geojson_dict
Out[36]: 
{'type': 'FeatureCollection',
 'features': [{'id': '0',
   'type': 'Feature',
   'properties': {'a': 1},
   'geometry': {'type': 'Point', 'coordinates': (1.0, 1.0)},
   'bbox': (1.0, 1.0, 1.0, 1.0)},
  {'id': '1',
   'type': 'Feature',
   'properties': {'a': 2},
   'geometry': {'type': 'Point', 'coordinates': (2.0, 2.0)},
   'bbox': (2.0, 2.0, 2.0, 2.0)}],
 'bbox': (1.0, 1.0, 2.0, 2.0)}

In [37]: geopandas.GeoDataFrame.from_features(geojson_dict) 
Out[37]: 
          geometry  a
0  POINT (1.0 1.0)  1
1  POINT (2.0 2.0)  2
```

But if you are in a position to share your original data, that is certainly welcome as well (then you can upload it to some cloud storage)

OK - let's try this example:

```
points_dict = {'type': 'FeatureCollection',
 'crs': {'type': 'name', 'properties': {'name': 'urn:ogc:def:crs:EPSG::2263'}},
 'features': [{'type': 'Feature',
   'properties': {},
   'geometry': {'type': 'Point',
    'coordinates': [995848.2333532274, 204748.7673894763]}},
  {'type': 'Feature',
   'properties': {},
   'geometry': {'type': 'Point',
    'coordinates': [995845.2825717181, 204700.63100281358]}},
  {'type': 'Feature',
   'properties': {},
   'geometry': {'type': 'Point',
    'coordinates': [990379.8443556428, 209054.7558247298]}},
  {'type': 'Feature',
   'properties': {},
   'geometry': {'type': 'Point',
    'coordinates': [995898.7837770581, 204751.01344797015]}},
  {'type': 'Feature',
   'properties': {},
   'geometry': {'type': 'Point',
    'coordinates': [990451.2119950503, 209002.9081593901]}},
  {'type': 'Feature',
   'properties': {},
   'geometry': {'type': 'Point',
    'coordinates': [990289.228723228, 208798.51322697103]}},
  {'type': 'Feature',
   'properties': {},
   'geometry': {'type': 'Point',
    'coordinates': [995876.3537037969, 204707.45874506235]}},
  {'type': 'Feature',
   'properties': {},
   'geometry': {'type': 'Point',
    'coordinates': [995954.2633248866, 204706.16314397752]}}]}

polys_nonulls_dict = {'type': 'FeatureCollection',
 'crs': {'type': 'name', 'properties': {'name': 'urn:ogc:def:crs:EPSG::2263'}},
 'features': [{'type': 'Feature',
   'properties': {},
   'geometry': {'type': 'Polygon',
    'coordinates': [[[996030.5157970488, 204702.45875506103],
      [996006.0709640533, 204900.9222688973],
      [995812.3367714733, 204877.1644423902],
      [995836.74321872, 204679.01457622647],
      [996030.5157970488, 204702.45875506103]]]}},
  {'type': 'Feature',
   'properties': {},
   'geometry': {'type': 'Polygon',
    'coordinates': [[[990506.6144433022, 209049.55012647808],
      [990370.9477198869, 209124.513887316],
      [990154.5203312188, 208732.8312717229],
      [990290.1870546341, 208657.8675108999],
      [990506.6144433022, 209049.55012647808]]]}}]}

polys_WITH_nulls_dict = {'type': 'FeatureCollection',
 'crs': {'type': 'name', 'properties': {'name': 'urn:ogc:def:crs:EPSG::2263'}},
 'features': [{'type': 'Feature',
   'properties': {},
   'geometry': {'type': 'Polygon',
    'coordinates': [[[996030.5157970488, 204702.45875506103],
      [996006.0709640533, 204900.9222688973],
      [995812.3367714733, 204877.1644423902],
      [995836.74321872, 204679.01457622647],
      [996030.5157970488, 204702.45875506103]]]}},
  {'type': 'Feature',
   'properties': {},
   'geometry': {'type': 'GeometryCollection', 'coordinates': [[]]}},
  {'type': 'Feature',
   'properties': {},
   'geometry': {'type': 'GeometryCollection', 'coordinates': [[]]}},
  {'type': 'Feature',
   'properties': {},
   'geometry': {'type': 'Polygon',
    'coordinates': [[[990506.6144433022, 209049.55012647808],
      [990370.9477198869, 209124.513887316],
      [990154.5203312188, 208732.8312717229],
      [990290.1870546341, 208657.8675108999],
      [990506.6144433022, 209049.55012647808]]]}}]}

correct_result = gpd.sjoin(gpd.GeoDataFrame.from_features(points_dict),gpd.GeoDataFrame.from_features(polys_nonulls_dict),op='intersects')

incorrect_result = gpd.sjoin(gpd.GeoDataFrame.from_features(points_dict),gpd.GeoDataFrame.from_features(polys_WITH_nulls_dict),op='intersects')
```
rtree  0.8.3
geopandas  0.5.1
shapely  1.6.4

Thanks!!



Thanks! I have found what is causing the issue and solution as well. For now you have to manually eliminate null geometry. `nulls.loc[~nulls.is_empty]`

At the moment, we are generating rtree index using 

```py
    right_df_bounds = right_df.geometry.apply(lambda x: x.bounds)
    stream = ((i, b, None) for i, b in enumerate(right_df_bounds))
    tree_idx = rtree.index.Index(stream)
```
while we can use simply `right_df.sindex`. It's generation is using almost the same code, but empty geometry is taken care of.

```py
stream = ((i, item.bounds, idx) for i, (idx, item) in
                      enumerate(self.geometry.iteritems())
                      if pd.notnull(item) and not item.is_empty)
```

I'll do PR to fix this.
Nope, not true anymore. This has been recently changed in #789. So master does not cause IndexError, but does no run anyway, raising `RTreeError: Coordinates must be in the form (minx, miny, maxx, maxy) or (x, y) for 2D indexes`.



Resolving patch:
diff --git a/geopandas/tools/sjoin.py b/geopandas/tools/sjoin.py
--- a/geopandas/tools/sjoin.py
+++ b/geopandas/tools/sjoin.py
@@ -106,7 +106,7 @@ def sjoin(
     # get rtree spatial index
     if tree_idx_right:
         idxmatch = left_df.geometry.apply(lambda x: x.bounds).apply(
-            lambda x: list(tree_idx.intersection(x))
+            lambda x: list(tree_idx.intersection(x)) if not x == () else []
         )
         idxmatch = idxmatch[idxmatch.apply(len) > 0]
         # indexes of overlapping boundaries
@@ -116,7 +116,7 @@ def sjoin(
     else:
         # tree_idx_df == 'left'
         idxmatch = right_df.geometry.apply(lambda x: x.bounds).apply(
-            lambda x: list(tree_idx.intersection(x))
+            lambda x: list(tree_idx.intersection(x)) if not x == () else []
         )
         idxmatch = idxmatch[idxmatch.apply(len) > 0]
         if idxmatch.shape[0] > 0:


Test patch:
diff --git a/geopandas/tools/tests/test_sjoin.py b/geopandas/tools/tests/test_sjoin.py
--- a/geopandas/tools/tests/test_sjoin.py
+++ b/geopandas/tools/tests/test_sjoin.py
@@ -3,7 +3,7 @@
 import numpy as np
 import pandas as pd
 
-from shapely.geometry import Point, Polygon
+from shapely.geometry import Point, Polygon, GeometryCollection
 
 import geopandas
 from geopandas import GeoDataFrame, GeoSeries, base, read_file, sjoin
@@ -319,6 +319,14 @@ def test_sjoin_outer(self):
         df = sjoin(self.pointdf, self.polydf, how="outer")
         assert df.shape == (21, 8)
 
+    def test_sjoin_empty_geometries(self):
+        # https://github.com/geopandas/geopandas/issues/944
+        empty = GeoDataFrame(geometry=[GeometryCollection()] * 3)
+        df = sjoin(self.pointdf.append(empty), self.polydf, how="left")
+        assert df.shape == (24, 8)
+        df2 = sjoin(self.pointdf, self.polydf.append(empty), how="left")
+        assert df2.shape == (21, 8)
+
 
 @pytest.mark.skipif(not base.HAS_SINDEX, reason="Rtree absent, skipping")
 class TestSpatialJoinNaturalEarth: