Repository: geopandas/geopandas

Issue:
Should dropping the geometry column produce a Pandas dataframe?
If I do an operation with loses the geometry column like 

    df[['a', 'b', 'c']]

or 

    df.drop('geometry')

Should I get back a vanilla Pandas DataFrame?

To be clear, the current answer is "no".  I'm curious if there is value to changing the current behavior.


Discussion:
Yes, it seems to make sense to me to return a DataFrame in those occasions, as it would otherwise be an invalid GeoDataFrame anyhow.
A `GeoDataFrame` with `geometry=[]` is still a `GeoDataFrame`. I wouldn't say it's invalid, no? And implicit type mutation strikes me as bad.

If you really want it, `pd.DataFrame(gdf.values)` will give it to you.
This is similar to selecting a single column from a GeoDataFrame: this also does not give a GeoSeries, but a Series:

```
In [5]: df = geopandas.GeoDataFrame({'a': [1, 2], 'b': [3, 4]}, geometry=[Point(1, 1), Point(2, 2)])

In [6]: df['a']
Out[6]: 
0    1
1    2
Name: a, dtype: int64

In [7]: type(df['a'])
Out[7]: pandas.core.series.Series
```

and I think this makes sense (only getting the geometry column will give a GeoSeries). Given this, I think it also makes sense to return a DataFrame for multiple columns (that do not contain the geometry column):

```
In [8]: df[['a', 'b']]
Out[8]: 
   a  b
0  1  3
1  2  4

In [9]: type(df[['a', 'b']])
Out[9]: pandas.core.frame.DataFrame
```

Actually, @mrocklin I get here already a DataFrame. Did you test this on master or on the geopandas-cython branch ?

> A GeoDataFrame with geometry=[] is still a GeoDataFrame. I wouldn't say it's invalid, no?

Not sure what you mean with `geometry=[]` ? The question is here is not about "no data in the geometry column" but "no geometry column" which is somewhat different.

You can actually create such a GeoDataFrame without geometry with the constructor and not passing any geometries (but then you get an error once you do something geometrical, so that is what I meant with 'invalid'):

```
In [12]: df = geopandas.GeoDataFrame({'a': [1, 2], 'b': [3, 4]})

In [13]: df
Out[13]: 
   a  b
0  1  3
1  2  4

In [14]: type(df)
Out[14]: geopandas.geodataframe.GeoDataFrame

In [15]: df.geometry
...
AttributeError: No geometry data set yet (expected in column 'geometry'.

In [16]: df.geom_type
...
AttributeError: No geometry data set yet (expected in column 'geometry'.


w.r.t. geometry I meant a `gpd.GeoDataFrame(geometry=[])` declaration. In this case the geometry column is set, but set to an empty list. All functions behave as expected.

You can get a construct with actual length using:

    gpd.GeoDataFrame({'a': [1], 'b': [2]}, geometry=[None]).geometry

Admittedly these are tangential edge cases.

I strongly associate `GeoSeries` with geometries and didn't even realize constructs like `gpd.GeoSeries(['a', 'b'])` were legal. Very well, I suppose you are right.
Ah, yes, an empty GeoDataFrame (no rows) can certainly exist. But in that case it also has the geometry column:

```
In [160]: df = geopandas.GeoDataFrame({'a': [], 'b': []}, geometry=[])

In [161]: df
Out[161]: 
Empty GeoDataFrame
Columns: [a, b, geometry]
Index: []

In [162]: type(df)
Out[162]: geopandas.geodataframe.GeoDataFrame

In [163]: df.columns
Out[163]: Index(['a', 'b', 'geometry'], dtype='object')
```

So I was rather thinking about the case there is data, but no geometry column:
```
In [165]: df = geopandas.GeoDataFrame({'a': [1, 2], 'b': [3, 4]})

In [166]: df
Out[166]: 
   a  b
0  1  3
1  2  4

In [167]: type(df)
Out[167]: geopandas.geodataframe.GeoDataFrame
```

This last one could be a DataFrame.

> gpd.GeoDataFrame({'a': [1], 'b': [2]}, geometry=[None]).geometry

This is fine I think, since we support missing values in the geometry column. So this is such a case.

> didn't even realize constructs like gpd.GeoSeries(['a', 'b']) were legal.

Yes, that is a similar case. Currently that returns a GeoSeries, but in the cython refactor branch it makes sense to return a Series (but currently raises error, see discussion here: https://github.com/geopandas/geopandas/issues/473#issuecomment-329620408)
Personally, I don't want things to change types unexpectedly. If I want a DataFrame I can already do:
`df = pd.DataFrame(gdf[[col for col in gdf.columns if col != gdf._geometry_column_name]])`

I would be open to having a helper method to get a dataframe without the geometry column though.
@jwilson8767 The question is to what extent is it *useful* that it stays a GeoDataFrame, as none of the extra geometry-related methods will work, and possibly also some over the methods of DataFrame that we override might break if there is no geometry.
Perhaps instead of "automatically" converting to a DataFrame, would adding a `.to_dataframe()` (maybe `.to_pandas()` or something) method be acceptable? 

This would be similar to the pattern present on Series, which allows them to be turned into dataframe with `.to_frame()`.


> Perhaps instead of "automatically" converting to a DataFrame, would adding a `.to_dataframe()` (maybe `.to_pandas()` or something) method be acceptable?
> 
> This would be similar to the pattern present on Series, which allows them to be turned into dataframe with `.to_frame()`.

Support to have a straight forward and descriptive reverse method.

Like in the NYC TAXI demo, where you could only temporarily want a geometry to do the spatial joins, but you are happy with only IDs of Areas afterwards.

https://gist.github.com/jorisvandenbossche/67be41a246c1281d7046b31690988321

Resolving patch:
diff --git a/geopandas/geodataframe.py b/geopandas/geodataframe.py
--- a/geopandas/geodataframe.py
+++ b/geopandas/geodataframe.py
@@ -13,7 +13,7 @@
 
 from geopandas.array import GeometryArray, GeometryDtype, from_shapely, to_wkb, to_wkt
 from geopandas.base import GeoPandasBase, is_geometry_type
-from geopandas.geoseries import GeoSeries
+from geopandas.geoseries import GeoSeries, _geoseries_constructor_with_fallback
 import geopandas.io
 from geopandas.explore import _explore
 from . import _compat as compat
@@ -23,6 +23,20 @@
 DEFAULT_GEO_COLUMN_NAME = "geometry"
 
 
+def _geodataframe_constructor_with_fallback(*args, **kwargs):
+    """
+    A flexible constructor for GeoDataFrame._constructor, which falls back
+    to returning a DataFrame (if a certain operation does not preserve the
+    geometry column)
+    """
+    df = GeoDataFrame(*args, **kwargs)
+    geometry_cols_mask = df.dtypes == "geometry"
+    if len(geometry_cols_mask) == 0 or geometry_cols_mask.sum() == 0:
+        df = pd.DataFrame(df)
+
+    return df
+
+
 def _ensure_geometry(data, crs=None):
     """
     Ensure the data is of geometry dtype or converted to it.
@@ -276,6 +290,9 @@ def set_geometry(self, col, drop=False, inplace=False, crs=None):
             frame = self
         else:
             frame = self.copy()
+            # if there is no previous self.geometry, self.copy() will downcast
+            if type(frame) == DataFrame:
+                frame = GeoDataFrame(frame)
 
         to_remove = None
         geo_column_name = self._geometry_column_name
@@ -1390,20 +1407,26 @@ def apply(self, func, axis=0, raw=False, result_type=None, args=(), **kwargs):
         result = super().apply(
             func, axis=axis, raw=raw, result_type=result_type, args=args, **kwargs
         )
-        if (
-            isinstance(result, GeoDataFrame)
-            and self._geometry_column_name in result.columns
-            and isinstance(result[self._geometry_column_name].dtype, GeometryDtype)
-        ):
-            # apply calls _constructor which resets geom col name to geometry
-            result._geometry_column_name = self._geometry_column_name
-            if self.crs is not None and result.crs is None:
-                result.set_crs(self.crs, inplace=True)
+        # Reconstruct gdf if it was lost by apply
+        if self._geometry_column_name in result.columns:
+            # axis=1 apply will split GeometryDType to object, try and cast back
+            try:
+                result = result.set_geometry(self._geometry_column_name)
+            except TypeError:
+                pass
+            else:
+                if self.crs is not None and result.crs is None:
+                    result.set_crs(self.crs, inplace=True)
+
         return result
 
     @property
     def _constructor(self):
-        return GeoDataFrame
+        return _geodataframe_constructor_with_fallback
+
+    @property
+    def _constructor_sliced(self):
+        return _geoseries_constructor_with_fallback
 
     def __finalize__(self, other, method=None, **kwargs):
         """propagate metadata from other to self"""
@@ -1671,6 +1694,7 @@ def explode(self, column=None, ignore_index=False, index_parts=None, **kwargs):
         df = (
             df_copy.drop(df_copy._geometry_column_name, axis=1)
             .join(exploded_geom)
+            .set_geometry(self._geometry_column_name)
             .__finalize__(self)
         )
 
@@ -1688,8 +1712,7 @@ def explode(self, column=None, ignore_index=False, index_parts=None, **kwargs):
         if f"__{level_str}" in df.columns:
             df = df.rename(columns={f"__{level_str}": level_str})
 
-        geo_df = df.set_geometry(self._geometry_column_name)
-        return geo_df
+        return df
 
     # overrides the pandas astype method to ensure the correct return type
     def astype(self, dtype, copy=True, errors="raise", **kwargs):
@@ -2209,3 +2232,6 @@ def _dataframe_set_geometry(self, col, drop=False, inplace=False, crs=None):
 
 
 DataFrame.set_geometry = _dataframe_set_geometry
+
+if compat.PANDAS_GE_10 and not compat.PANDAS_GE_11:  # i.e. on pandas 1.0.x
+    _geodataframe_constructor_with_fallback._from_axes = GeoDataFrame._from_axes
diff --git a/geopandas/geoseries.py b/geopandas/geoseries.py
--- a/geopandas/geoseries.py
+++ b/geopandas/geoseries.py
@@ -627,13 +627,6 @@ def apply(self, func, convert_dtype=True, args=(), **kwargs):
                 result.set_crs(self.crs, inplace=True)
         return result
 
-    def __finalize__(self, other, method=None, **kwargs):
-        """propagate metadata from other to self"""
-        # NOTE: backported from pandas master (upcoming v0.13)
-        for name in self._metadata:
-            object.__setattr__(self, name, getattr(other, name, None))
-        return self
-
     def isna(self):
         """
         Detect missing values.
diff --git a/geopandas/tools/sjoin.py b/geopandas/tools/sjoin.py
--- a/geopandas/tools/sjoin.py
+++ b/geopandas/tools/sjoin.py
@@ -343,6 +343,7 @@ def _frame_join(join_df, left_df, right_df, how, lsuffix, rsuffix):
             )
             .set_index(index_right)
             .drop(["_key_left", "_key_right"], axis=1)
+            .set_geometry(right_df.geometry.name)
         )
         if isinstance(index_right, list):
             joined.index.names = right_index_name


Test patch:
diff --git a/geopandas/io/tests/test_file.py b/geopandas/io/tests/test_file.py
--- a/geopandas/io/tests/test_file.py
+++ b/geopandas/io/tests/test_file.py
@@ -214,7 +214,7 @@ def test_to_file_int64(tmpdir, df_points):
 
 
 def test_to_file_empty(tmpdir):
-    input_empty_df = GeoDataFrame()
+    input_empty_df = GeoDataFrame(columns=["geometry"])
     tempfilename = os.path.join(str(tmpdir), "test.shp")
     with pytest.raises(ValueError, match="Cannot write empty DataFrame to file."):
         input_empty_df.to_file(tempfilename)
diff --git a/geopandas/tests/test_op_output_types.py b/geopandas/tests/test_op_output_types.py
new file mode 100644
--- /dev/null
+++ b/geopandas/tests/test_op_output_types.py
@@ -0,0 +1,216 @@
+import pandas as pd
+import pyproj
+import pytest
+
+from shapely.geometry import Point
+
+from geopandas import GeoDataFrame, GeoSeries
+
+
+crs_osgb = pyproj.CRS(27700)
+crs_wgs = pyproj.CRS(4326)
+
+
+N = 10
+
+
+@pytest.fixture(params=["geometry", "point"])
+def df(request):
+    geo_name = request.param
+
+    df = GeoDataFrame(
+        [
+            {
+                "value1": x + y,
+                "value2": x * y,
+                geo_name: Point(x, y),  # rename this col in tests
+            }
+            for x, y in zip(range(N), range(N))
+        ],
+        crs=crs_wgs,
+        geometry=geo_name,
+    )
+    # want geometry2 to be a GeoSeries not Series, test behaviour of non geom col
+    df["geometry2"] = df[geo_name].set_crs(crs_osgb, allow_override=True)
+    return df
+
+
+def _check_metadata_gdf(gdf, geo_name="geometry", crs=crs_wgs):
+    assert gdf._geometry_column_name == geo_name
+    assert gdf.geometry.name == geo_name
+    assert gdf.crs == crs
+
+
+def _check_metadata_gs(gs, name="geometry", crs=crs_wgs):
+    assert gs.name == name
+    assert gs.crs == crs
+
+
+def assert_object(result, expected_type, geo_name="geometry", crs=crs_wgs):
+    """
+    Helper method to make tests easier to read. Checks result is of the expected
+    type. If result is a GeoDataFrame or GeoSeries, checks geo_name
+    and crs match. If geo_name is None, then we expect a GeoDataFrame
+    where the geometry column is invalid/ isn't set. This is never desirable,
+    but is a reality of this first stage of implementation.
+    """
+    assert type(result) is expected_type
+
+    if expected_type == GeoDataFrame:
+        if geo_name is not None:
+            _check_metadata_gdf(result, geo_name=geo_name, crs=crs)
+        else:
+            with pytest.raises(AttributeError, match="No geometry data set yet"):
+                result.geometry.name  # be explicit that geometry is invalid here
+    elif expected_type == GeoSeries:
+        _check_metadata_gs(result, name=geo_name, crs=crs)
+
+
+def test_getitem(df):
+    geo_name = df.geometry.name
+    assert_object(df[["value1", "value2"]], pd.DataFrame)
+    assert_object(df[[geo_name, "geometry2"]], GeoDataFrame, geo_name)
+    assert_object(df[[geo_name]], GeoDataFrame, geo_name)
+    assert_object(df[["geometry2", "value1"]], pd.DataFrame)
+    assert_object(df[["geometry2"]], pd.DataFrame)
+    assert_object(df[["value1"]], pd.DataFrame)
+    # Series
+    assert_object(df[geo_name], GeoSeries, geo_name)
+    assert_object(df["geometry2"], GeoSeries, "geometry2", crs=crs_osgb)
+    assert_object(df["value1"], pd.Series)
+
+
+def test_loc(df):
+    geo_name = df.geometry.name
+    assert_object(df.loc[:, ["value1", "value2"]], pd.DataFrame)
+    assert_object(df.loc[:, [geo_name, "geometry2"]], GeoDataFrame, geo_name)
+    assert_object(df.loc[:, [geo_name]], GeoDataFrame, geo_name)
+    # TODO: should this set the geometry column to geometry2 or return a DataFrame?
+    assert_object(df.loc[:, ["geometry2", "value1"]], GeoDataFrame, None)
+    assert_object(df.loc[:, ["geometry2"]], GeoDataFrame, None)
+    # Ideally this would mirror __getitem__ and below would be true
+    # assert_object(df.loc[:, ["geometry2", "value1"]], pd.DataFrame)
+    # assert_object(df.loc[:, ["geometry2"]], pd.DataFrame)
+    assert_object(df.loc[:, ["value1"]], pd.DataFrame)
+    # Series
+    assert_object(df.loc[:, geo_name], GeoSeries, geo_name)
+    assert_object(df.loc[:, "geometry2"], GeoSeries, "geometry2", crs=crs_osgb)
+    assert_object(df.loc[:, "value1"], pd.Series)
+
+
+def test_iloc(df):
+    geo_name = df.geometry.name
+    assert_object(df.iloc[:, 0:2], pd.DataFrame)
+    assert_object(df.iloc[:, 2:4], GeoDataFrame, geo_name)
+    assert_object(df.iloc[:, [2]], GeoDataFrame, geo_name)
+    # TODO: should this set the geometry column to geometry2 or return a DataFrame?
+    assert_object(df.iloc[:, [3, 0]], GeoDataFrame, None)
+    assert_object(df.iloc[:, [3]], GeoDataFrame, None)
+    # Ideally this would mirror __getitem__ and below would be true
+    # assert_object(df.iloc[:, [3, 0]], pd.DataFrame)
+    # assert_object(df.iloc[:, [3]], pd.DataFrame)
+    assert_object(df.iloc[:, [0]], pd.DataFrame)
+    # Series
+    assert_object(df.iloc[:, 2], GeoSeries, geo_name)
+    assert_object(df.iloc[:, 3], GeoSeries, "geometry2", crs=crs_osgb)
+    assert_object(df.iloc[:, 0], pd.Series)
+
+
+def test_squeeze(df):
+    geo_name = df.geometry.name
+    assert_object(df[[geo_name]].squeeze(), GeoSeries, geo_name)
+
+    # Not ideal behaviour, but this is consistent with __getitem__
+    assert_object(df[["geometry2"]].squeeze(), pd.Series)
+
+
+def test_to_frame(df):
+    geo_name = df.geometry.name
+    # TODO this reflects current behaviour, but we should fix
+    #  GeoSeries._constructor_expanddim so this doesn't happen
+    res1 = df[geo_name].to_frame()
+    if geo_name == "geometry":  # -> this should be doable for any geo_name
+        assert_object(res1, GeoDataFrame, geo_name)
+    assert res1._geometry_column_name == "geometry"  # -> should be geo_name
+
+    res2 = df["geometry2"].to_frame()
+    assert type(res2) is GeoDataFrame
+    assert res2._geometry_column_name == "geometry"  # -> should be geometry2
+    assert res2.crs is None  # -> should be crs_osgb
+    # also res2.geometry should not crash because geometry isn't set
+
+    res3 = df["value1"].to_frame()
+    assert_object(res3, pd.DataFrame)
+
+
+def test_reindex(df):
+    geo_name = df.geometry.name
+    assert_object(df.reindex(columns=["value1", "value2"]), pd.DataFrame)
+    assert_object(df.reindex(columns=[geo_name, "geometry2"]), GeoDataFrame, geo_name)
+    assert_object(df.reindex(columns=[geo_name]), GeoDataFrame, geo_name)
+    assert_object(df.reindex(columns=["new_col", geo_name]), GeoDataFrame, geo_name)
+    # TODO: should this set the geometry column to geometry2 or return a DataFrame?
+    assert_object(df.reindex(columns=["geometry2", "value1"]), GeoDataFrame, None)
+    assert_object(df.reindex(columns=["geometry2"]), GeoDataFrame, None)
+    # Ideally this would mirror __getitem__ and below would be true
+    # assert_object(df.reindex(columns=["geometry2", "value1"]), pd.DataFrame)
+    # assert_object(df.reindex(columns=["geometry2"]), pd.DataFrame)
+    assert_object(df.reindex(columns=["value1"]), pd.DataFrame)
+
+    # reindexing the rows always preserves the GeoDataFrame
+    assert_object(df.reindex(index=[0, 1, 20]), GeoDataFrame, geo_name)
+
+    # reindexing both rows and columns
+    assert_object(
+        df.reindex(index=[0, 1, 20], columns=[geo_name]), GeoDataFrame, geo_name
+    )
+    assert_object(df.reindex(index=[0, 1, 20], columns=["value1"]), pd.DataFrame)
+
+
+def test_drop(df):
+    geo_name = df.geometry.name
+    assert_object(df.drop(columns=[geo_name, "geometry2"]), pd.DataFrame)
+    assert_object(df.drop(columns=["value1", "value2"]), GeoDataFrame, geo_name)
+    cols = ["value1", "value2", "geometry2"]
+    assert_object(df.drop(columns=cols), GeoDataFrame, geo_name)
+    # TODO: should this set the geometry column to geometry2 or return a DataFrame?
+    assert_object(df.drop(columns=[geo_name, "value2"]), GeoDataFrame, None)
+    assert_object(df.drop(columns=["value1", "value2", geo_name]), GeoDataFrame, None)
+    # Ideally this would mirror __getitem__ and below would be true
+    # assert_object(df.drop(columns=[geo_name, "value2"]), pd.DataFrame)
+    # assert_object(df.drop(columns=["value1", "value2", geo_name]), pd.DataFrame)
+    assert_object(df.drop(columns=["geometry2", "value2", geo_name]), pd.DataFrame)
+
+
+def test_apply(df):
+    geo_name = df.geometry.name
+
+    def identity(x):
+        return x
+
+    # axis = 0
+    assert_object(df[["value1", "value2"]].apply(identity), pd.DataFrame)
+    assert_object(df[[geo_name, "geometry2"]].apply(identity), GeoDataFrame, geo_name)
+    assert_object(df[[geo_name]].apply(identity), GeoDataFrame, geo_name)
+    assert_object(df[["geometry2", "value1"]].apply(identity), pd.DataFrame)
+    assert_object(df[["geometry2"]].apply(identity), pd.DataFrame)
+    assert_object(df[["value1"]].apply(identity), pd.DataFrame)
+
+    # axis = 0, Series
+    assert_object(df[geo_name].apply(identity), GeoSeries, geo_name)
+    assert_object(df["geometry2"].apply(identity), GeoSeries, "geometry2", crs=crs_osgb)
+    assert_object(df["value1"].apply(identity), pd.Series)
+
+    # axis = 0, Series, no longer geometry
+    assert_object(df[geo_name].apply(lambda x: str(x)), pd.Series)
+    assert_object(df["geometry2"].apply(lambda x: str(x)), pd.Series)
+
+    # axis = 1
+    assert_object(df[["value1", "value2"]].apply(identity, axis=1), pd.DataFrame)
+    assert_object(
+        df[[geo_name, "geometry2"]].apply(identity, axis=1), GeoDataFrame, geo_name
+    )
+    assert_object(df[[geo_name]].apply(identity, axis=1), GeoDataFrame, geo_name)
+    assert_object(df[["geometry2", "value1"]].apply(identity, axis=1), pd.DataFrame)
+    assert_object(df[["geometry2"]].apply(identity, axis=1), pd.DataFrame)
+    assert_object(df[["value1"]].apply(identity, axis=1), pd.DataFrame)
diff --git a/geopandas/tests/test_pandas_methods.py b/geopandas/tests/test_pandas_methods.py
--- a/geopandas/tests/test_pandas_methods.py
+++ b/geopandas/tests/test_pandas_methods.py
@@ -139,8 +139,9 @@ def test_reindex(s, df):
     assert isinstance(res.geometry, GeoSeries)
     assert_frame_equal(res, df[["value1", "geometry"]])
 
-    # TODO df.reindex(columns=['value1', 'value2']) still returns GeoDataFrame,
-    # should it return DataFrame instead ?
+    res = df.reindex(columns=["value1", "value2"])
+    assert type(res) == pd.DataFrame
+    assert_frame_equal(res, df[["value1", "value2"]])
 
 
 def test_take(s, df):
@@ -587,13 +588,12 @@ def test_apply_no_geometry_result(df, crs):
     if crs:
         df = df.set_crs(crs)
     result = df.apply(lambda col: col.astype(str), axis=0)
-    # TODO this should actually not return a GeoDataFrame
-    assert isinstance(result, GeoDataFrame)
+    assert type(result) is pd.DataFrame
     expected = df.astype(str)
     assert_frame_equal(result, expected)
 
     result = df.apply(lambda col: col.astype(str), axis=1)
-    assert isinstance(result, GeoDataFrame)
+    assert type(result) is pd.DataFrame
     assert_frame_equal(result, expected)
 
 
diff --git a/geopandas/tests/test_plotting.py b/geopandas/tests/test_plotting.py
--- a/geopandas/tests/test_plotting.py
+++ b/geopandas/tests/test_plotting.py
@@ -303,7 +303,7 @@ def test_empty_plot(self):
         with pytest.warns(UserWarning):
             ax = s.plot()
         assert len(ax.collections) == 0
-        df = GeoDataFrame([])
+        df = GeoDataFrame([], columns=["geometry"])
         with pytest.warns(UserWarning):
             ax = df.plot()
         assert len(ax.collections) == 0