Repository: geopandas/geopandas

Issue:
PROPOSAL: Load subset of columns on read file
Related:
- https://gis.stackexchange.com/questions/129414/only-read-specific-attribute-columns-of-a-shapefile-with-geopandas-fiona#129422
- https://gis.stackexchange.com/questions/348064/rasterising-multiple-shapefiles-into-a-single-raster/348065#348065
- #1157 

Thoughts?

Discussion:
snowman2: @brendan-ward, I am thinking about modifying this PR so it enables `ignore_geometry` to work and then moving the field filtering functionality upstream to Fiona [Issue](https://github.com/Toblerity/Fiona/issues/887). Thoughts?
martinfleis: > moving the field filtering functionality upstream to Fiona.

This makes sense to me.
brendan-ward: @snowman2 just so I follow, for this PR you'll stick to filtering fields after fetching all from Fiona, and then once (if?) opt-in fields support lands in Fiona, update here to use that?

That sounds reasonable to me.

I think my only hangup at this point is around the consistency of reference to geometry as a named column within `columns` vs enabled / disabled via additional parameters.  Ideally, the interfaces in geopandas that get columns & geometries from a data source should use similar patterns, but the challenge is that each of the underlying data sources presents those in a different way.

* From a Fiona data source and by way of `from_features` API, there is only ever one (?) geometry, and it always gets named here as `geometry`, so I see your point here about not necessarily wanting to include `geometry` in `columns` parameter.  In this case `geometry` is a special, named column returned outside the context of the columns that are requested.
* From a Parquet data source (via #1180), there may be multiple geometry columns, they may have arbitrary names, so we need to allow `geometry` or any other named geometry column to be referenced in `columns` there (not appropriate to have an extra parameter to include / not include geometry).
* From an SQL data source, the columns (including geometry) are controlled by the underlying SQL query, and the geometry column is noted via a parameter (for decoding, not selection).  But in a sense, the SQL query is similar to the `columns` (including geometry) pattern above.

What do you think about the consistency / convenience / clarity tradeoffs?
snowman2: > for this PR you'll stick to filtering fields after fetching all from Fiona, and then once (if?) opt-in fields support lands in Fiona, update here to use that?

Actually, I was thinking about removing the `columns` argument entirely and recommend the user use the filtering capabilities through Fiona for now. Then, just keep the parts that enable the user to skip loading in the geometry if they so desire via the Fiona `ignore_geometry` argument. That would make this more of a bugfix PR and not introduce any new features.

> What do you think about the consistency / convenience / clarity tradeoffs?

Those are all good points and should be thought through a bit. I am thinking this should be kept track of in the original issue (or maybe make a new one?).
brendan-ward: Thanks for clarifying.  Forwarding `ignore_fields` or `fields` to Fiona seems like a very reasonable thing to do in this PR (which simplifies compatibility with `fields` if and when it lands there).  Thanks also for following this through; apologies if I was pushing for more complexity than appropriate to add at this time.

If you agree that the API consistency issues are something to consider more broadly, a dedicated issue for that might be best.  That way the original issue can close out on merge of this PR.
snowman2: > Forwarding ignore_fields or fields to Fiona seems like a very reasonable thing to do in this PR (which simplifies compatibility with fields if and when it lands there).

Since the `kwargs` are already passed into `fiona.open`, I realized that making the change upstream in `Fiona` would probably be the best place for it. That way independent of the `geopandas` version, you retain the filter behavior if you have the version of Fiona that will hopefully eventually have this capability.

> apologies if I was pushing for more complexity than appropriate to add at this time.

No worries there - you added some insightful thoughts in this PR.

> If you agree that the API consistency issues are something to consider more broadly, a dedicated issue for that might be best. That way the original issue can close out on merge of this PR.

I think that would be a good idea :+1:


snowman2: I think this is ready for review if there are any takers.
snowman2: > Can you add a note to docs about ignore_geometry as well?

Sure
martinfleis: Thank you @snowman2! I'll wait if @brendan-ward has any additional comments, but in my eyes it is ready to be merged.
jorisvandenbossche: @snowman2 thanks for working on this!

One more thing: while I like the enhancement to `read_file`, the fact that `GeoDataFrame.from_features` now potentially does not return a GeoDataFrame feels a bit wrong to me (since it's a GeoDataFrame class method to construct a GeoDataFrame). 

So I am wondering, can't we handle the "ignore_geometry" fully inside `read_file` ? In the end, if ignoring the geometry, `GeoDataFrame.from_features` is doing a lot that is not needed, and reading the properties from the fiona collection should be something simple as:

```
pd.DataFrame([record["properties"] for record in f_filt], columns=columns) 
```

snowman2: > One more thing: while I like the enhancement to `read_file`, the fact that `GeoDataFrame.from_features` now potentially does not return a GeoDataFrame feels a bit wrong to me (since it's a GeoDataFrame class method to construct a GeoDataFrame).
> 
> So I am wondering, can't we handle the "ignore_geometry" fully inside `read_file` ? In the end, if ignoring the geometry, `GeoDataFrame.from_features` is doing a lot that is not needed, and reading the properties from the fiona collection should be something simple as:
> 
> ```
> pd.DataFrame([record["properties"] for record in f_filt], columns=columns) 
> ```

I agree with you on that. I just made the change.

snowman2: Alright, I think this has addressed the most recent review comments. It is now ready for another round of reviews.

Patch:
diff --git a/doc/source/io.rst b/doc/source/io.rst
index 4d07c8a0..304b7976 100644
--- a/doc/source/io.rst
+++ b/doc/source/io.rst
@@ -116,6 +116,31 @@ or a slice object.
         rows=slice(10, 20),
     )
 
+Field/Column Filters
+^^^^^^^^^^^^^^^^^^^^
+
+Load in a subset of fields from the file:
+
+.. note:: Requires Fiona 1.8+
+
+.. code-block:: python
+
+    gdf = geopandas.read_file(
+        geopandas.datasets.get_path("naturalearth_lowres"),
+        ignore_fields=["iso_a3", "gdp_md_est"],
+    )
+
+Skip loading geometry from the file:
+
+.. note:: Requires Fiona 1.8+
+.. note:: Returns :obj:`pandas.DataFrame`
+
+.. code-block:: python
+
+    pdf = geopandas.read_file(
+        geopandas.datasets.get_path("naturalearth_lowres"),
+        ignore_geometry=True,
+    )
 
 
 Writing Spatial Data
diff --git a/geopandas/io/file.py b/geopandas/io/file.py
index 73b48a98..26f4243d 100644
--- a/geopandas/io/file.py
+++ b/geopandas/io/file.py
@@ -44,23 +44,23 @@ def read_file(filename, bbox=None, mask=None, rows=None, **kwargs):
 
     Parameters
     ----------
-    filename: str, path object or file-like object
+    filename : str, path object or file-like object
         Either the absolute or relative path to the file or URL to
         be opened, or any object with a read() method (such as an open file
         or StringIO)
-    bbox: tuple | GeoDataFrame or GeoSeries | shapely Geometry, default None
+    bbox : tuple | GeoDataFrame or GeoSeries | shapely Geometry, default None
         Filter features by given bounding box, GeoSeries, GeoDataFrame or a
         shapely geometry. CRS mis-matches are resolved if given a GeoSeries
         or GeoDataFrame. Cannot be used with mask.
-    mask: dict | GeoDataFrame or GeoSeries | shapely Geometry, default None
+    mask : dict | GeoDataFrame or GeoSeries | shapely Geometry, default None
         Filter for features that intersect with the given dict-like geojson
         geometry, GeoSeries, GeoDataFrame or shapely geometry.
         CRS mis-matches are resolved if given a GeoSeries or GeoDataFrame.
         Cannot be used with bbox.
-    rows: int or slice, default None
+    rows : int or slice, default None
         Load in specific rows by passing an integer (first `n` rows) or a
         slice() object.
-    **kwargs:
+    **kwargs :
         Keyword args to be passed to the `open` or `BytesCollection` method
         in the fiona library when opening the file. For more information on
         possible keywords, type:
@@ -72,7 +72,8 @@ def read_file(filename, bbox=None, mask=None, rows=None, **kwargs):
 
     Returns
     -------
-    :obj:`geopandas.GeoDataFrame`
+    :obj:`geopandas.GeoDataFrame` or :obj:`pandas.DataFrame` :
+        If `ignore_geometry=True` a :obj:`pandas.DataFrame` will be returned.
 
     Notes
     -----
@@ -128,11 +129,16 @@ def read_file(filename, bbox=None, mask=None, rows=None, **kwargs):
                 f_filt = features.filter(bbox=bbox, mask=mask)
             else:
                 f_filt = features
+            # get list of columns
+            columns = list(features.schema["properties"])
+            if kwargs.get("ignore_geometry", False):
+                return pd.DataFrame(
+                    [record["properties"] for record in f_filt], columns=columns
+                )
 
-            columns = list(features.meta["schema"]["properties"]) + ["geometry"]
-            gdf = GeoDataFrame.from_features(f_filt, crs=crs, columns=columns)
-
-    return gdf
+            return GeoDataFrame.from_features(
+                f_filt, crs=crs, columns=columns + ["geometry"]
+            )
 
 
 def to_file(


Test patch:
diff --git a/geopandas/io/tests/test_file.py b/geopandas/io/tests/test_file.py
index e564e8ab..443c7b01 100644
--- a/geopandas/io/tests/test_file.py
+++ b/geopandas/io/tests/test_file.py
@@ -414,11 +414,35 @@ def test_read_file_filtered__rows_bbox__polygon(df_nybb):
     assert filtered_df_shape == (1, 5)
 
 
-def read_file_filtered_rows_invalid():
+def test_read_file_filtered_rows_invalid():
     with pytest.raises(TypeError):
         read_file(geopandas.datasets.get_path("nybb"), rows="not_a_slice")
 
 
+@pytest.mark.skipif(
+    LooseVersion(fiona.__version__) < LooseVersion("1.8"),
+    reason="Ignore geometry only available in Fiona 1.8",
+)
+def test_read_file__ignore_geometry():
+    pdf = geopandas.read_file(
+        geopandas.datasets.get_path("naturalearth_lowres"), ignore_geometry=True,
+    )
+    assert "geometry" not in pdf.columns
+    assert isinstance(pdf, pd.DataFrame) and not isinstance(pdf, geopandas.GeoDataFrame)
+
+
+@pytest.mark.skipif(
+    LooseVersion(fiona.__version__) < LooseVersion("1.8"),
+    reason="Ignore fields only available in Fiona 1.8",
+)
+def test_read_file__ignore_all_fields():
+    gdf = geopandas.read_file(
+        geopandas.datasets.get_path("naturalearth_lowres"),
+        ignore_fields=["pop_est", "continent", "name", "iso_a3", "gdp_md_est"],
+    )
+    assert gdf.columns.tolist() == ["geometry"]
+
+
 def test_read_file_filtered_with_gdf_boundary(df_nybb):
     full_df_shape = df_nybb.shape
     nybb_filename = geopandas.datasets.get_path("nybb")