Repository: geopandas/geopandas

Issue:
ENH: Carry original schema and use it during to_file
Following the discussion in #1185, I have drafted a PR which saves schema during `read_file` to `gdf.schema` and then during `to_file` checks if the original schema for each column is still applicable. If so, it uses it, if not it infers new one based on the dtype as we do it now.

I am pretty sure that there will be some corner cases which are not covered here, but I wanted to have this PR opened so we can discuss the implementation. But for the cases described in #1185 and earlier in #177, this should work.

```
path = gpd.datasets.get_path('nybb')
gdf = gpd.read_file(path)
gdf.schema  # original schema from fiona

{'properties': OrderedDict([('BoroCode', 'int:4'),
              ('BoroName', 'str:32'),
              ('Shape_Leng', 'float:19.11'),
              ('Shape_Area', 'float:19.11')]),
 'geometry': 'Polygon'}
```

```
gpd.io.file.infer_schema(gdf)
{'geometry': 'MultiPolygon',

 'properties': OrderedDict([('BoroCode', 'int:4'),
              ('BoroName', 'str:32'),
              ('Shape_Leng', 'float:19.11'),
              ('Shape_Area', 'float:19.11')])}
```

On master:
```
gpd.io.file.infer_schema(gdf)

{'geometry': 'MultiPolygon',
 'properties': OrderedDict([('BoroCode', 'int'),
              ('BoroName', 'str'),
              ('Shape_Leng', 'float'),
              ('Shape_Area', 'float')])}
```

Closes #1185 


Discussion:
# [Codecov](https://codecov.io/gh/geopandas/geopandas/pull/1200?src=pr&el=h1) Report
> Merging [#1200](https://codecov.io/gh/geopandas/geopandas/pull/1200?src=pr&el=desc) into [master](https://codecov.io/gh/geopandas/geopandas/commit/1e379027d09e65b55512408e0202e323d2a2e566?src=pr&el=desc) will **decrease** coverage by `0.03%`.
> The diff coverage is `89.47%`.

[![Impacted file tree graph](https://codecov.io/gh/geopandas/geopandas/pull/1200/graphs/tree.svg?width=650&token=6TOPMcCBhB&height=150&src=pr)](https://codecov.io/gh/geopandas/geopandas/pull/1200?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #1200      +/-   ##
==========================================
- Coverage   89.85%   89.81%   -0.04%     
==========================================
  Files          21       21              
  Lines        2040     2052      +12     
==========================================
+ Hits         1833     1843      +10     
- Misses        207      209       +2
```


| [Impacted Files](https://codecov.io/gh/geopandas/geopandas/pull/1200?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [geopandas/geodataframe.py](https://codecov.io/gh/geopandas/geopandas/pull/1200/diff?src=pr&el=tree#diff-Z2VvcGFuZGFzL2dlb2RhdGFmcmFtZS5weQ==) | `97.92% <100%> (+0.02%)` | :arrow_up: |
| [geopandas/io/file.py](https://codecov.io/gh/geopandas/geopandas/pull/1200/diff?src=pr&el=tree#diff-Z2VvcGFuZGFzL2lvL2ZpbGUucHk=) | `91.75% <86.66%> (-1.43%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/geopandas/geopandas/pull/1200?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/geopandas/geopandas/pull/1200?src=pr&el=footer). Last update [1e37902...1c8d36c](https://codecov.io/gh/geopandas/geopandas/pull/1200?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

As explained in https://github.com/geopandas/geopandas/issues/1185#issuecomment-553287077, I have now added two other changes:
1) `infer_schema` now supports pandas Int64 - on master it fails (it might fail elsewhere as well).
2) `from_features` now checks if the conversion from int to float happened due to nan and converts affected columns to Int64.

it is a bit rough, but it could be a way how to preserve schema.
I am not sure this can be done in general to cover all possible use cases.

For example, does the 32 in `'str:32'` denote the max length of the string? If so, you can have a case where in the meantime you change some values in the dataframe that alters the length but not type, and then we might truncate values because our inference just uses "str" (all hypothetical thinking, didn't test). 
Also with numeric types, those can change type of bit-width during some operations.


> infer_schema now supports pandas Int64 - on master it fails (it might fail elsewhere as well).

That's certainly a valuable change!

> from_features now checks if the conversion from int to float happened due to nan and converts affected columns to Int64.

I am not sure we should do this automatically. The nullable integer type of pandas is still experimental, and there are quite some operations that are not yet supported by it.
Hmm, what would be the solution here? Raise a warning that int has been converted to float and one can force int using `.astype('Int64')` (with some consequences)? And support Int64 in infer_schema? That might be cleaner, we are not able to cover all possible cases as you say. 

Resolving patch:
diff --git a/geopandas/io/file.py b/geopandas/io/file.py
--- a/geopandas/io/file.py
+++ b/geopandas/io/file.py
@@ -123,10 +123,10 @@ def to_file(df, filename, driver="ESRI Shapefile", schema=None, **kwargs):
 
 
 def infer_schema(df):
-    try:
-        from collections import OrderedDict
-    except ImportError:
-        from ordereddict import OrderedDict
+    from collections import OrderedDict
+
+    # TODO: test pandas string type and boolean type once released
+    types = {"Int64": "int", "string": "str", "boolean": "bool"}
 
     def convert_type(column, in_type):
         if in_type == object:
@@ -134,7 +134,10 @@ def convert_type(column, in_type):
         if in_type.name.startswith("datetime64"):
             # numpy datetime type regardless of frequency
             return "datetime"
-        out_type = type(np.zeros(1, in_type).item()).__name__
+        if str(in_type) in types:
+            out_type = types[str(in_type)]
+        else:
+            out_type = type(np.zeros(1, in_type).item()).__name__
         if out_type == "long":
             out_type = "int"
         if not _FIONA18 and out_type == "bool":


Test patch:
diff --git a/geopandas/io/tests/test_file.py b/geopandas/io/tests/test_file.py
--- a/geopandas/io/tests/test_file.py
+++ b/geopandas/io/tests/test_file.py
@@ -7,11 +7,13 @@
 
 import fiona
 import numpy as np
+import pandas as pd
 from shapely.geometry import Point, Polygon, box
 
 import geopandas
 from geopandas import GeoDataFrame, read_file
 from geopandas.io.file import fiona_env, _FIONA18
+from geopandas._compat import PANDAS_GE_024
 
 from geopandas.testing import assert_geodataframe_equal, assert_geoseries_equal
 from geopandas.tests.util import PACKAGE_DIR, validate_boro_df
@@ -183,6 +185,17 @@ def test_to_file_types(tmpdir, df_points):
     df.to_file(tempfilename)
 
 
+@pytest.mark.skipif(not PANDAS_GE_024, reason="pandas >= 0.24 needed")
+def test_to_file_int64(tmpdir, df_points):
+    tempfilename = os.path.join(str(tmpdir), "int64.shp")
+    geometry = df_points.geometry
+    df = GeoDataFrame(geometry=geometry)
+    df["data"] = pd.array([1, np.nan] * 5, dtype=pd.Int64Dtype())
+    df.to_file(tempfilename)
+    df_read = GeoDataFrame.from_file(tempfilename)
+    assert_geodataframe_equal(df_read, df, check_dtype=False, check_like=True)
+
+
 def test_to_file_empty(tmpdir):
     input_empty_df = GeoDataFrame()
     tempfilename = os.path.join(str(tmpdir), "test.shp")
diff --git a/geopandas/io/tests/test_file_geom_types_drivers.py b/geopandas/io/tests/test_file_geom_types_drivers.py
--- a/geopandas/io/tests/test_file_geom_types_drivers.py
+++ b/geopandas/io/tests/test_file_geom_types_drivers.py
@@ -287,19 +287,7 @@ def geodataframe(request):
     return request.param
 
 
-@pytest.fixture(
-    params=[
-        "GeoJSON",
-        "ESRI Shapefile",
-        pytest.param(
-            "GPKG",
-            marks=pytest.mark.skipif(
-                (sys.version_info < (3, 0)) and sys.platform.startswith("win"),
-                reason="GPKG tests failing on AppVeyor for Python 2.7",
-            ),
-        ),
-    ]
-)
+@pytest.fixture(params=["GeoJSON", "ESRI Shapefile", "GPKG"])
 def ogr_driver(request):
     return request.param
 
diff --git a/geopandas/io/tests/test_infer_schema.py b/geopandas/io/tests/test_infer_schema.py
--- a/geopandas/io/tests/test_infer_schema.py
+++ b/geopandas/io/tests/test_infer_schema.py
@@ -9,8 +9,12 @@
     Polygon,
 )
 
+import pandas as pd
+import numpy as np
+import pytest
 from geopandas import GeoDataFrame
 from geopandas.io.file import _FIONA18, infer_schema
+from geopandas._compat import PANDAS_GE_024
 
 # Credit: Polygons below come from Montreal city Open Data portal
 # http://donnees.ville.montreal.qc.ca/dataset/unites-evaluation-fonciere
@@ -313,3 +317,15 @@ def test_infer_schema_null_geometry_all():
     # None geometry type in then replaced by 'Unknown'
     # (default geometry type supported by Fiona)
     assert infer_schema(df) == {"geometry": "Unknown", "properties": OrderedDict()}
+
+
+@pytest.mark.skipif(not PANDAS_GE_024, reason="pandas >= 0.24 needed")
+def test_infer_schema_int64():
+    int64col = pd.array([1, np.nan], dtype=pd.Int64Dtype())
+    df = GeoDataFrame(geometry=[city_hall_entrance, city_hall_balcony])
+    df["int64"] = int64col
+
+    assert infer_schema(df) == {
+        "geometry": "Point",
+        "properties": OrderedDict([("int64", "int")]),
+    }