Repository: pandas-dev/pandas
Pull request: BUG GH22858 When creating empty dataframe, only cast int to float if index given
- [X] closes #22858 
- [X] tests added / passed
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Previously, when creating a dataframe with no data of dtype int, the dtype would be changed to float. This is necessary when a predefined number of rows is included as the index parameter, so that they can be filled with nan. However, when no index is passed, this cast is unexpected. This PR changes it so dtype is only altered when necessary.

Problem statement:
BUG/ENH: Bad columns dtype when creating empty DataFrame
#### Code Sample

```python
>>> df = pd.DataFrame(columns=list('ABC'), dtype='int64')
>>> df
Empty DataFrame
Columns: [A, B, C]
Index: []
>>> df.dtypes
A    float64
B    float64
C    float64
dtype: object
```
#### Problem description

When creating a DataFrame with no rows, the presence of a `dtype` argument may convert the columns into `float64`. The problem does not happen if the DataFrame has one or more rows:

```python
>>> df = pd.DataFrame([[1, 2, 3]], columns=list('ABC'), dtype='int64')
>>> df
   A  B  C
0  1  2  3
>>> df.dtypes
A    int64
B    int64
C    int64
dtype: object
```

#### Expected Output

```python
>>> df = pd.DataFrame(columns=list('ABC'), dtype='int64')
>>> df.dtypes
A    int64
B    int64
C    int64
dtype: object
```

#### Output of ``pd.show_versions()``

<details>

commit: None
python: 3.6.6.final.0
python-bits: 64
OS: Linux
OS-release: 4.18.5-arch1-1-ARCH
machine: x86_64
processor: 
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.23.4
pytest: 3.8.0
pip: 10.0.1
setuptools: 40.2.0
Cython: 0.28.5
numpy: 1.15.1
scipy: 1.1.0
pyarrow: 0.9.0
xarray: 0.10.8
IPython: 6.5.0
sphinx: 1.7.9
patsy: 0.5.0
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: 1.2.1
tables: 3.4.4
numexpr: 2.6.8
feather: 0.4.0
matplotlib: 2.2.3
openpyxl: 2.5.5
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.1.0
lxml: 4.2.5
bs4: 4.6.3
html5lib: 1.0.1
sqlalchemy: 1.2.11
pymysql: 0.9.2
psycopg2: None
jinja2: 2.10
s3fs: 0.1.6
fastparquet: 0.1.6
pandas_gbq: None
pandas_datareader: None

</details>

Hints from issue discussion:
This seems to be intended behaviour, as demonstrated by the following test in pandas/tests/frame/test_constructor.py::TestDataFramConstructors::test_constructor_corner
```
df = DataFrame(index=lrange(10), columns=['a', 'b'], dtype=int)
    assert df.values.dtype == np.dtype('float64')
```

The code responsible for this behaviour is found in pandas/core/dtypes/cast.py, on line 1223. Commenting out these two lines causes the above test, and no others, to fail in the pytest suite.
```
if is_integer_dtype(dtype) and isna(value):
    dtype = np.float64
```
I don't feel this is intended behavior, but it may be a rough corner produced by the code you mentioned.

In the issue sample, the columns are empty, therefore, no need to upcast to `float`:

 ```python
>>> df = pd.DataFrame(columns=list('ABC'), dtype='int64')
>>> df
Empty DataFrame
Columns: [A, B, C]
Index: []
```

In the test case you mentioned, though, the `DataFrame` must be filled with `NaN` and therefore `float` is needed:

```python
>>> df = pd.DataFrame(index=range(10), columns=['a', 'b'], dtype=int)
>>> df
    a   b
0 NaN NaN
1 NaN NaN
2 NaN NaN
3 NaN NaN
4 NaN NaN
5 NaN NaN
6 NaN NaN
7 NaN NaN
8 NaN NaN
9 NaN NaN
```

Good point. Theoretically it could be fixed by making the int cast to float only if an lrange is specified. I can try it out later and submit a PR if the tests pass.

Commit message: BUG-22858 Update whatsnew and make test more specific

Patch to review:
diff --git a/doc/source/whatsnew/v0.24.0.txt b/doc/source/whatsnew/v0.24.0.txt
index 9b71ab656920d..d69852d44f6a7 100644
--- a/doc/source/whatsnew/v0.24.0.txt
+++ b/doc/source/whatsnew/v0.24.0.txt
@@ -194,6 +194,7 @@ Other Enhancements
 - :meth:`Index.to_frame` now supports overriding column name(s) (:issue:`22580`).
 - New attribute :attr:`__git_version__` will return git commit sha of current build (:issue:`21295`).
 - Compatibility with Matplotlib 3.0 (:issue:`22790`).
+- A newly constructed empty :class:`DataFrame` of integers will now only be cast to ``float64`` if an index is specified (:issue:`22858`)
 
 .. _whatsnew_0240.api_breaking:
 
diff --git a/pandas/core/dtypes/cast.py b/pandas/core/dtypes/cast.py
index 410e061c895db..27ec841b5dc9f 100644
--- a/pandas/core/dtypes/cast.py
+++ b/pandas/core/dtypes/cast.py
@@ -1220,7 +1220,7 @@ def construct_1d_arraylike_from_scalar(value, length, dtype):
             dtype = dtype.dtype
 
         # coerce if we have nan for an integer dtype
-        if is_integer_dtype(dtype) and isna(value):
+        if is_integer_dtype(dtype) and isna(value) and length:
             dtype = np.float64
         subarr = np.empty(length, dtype=dtype)
         subarr.fill(value)
diff --git a/pandas/tests/frame/test_constructors.py b/pandas/tests/frame/test_constructors.py
index 2f1c9e05a01b0..89faec95edebb 100644
--- a/pandas/tests/frame/test_constructors.py
+++ b/pandas/tests/frame/test_constructors.py
@@ -806,6 +806,9 @@ def test_constructor_corner(self):
         df = DataFrame(index=lrange(10), columns=['a', 'b'], dtype=object)
         assert df.values.dtype == np.object_
 
+        df = DataFrame(columns=['a', 'b'], dtype="int64")
+        assert df.values.dtype == np.dtype('int64')
+
         # does not error but ends up float
         df = DataFrame(index=lrange(10), columns=['a', 'b'], dtype=int)
         assert df.values.dtype == np.dtype('float64')


Code review comments:
diff_hunk: @@ -194,6 +194,7 @@ Other Enhancements
 - :meth:`Index.to_frame` now supports overriding column name(s) (:issue:`22580`).
 - New attribute :attr:`__git_version__` will return git commit sha of current build (:issue:`21295`).
 - Compatibility with Matplotlib 3.0 (:issue:`22790`).
+- A newly constructed empty :class:`DataFrame` of integers will now only be cast to ``float64`` if an index is specified (:issue:`22858`)
path: doc/source/whatsnew/v0.24.0.txt
text: @user1:
Hmm I think the wording `DataFrame of integers` is somewhat misleading as there aren't actually any integers here; the type is just passed explicitly during construction and not subsequently coerced.