Repository: pandas-dev/pandas
Pull request: TST: new test for subset of a MultiIndex dtype
Tried unsuccessfully reproducing Issue #20757 on pandas: 0.23.0 and 0.26.0.dev0+734.g0de99558b.dirty. That's why I decided to keep the input data from Issue #20757 unchanged.

- [x] closes #20757
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Problem statement:
Selecting "subsets" of a `MultiIndex` `DataFrame` sometimes changes `dtypes`
#### Code Sample, a copy-pastable example if possible
Definition of data and columns:
<details>

```python
from numpy import nan
data = [['n',  1,  0,  False,  2,  1,  False,  0,  0,  False,  2,  0,  False,  0,  1,  False,  1,  1,  False,  'o',
  1521734085.289453,  'p',  3233,  1521734085.289494]]
columns = [('a', 'd', 'i', nan, nan),
 ('a', 'd', 'j', 0.0, 'k'),
 ('a', 'd', 'j', 0.0, 'l'),
 ('a', 'd', 'j', 0.0, 'm'),
 ('a', 'd', 'j', 1.0, 'k'),
 ('a', 'd', 'j', 1.0, 'l'),
 ('a', 'd', 'j', 1.0, 'm'),
 ('a', 'd', 'j', 2.0, 'k'),
 ('a', 'd', 'j', 2.0, 'l'),
 ('a', 'd', 'j', 2.0, 'm'),
 ('a', 'd', 'j', 3.0, 'k'),
 ('a', 'd', 'j', 3.0, 'l'),
 ('a', 'd', 'j', 3.0, 'm'),
 ('a', 'd', 'j', 4.0, 'k'),
 ('a', 'd', 'j', 4.0, 'l'),
 ('a', 'd', 'j', 4.0, 'm'),
 ('a', 'd', 'j', 5.0, 'k'),
 ('a', 'd', 'j', 5.0, 'l'),
 ('a', 'd', 'j', 5.0, 'm'),
 ('b', 'f', nan, nan, nan),
 ('b', 'h', nan, nan, nan),
 ('c', 'e', nan, nan, nan),
 ('c', 'g', nan, nan, nan),
 ('c', 'h', nan, nan, nan)]
```

</details>

```python
pd.DataFrame(data, columns=pd.MultiIndex.from_tuples(columns)).dtypes.a.d.i
# object

pd.DataFrame(data, columns=pd.MultiIndex.from_tuples(columns)).a.d.i.dtypes
# float64
```

this causes for example:

```python
pd.DataFrame(np.array(data), columns=pd.MultiIndex.from_tuples(columns)).a.d.i 
# "n", dtype: object

pd.DataFrame(data, columns=pd.MultiIndex.from_tuples(columns)).a.d.i 
#  nan, dtype: float32
```

#### Problem description
I think the example is self explaining

#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.12.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-119-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: en_US.utf8
LANG: en_US.UTF-8
LOCALE: None.None

pandas: 0.23.0.dev0+38.g6552718
pytest: 2.8.7
pip: 9.0.1
setuptools: 20.7.0
Cython: 0.23.4
numpy: 1.14.2
scipy: 1.0.0
pyarrow: None
xarray: None
IPython: 5.5.0
sphinx: 1.3.6
patsy: 0.4.1
dateutil: 2.7.2
pytz: 2018.3
blosc: None
bottleneck: None
tables: 3.2.2
numexpr: 2.6.4
feather: None
matplotlib: 2.1.2
openpyxl: 2.3.0
xlrd: 0.9.4
xlwt: 0.7.5
xlsxwriter: 0.7.3
lxml: 3.5.0
bs4: 4.4.1
html5lib: 0.9999999
sqlalchemy: 1.0.11
pymysql: None
psycopg2: 2.6.1 (dt dec mx pq3 ext lo64)
jinja2: 2.8
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

</details>

Hints from issue discussion:
This looks fixed on master; could use a test:

```
In [110]: pd.DataFrame(data, columns=pd.MultiIndex.from_tuples(columns)).dtypes.a.d.i
     ...:
Out[110]:
NaN  NaN    object
dtype: object

In [111]: pd.DataFrame(data, columns=pd.MultiIndex.from_tuples(columns)).a.d.i.dtypes
     ...:
Out[111]:
NaN  NaN    object
dtype: object
```

Commit message: change input data

Patch to review:
diff --git a/pandas/tests/test_multilevel.py b/pandas/tests/test_multilevel.py
index 79c9fe2b60bd9..a09b7cacc5442 100644
--- a/pandas/tests/test_multilevel.py
+++ b/pandas/tests/test_multilevel.py
@@ -1932,6 +1932,69 @@ def test_repeat(self):
         m_df = Series(data, index=m_idx)
         assert m_df.repeat(3).shape == (3 * len(data),)
 
+    def test_subsets_multiindex_dtype(self):
+        # GH 20757
+        data = [
+            [
+                "n",
+                1,
+                0,
+                False,
+                2,
+                1,
+                False,
+                0,
+                0,
+                False,
+                2,
+                0,
+                False,
+                0,
+                1,
+                False,
+                1,
+                1,
+                False,
+                "o",
+                1521734085.289453,
+                "p",
+                3233,
+                1521734085.289494,
+            ]
+        ]
+
+        columns = [
+            ("a", "d", "i", np.nan, np.nan),
+            ("a", "d", "j", 0.0, "k"),
+            ("a", "d", "j", 0.0, "l"),
+            ("a", "d", "j", 0.0, "m"),
+            ("a", "d", "j", 1.0, "k"),
+            ("a", "d", "j", 1.0, "l"),
+            ("a", "d", "j", 1.0, "m"),
+            ("a", "d", "j", 2.0, "k"),
+            ("a", "d", "j", 2.0, "l"),
+            ("a", "d", "j", 2.0, "m"),
+            ("a", "d", "j", 3.0, "k"),
+            ("a", "d", "j", 3.0, "l"),
+            ("a", "d", "j", 3.0, "m"),
+            ("a", "d", "j", 4.0, "k"),
+            ("a", "d", "j", 4.0, "l"),
+            ("a", "d", "j", 4.0, "m"),
+            ("a", "d", "j", 5.0, "k"),
+            ("a", "d", "j", 5.0, "l"),
+            ("a", "d", "j", 5.0, "m"),
+            ("b", "f", np.nan, np.nan, np.nan),
+            ("b", "h", np.nan, np.nan, np.nan),
+            ("c", "e", np.nan, np.nan, np.nan),
+            ("c", "g", np.nan, np.nan, np.nan),
+            ("c", "h", np.nan, np.nan, np.nan),
+        ]
+
+        columns = pd.MultiIndex.from_tuples(columns)
+        df = pd.DataFrame(data, columns=columns)
+
+        tm.assert_equal(df.dtypes.a.d.i, df.a.d.i.dtypes)
+
 
 class TestSorted(Base):
     """ everything you wanted to test about sorting """


Code review comments:
diff_hunk: @@ -1932,6 +1932,69 @@ def test_repeat(self):
         m_df = Series(data, index=m_idx)
         assert m_df.repeat(3).shape == (3 * len(data),)
 
+    def test_subsets_multiindex_dtype(self):
+        # GH 20757
+        data = [
+            [
+                "n",
+                1,
+                0,
+                False,
+                2,
+                1,
+                False,
+                0,
+                0,
+                False,
+                2,
+                0,
+                False,
+                0,
+                1,
+                False,
+                1,
+                1,
+                False,
+                "o",
+                1521734085.289453,
+                "p",
+                3233,
+                1521734085.289494,
+            ]
+        ]
+
+        columns = [
+            ("a", "d", "i", np.nan, np.nan),
+            ("a", "d", "j", 0.0, "k"),
+            ("a", "d", "j", 0.0, "l"),
+            ("a", "d", "j", 0.0, "m"),
+            ("a", "d", "j", 1.0, "k"),
+            ("a", "d", "j", 1.0, "l"),
+            ("a", "d", "j", 1.0, "m"),
+            ("a", "d", "j", 2.0, "k"),
+            ("a", "d", "j", 2.0, "l"),
+            ("a", "d", "j", 2.0, "m"),
+            ("a", "d", "j", 3.0, "k"),
+            ("a", "d", "j", 3.0, "l"),
+            ("a", "d", "j", 3.0, "m"),
+            ("a", "d", "j", 4.0, "k"),
+            ("a", "d", "j", 4.0, "l"),
+            ("a", "d", "j", 4.0, "m"),
+            ("a", "d", "j", 5.0, "k"),
+            ("a", "d", "j", 5.0, "l"),
+            ("a", "d", "j", 5.0, "m"),
+            ("b", "f", np.nan, np.nan, np.nan),
+            ("b", "h", np.nan, np.nan, np.nan),
+            ("c", "e", np.nan, np.nan, np.nan),
+            ("c", "g", np.nan, np.nan, np.nan),
+            ("c", "h", np.nan, np.nan, np.nan),
+        ]
+
+        columns = pd.MultiIndex.from_tuples(columns)
+        df = pd.DataFrame(data, columns=columns)
+
+        tm.assert_equal(df.dtypes.a.d.i, df.a.d.i.dtypes)
path: pandas/tests/test_multilevel.py
text: @user1:
can you use
```
result=
expected=
tm.assert_frame_equal(result, equal)
```


@author:
Thanks for the feedback, @user1! Currently using `tm.assert_series_equal(result, expected)`.

diff_hunk: @@ -1932,6 +1932,69 @@ def test_repeat(self):
         m_df = Series(data, index=m_idx)
         assert m_df.repeat(3).shape == (3 * len(data),)
 
+    def test_subsets_multiindex_dtype(self):
+        # GH 20757
+        data = [
+            [
+                "n",
+                1,
+                0,
+                False,
+                2,
+                1,
+                False,
+                0,
+                0,
+                False,
path: pandas/tests/test_multilevel.py
text: @user1:
can you make this a bit more programatic, rather than hardcoding these values, it would  be easier to copy-paste

@author:
Thanks for the feedback, @user1! Reproduced the same behaviour with simpler input data.