Repository: pandas-dev/pandas
Pull request: BUG: loc dropping levels when df has only one row
- [x] closes #10521
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Originally this case was dispatched to xs, which dispatched to ``_get_loc_level``, which raised if a slice is not ``slice(None, None)`` and went back to our original function. If the ``DataFrame`` contains only one row, ``_get_loc_level`` does not raise and hence we drop levels. 

I added len(self.obj) for performance reasons becasue I am not sure if the any check for slices is faster than supressing the IndexingError in general.

Problem statement:
BUG: inconsisten multi-level indexing when levels are dropped
`DF.loc['A', :, 1]` returns a DataFrame with the full MultiIndex if the second level has more than one entry, and a truncated index if the second level has only one entry (same as, e.g., `DF.loc['A',0,1]`). Is this the intended behavior?

Hints from issue discussion:
pls show the frame u r indexing as well as pd.show_versions()

Here is a simplified example. In the real scenario I am using a selector of the form [idx1, :, idx2] where the resulting DataFrame may have one or several values for the second level.

``` python
df = pd.DataFrame({'a': list('abcd'), 'b': ['a','b']*2, 'c': ['a']*4, 'd':0})
df = df.set_index(['a', 'b', 'c']).sortlevel()
print df
print df.loc['a',:,'a']

df_ = df[:1]
print df_
print df_.loc['a',:,'a']

```

```
INSTALLED VERSIONS
------------------
commit: None
python: 2.7.10.final.0
python-bits: 64
OS: Linux
OS-release: 2.6.32-504.23.4.el6.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: C

pandas: 0.16.2
nose: 1.3.7
Cython: 0.21
numpy: 1.9.2
scipy: 0.15.1
statsmodels: 0.5.0
IPython: 3.2.0
sphinx: 1.2.3
patsy: 0.3.0
dateutil: 2.4.2
pytz: 2015.4
bottleneck: None
tables: 3.1.1
numexpr: 2.3.1
matplotlib: 1.4.3
openpyxl: 1.8.5
xlrd: 0.9.3
xlwt: 0.7.5
xlsxwriter: 0.5.7
lxml: 3.4.0
bs4: 4.3.2
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 0.9.7
pymysql: 0.6.6.None
psycopg2: None
```

This is hitting an 'older' piece of code that I haven't really touched (well _afraid_ is more like it :)
that actually should be eliminated because it has results that _might_ drop levels, when in fact that decision should be taken at a higher level.

The solution is to basically use [slicers](http://pandas.pydata.org/pandas-docs/stable/advanced.html#using-slicers), which always give you back the original dimension (the full dims). These are like `.xs` but with consistent semantics.

Note that you MUST fully specify all dimensions (e.g. index & columns)

```
In [5]: df.loc[('a',slice(None),'a'),:]
Out[5]: 
       d
a b c   
a a a  0

In [4]: df_.loc[('a',slice(None),'a'),:]
Out[4]: 
       d
a b c   
a a a  0
```

I would actually write this like this, with the `IndexSlice` helper to make nicer look-and-feel.

```
In [8]: idx = pd.IndexSlice

In [9]: df_.loc[idx['a',:,'a'],:]
Out[9]: 
       d
a b c   
a a a  0
```

So I guess this is a bug. Want to dive in?

I'm happy with the slicer notation, though I hope others don't have to figure this out by backtracking mysterious behavior in their code. 

yep, this is an older wart, which requires removing quite of bit of older code (and then fixing anything that the 'newer' way doesn't support). Not that hard, but requires some effort.

Commit message: Merge branches 'master' and 'master' of https://github.com/pandas-dev/pandas into 10521

 Conflicts:
	doc/source/whatsnew/v1.2.0.rst

Patch to review:
diff --git a/doc/source/whatsnew/v1.2.0.rst b/doc/source/whatsnew/v1.2.0.rst
index 501e2878ab135..2728ab6e72ce4 100644
--- a/doc/source/whatsnew/v1.2.0.rst
+++ b/doc/source/whatsnew/v1.2.0.rst
@@ -637,6 +637,7 @@ Indexing
 - Bug in :meth:`DataFrame.loc` and :meth:`DataFrame.__getitem__`  raising ``KeyError`` when columns were :class:`MultiIndex` with only one level (:issue:`29749`)
 - Bug in :meth:`Series.__getitem__` and :meth:`DataFrame.__getitem__` raising blank ``KeyError`` without missing keys for :class:`IntervalIndex` (:issue:`27365`)
 - Bug in setting a new label on a :class:`DataFrame` or :class:`Series` with a :class:`CategoricalIndex` incorrectly raising ``TypeError`` when the new label is not among the index's categories (:issue:`38098`)
+- Bug in :meth:`DataFrame.loc` dropping levels of :class:`MultiIndex` when :class:`DataFrame` used as input has only one row (:issue:`10521`)
 
 Missing
 ^^^^^^^
diff --git a/pandas/core/indexing.py b/pandas/core/indexing.py
index 6aa031af64833..cb4198e568331 100644
--- a/pandas/core/indexing.py
+++ b/pandas/core/indexing.py
@@ -838,8 +838,10 @@ def _getitem_nested_tuple(self, tup: Tuple):
             if self.name != "loc":
                 # This should never be reached, but lets be explicit about it
                 raise ValueError("Too many indices")
-            with suppress(IndexingError):
-                return self._handle_lowerdim_multi_index_axis0(tup)
+            if len(self.obj) > 1 or not any(isinstance(x, slice) for x in tup):
+                # GH#10521 IndexingError is not raised for slices for objs with one row
+                with suppress(IndexingError):
+                    return self._handle_lowerdim_multi_index_axis0(tup)
 
             # this is a series with a multi-index specified a tuple of
             # selectors
diff --git a/pandas/tests/indexing/multiindex/test_loc.py b/pandas/tests/indexing/multiindex/test_loc.py
index 42525fc575397..489cc9ef54973 100644
--- a/pandas/tests/indexing/multiindex/test_loc.py
+++ b/pandas/tests/indexing/multiindex/test_loc.py
@@ -695,3 +695,13 @@ def test_loc_getitem_index_differently_ordered_slice_none():
         columns=["a", "b"],
     )
     tm.assert_frame_equal(result, expected)
+
+
+def test_loc_getitem_drops_levels_for_one_row_dataframe():
+    # GH#10521
+    df = DataFrame({"a": ["a"], "b": ["b"], "c": ["a"], "d": 0}).set_index(
+        ["a", "b", "c"]
+    )
+    expected = df.copy()
+    result = df.loc["a", :, "a"]
+    tm.assert_frame_equal(result, expected)


Code review comments:
diff_hunk: @@ -695,3 +695,13 @@ def test_loc_getitem_index_differently_ordered_slice_none():
         columns=["a", "b"],
     )
     tm.assert_frame_equal(result, expected)
+
+
+def test_loc_getitem_drops_levels_for_one_row_dataframe():
+    # GH#10521
+    df = DataFrame({"a": ["a"], "b": ["b"], "c": ["a"], "d": 0}).set_index(
path: pandas/tests/indexing/multiindex/test_loc.py
text: @user1:
personal preference: to make it super-obvious what is and isn't being tested -- in particular that `set_index` is not being tested -- id prefer this as

```
mi = MultiIndex.from_product(...)
df = DataFrame({"d": 0}, index=mi)
```

@user1:
this would also be easier to follow if the values and the columns didnt overlap

@author:
Thx, simplified it