Repository: pandas-dev/pandas

Issue:
mi.drop(x).get_loc_level(x) returns empty slice (rather than raising KeyError)
#### Code Sample

```python
import pandas as pd

df = pd.DataFrame(dict(value=[0, 1], group=['filled','empty']))
groups = df.groupby('group')

def remove1 (group):
	return group[group.value != 1]['value']
trimmed = groups.apply(remove1)


print("Trimmed.loc['empty']:\n",trimmed.loc['empty'])
```
#### Problem description

Since Pandas 0.23, if a group (ex. 'empty') is emptied by GroupBy.apply(), then accessing the result (ex. trimmed) through trimmed.loc['empty'] returns the error ```KeyError: 'the label [empty] is not in the [index]'``` (full output below). This is possible related to https://github.com/pandas-dev/pandas/issues/21624.

Despite what the error says, printing trimmed.indexes shows that 'empty' is a valid index:
```
MultiIndex(levels=[['empty', 'filled'], [0, 2]],
           labels=[[1, 1], [0, 1]],
           names=['group', None])
```

Full Output:
```
Traceback (most recent call last):
  File ".../python3.6/site-packages/pandas/core/indexing.py", line 1790, in _validate_key
    error()
  File ".../python3.6/site-packages/pandas/core/indexing.py", line 1785, in error
    axis=self.obj._get_axis_name(axis)))
KeyError: 'the label [empty] is not in the [index]'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "pandasExample.py", line 14, in <module>
    print("Trimmed.loc['empty']:\n",trimmed.loc['empty'])
  File ".../python3.6/site-packages/pandas/core/indexing.py", line 1478, in __getitem__
    return self._getitem_axis(maybe_callable, axis=axis)
  File ".../python3.6/site-packages/pandas/core/indexing.py", line 1911, in _getitem_axis
    self._validate_key(key, axis)
  File ".../python3.6/site-packages/pandas/core/indexing.py", line 1798, in _validate_key
    error()
  File ".../python3.6/site-packages/pandas/core/indexing.py", line 1785, in error
    axis=self.obj._get_axis_name(axis)))
KeyError: 'the label [empty] is not in the [index]'
```
#### Expected Output

Prior to 0.23, an empty Series would be returned.

0.22.0 output:
```
Trimmed.loc['empty']:
 Series([], Name: value, dtype: int64)
```

#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit: None
python: 3.6.0.final.0
python-bits: 64
OS: Darwin
OS-release: 17.4.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.23.3
pytest: 3.6.4
pip: 10.0.1
setuptools: 39.2.0
Cython: None
numpy: 1.15.0
scipy: 1.1.0
pyarrow: None
xarray: None
IPython: 6.5.0
sphinx: 1.7.6
patsy: None
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.1
html5lib: 1.0.1
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

</details>



Discussion:
Can you try on master? This seemed to be working for me.
Thanks for the quick response! Am I going to need to clone and build locally, or is the master on Conda or PIP?
0.23.4 was just released so you can try upgrading to that and see if the issue is there (I have not personally), but if not you would clone and build
I upgraded to 0.23.4 and had the same issue. However, it is fixed on master.
Thanks for confirming. Good to know its "fixed," though I'm actually not sure if `empty` really should be part of the resulting indexing given there is nothing being returned for it as part of the apply operation. If so then we could close this out with a test case to ensure this behavior remains in place, but if not we may want to rethink what we allow here.

@jreback any thoughts?
In the original program where I encountered the error, ```GroupBy.apply()``` was being used to reduce a dataset to its outliers, which might or might not exist for a given group. Reverting to the 0.23.0-0.23.4 behavior would mean that I would have to implement a try-catch when going back through the now-trimmed outlier dataset since its ```MultiIndex``` still shows the indexes for emptied groups. If the ```MultiIndex``` returned by ```GroupBy.apply()``` was changed to reflect the behavior, I would then be required to check against the ```MultiIndex```, still adding additional code/complexity.
Understood on the specific problem you are trying to solve but I'm not aware of any other places where this type of behavior is encouraged in the code-base, i.e. it may be non-idiomatic.

@toobaz may have some thoughts on this as well
> I upgraded to 0.23.4 and had the same issue. However, it is fixed on master.

I'm afraid the fact that this works on master is the actual bug. This is related to #19027 .

@HuntJSparra I can understand what you have in mind, but you are simply looking in an index for a value which is not there, and this would only work in the past - and works again in master - because of an implementation glitch. To be honest I don't even see the try-catch as the right way to go: since some groups just don't have outliers, you might want to loop over ``trimmed.index.unique(level='group')`` rather than on all original groups.

In any case, we must understand what happened in master that made this work again.
Simpler way to reproduce:
```
In [2]: pd.MultiIndex.from_product([[0, 1], [2, 3]]).drop(1).get_loc_level(1)
Out[2]: (slice(2, 2, None), Int64Index([], dtype='int64'))

```

Resolving patch:
diff --git a/pandas/core/indexes/multi.py b/pandas/core/indexes/multi.py
--- a/pandas/core/indexes/multi.py
+++ b/pandas/core/indexes/multi.py
@@ -2185,11 +2185,6 @@ def _maybe_to_slice(loc):
 
         if not isinstance(key, tuple):
             loc = self._get_level_indexer(key, level=0)
-
-            # _get_level_indexer returns an empty slice if the key has
-            # been dropped from the MultiIndex
-            if isinstance(loc, slice) and loc.start == loc.stop:
-                raise KeyError(key)
             return _maybe_to_slice(loc)
 
         keylen = len(key)
@@ -2443,14 +2438,21 @@ def convert_indexer(start, stop, step, indexer=indexer, labels=labels):
 
         else:
 
-            loc = level_index.get_loc(key)
-            if isinstance(loc, slice):
-                return loc
-            elif level > 0 or self.lexsort_depth == 0:
-                return np.array(labels == loc, dtype=bool)
+            code = level_index.get_loc(key)
+
+            if level > 0 or self.lexsort_depth == 0:
+                # Desired level is not sorted
+                locs = np.array(labels == code, dtype=bool, copy=False)
+                if not locs.any():
+                    # The label is present in self.levels[level] but unused:
+                    raise KeyError(key)
+                return locs
 
-            i = labels.searchsorted(loc, side='left')
-            j = labels.searchsorted(loc, side='right')
+            i = labels.searchsorted(code, side='left')
+            j = labels.searchsorted(code, side='right')
+            if i == j:
+                # The label is present in self.levels[level] but unused:
+                raise KeyError(key)
             return slice(i, j)
 
     def get_locs(self, seq):


Test patch:
diff --git a/pandas/tests/groupby/test_apply.py b/pandas/tests/groupby/test_apply.py
--- a/pandas/tests/groupby/test_apply.py
+++ b/pandas/tests/groupby/test_apply.py
@@ -271,10 +271,7 @@ def test_apply_chunk_view():
     df = DataFrame({'key': [1, 1, 1, 2, 2, 2, 3, 3, 3],
                     'value': compat.lrange(9)})
 
-    # return view
-    f = lambda x: x[:2]
-
-    result = df.groupby('key', group_keys=False).apply(f)
+    result = df.groupby('key', group_keys=False).apply(lambda x: x[:2])
     expected = df.take([0, 1, 3, 4, 6, 7])
     tm.assert_frame_equal(result, expected)
 
@@ -518,6 +515,19 @@ def test_func(x):
     tm.assert_frame_equal(result2, expected2)
 
 
+def test_groupby_apply_return_empty_chunk():
+    # GH 22221: apply filter which returns some empty groups
+    df = pd.DataFrame(dict(value=[0, 1], group=['filled', 'empty']))
+    groups = df.groupby('group')
+    result = groups.apply(lambda group: group[group.value != 1]['value'])
+    expected = pd.Series([0], name='value',
+                         index=MultiIndex.from_product([['empty', 'filled'],
+                                                        [0]],
+                                                       names=['group', None]
+                                                       ).drop('empty'))
+    tm.assert_series_equal(result, expected)
+
+
 def test_apply_with_mixed_types():
     # gh-20949
     df = pd.DataFrame({'A': 'a a b'.split(), 'B': [1, 2, 3], 'C': [4, 6, 5]})
diff --git a/pandas/tests/indexes/multi/test_indexing.py b/pandas/tests/indexes/multi/test_indexing.py
--- a/pandas/tests/indexes/multi/test_indexing.py
+++ b/pandas/tests/indexes/multi/test_indexing.py
@@ -271,6 +271,10 @@ def test_get_loc_level():
     assert new_index is None
 
     pytest.raises(KeyError, index.get_loc_level, (2, 2))
+    # GH 22221: unused label
+    pytest.raises(KeyError, index.drop(2).get_loc_level, 2)
+    # Unused label on unsorted level:
+    pytest.raises(KeyError, index.drop(1, level=2).get_loc_level, 2, 2)
 
     index = MultiIndex(levels=[[2000], lrange(4)], labels=[np.array(
         [0, 0, 0, 0]), np.array([0, 1, 2, 3])])