Repository: pandas-dev/pandas

Issue:
Proposal to change behaviour with .loc and missing keys
```python
In [2]: pd.Series([1, 2, 3]).loc[[2,3]]
Out[2]: 
2    3.0
3    NaN
dtype: float64

In [3]: pd.Series([1, 2, 3]).loc[[3]]
[...]
KeyError: 'None of [[3]] are in the [index]'
```
#### Problem description

Although coherent (except for some unfortunate side-effects - some of them below) with the [docs](http://pandas.pydata.org/pandas-docs/stable/indexing.html#selection-by-label) where they say "At least 1 of the labels for which you ask, must be in the index or a ``KeyError`` will be raised!", the current behavior is - I claim - a terrible choice for both developers and users.

There are (at least) three ways to behave with missing labels:
1. you raise an error if requested at least one missing label
2. you raise an error if requested _only_ missing labels
2a ... while  if at least one label is present, missing labels become ``NaN`` __(current)__
2b. ... while if at least one label is present, missing labels are silently dropped
3. you never raise an error for missing labels
3a ... and they become ``NaN``
3b. ... and they are silently dropped

#### For developers

Options 1. and 3. are both much easier to implement, because in both cases you can reduce the question "am I going to get an error?" in smaller pieces - e.g. when indexing a ``MultiIndex``, you will get an error if you get an error on any of its levels. Option 2. is instead more complicated (and computationally expensive), because you need to first aggregate in some way across levels/axes, and only then can you decide whether to raise an error or not. Several incoherences came out as a consequence of this choice, some of them still unsolved, such as #15452, [this](https://github.com/pandas-dev/pandas/pull/15615#issuecomment-287826997), the fact that ``pd.Series(range(2)).loc[[]]`` does not raise, and the fact that ``pd.DataFrame.ix[[missing_label]]`` doesn't either.

#### Other consequences of 2.

Additionally, it was decided that the behavior with missing labels would be to introduce ``NaN``s (rather than to drop them), and I think this was also not a good choice (and indeed partial indexing ``MultiIndex``es does not behave this way - it couldn't). I think it is also undocumented.

And finally, since the above wouldn't always tell you what to do when there are missing labels in a ``MultiIndex``, it was decided that ``.loc`` would rather behave as ``.reindex`` when there are missing _and incomplete_ labels, which is totally unexpected and, I think, undocumented.

Notice that these further issues (and more in general, the question "what to do when some labels are missing and you are not raising an error") would partially still hold with 3, but could be dealt with, I think, more elegantly.

#### For users

I think the current behavior is annoying to users not just because of those "__Other consequences__", but also because it is more complicated to describe in terms of set operation on labels/indices. For instance, with options 1. and 3.

``pd.concat([chunk.loc[something] for chunk in chunks])``

and

``pd.concat(chunks).loc[something]``

both return the same result (or raise). Instead with 2. it actually depends on how missing labels are distributed across chunks.

#### (Why this?)

It is worth understanding why 2. was picked in the first place, and I think the answer is "to be coherent with intervals". But I think it's not worth the damage - after all, an iterable and an interval are different objects. And moreover, introducing ``NaN``s for missing labels is anyway incoherent with intervals.

#### Backward incompatibility

Option 1. is, I think, the best, because it is also coherent with numpy's behavior with out-of-bounds indices (e.g. ``np.array([[1,2], [3,4]])[0,[1,3]]`` raises an ``IndexError``).

But while clearly both 1. and 3. could break some existing code, 3. would be better from this point of view, in the sense that it would break only code _assuming_ that an error is raised. Although one might even claim that 1., by breaking code which looks for missing labels, can help discover bugs in user code (not a great argument, I know).

So overall I am not sure about what we should pick between 1. and 3. But I really think we should leave 2., and that the later it is done, the worse. @jreback , @jorisvandenbossche if you want to tell me your thoughts about this, I can elaborate on what we could do with the "__Other consequences__" in the desired option.

Then if you approve the change, I'm willing to help in implementing it.


Discussion:
(another advantage of 3. would be that it's easier to implement - but still, I think we would want to change the behaviour on missing labels: namely, to drop them)
xref #10549, #10695
xref (slightly) to https://github.com/pandas-dev/pandas/issues/9213, https://github.com/pandas-dev/pandas/issues/9595
@toobaz if you'd pull any nuggets from those would be helpful 
Right: I think #10549 is, as I commented, not really a bug as long as __2.__ is the rule. The reporter is expecting a ``reindex``-like behavior, but then he rightly mentions the fact that it would be undefined for partial indexing, and so he states that he would have expected an error instead. He is in some sense asking for option __1.__ above, but the bug is instead quite orthogonal to the choice between __2.__ and __3.__ (and in both cases, I suggest we solve the problem by stopping inserting ``NaN``s, and we drop missing labels instead).

Of #10695, I clearly don't like the proposal to add a new indexer, but apart from this they are just asking for option __1.__ above, and I think they are right in preferring it to the current state of things.
(I _can_ try to see if it is feasible to have a ``strict`` argument for ``.loc`` which switches from __1.__ to __3.__... but I'm not sure it would be easy to implement)

By the way: I guess @shoyer might be interested too

Worth mentioning that __1.__ is more coherent with ``.loc.__setitem__``, which raises for missing keys (in lists). But then, it does add individual missing keys, so this is again not a very strong argument (I rather think that we should fix ``.loc.__setitem__`` and make it add missing labels if we decide to pick __3.__ - unless with partial indexing of ``MultiIndexes``, where it should probably drop them).
yeah I would be ok with 1). The reason for the current behavior are twofold:

1) ``.loc`` and ``.reindex`` should be equivalent (though ``.reindex`` cannot be an lvalue), because it was confusing when to use these. IOW lots of complaints about I want to overwrite this label wether it exists or not (that's why partial setting works).

note that I think we are going to get rid of all of the expanding stuff in pandas2 anyhow. You will have to explicitly ``.reindex``  first, then assign.

2) interactive work with a long list of labels could be confusing when 1 is missing on accident. IOW a typo. Though the counter-argument here is that this error should be raised immediately.

So the driving force was actually 1).
> 3. you never raise an error for missing labels

I think this third option needs clarification, as you can still either drop missing labels or introduce NaNs (so in that regard maybe 4 options or a 3a and 3b).

> [@jreback] note that I think we are going to get rid of all of the expanding stuff in pandas2 anyhow. You will have to explicitly .reindex first, then assign.

I don't think this is true? (or is there an issue on the pandas2 tracker for this?) For example being to assign one or multiple columns is an expanding indexing operation on assignment, and not an ability we want to loose?

@jreback When you say "I would be ok with 1)", you are OK with `loc` and `reindex` no longer being equivalent? And also with the difference between getting (no enlargement) and setting (enlargement allowed)? As those would be the consequence of choosing for option 1. 

---

If we would start fresh, I would, with my current knowledge, be in favor of `loc` being strict for getting (no missing labels allowed), i.e. option 1. 

The question for me is more: can we justify such a breaking change? (as it will break people's code) Are there ways to ease a transition? (I think in principle we can first have a warning in case it will raise an error in a next release?)
cc @pandas-dev/pandas-core 
> I don't think this is true? (or is there an issue on the pandas2 tracker for this?) For example being to assign one or multiple columns is an expanding indexing operation on assignment, and not an ability we want to loose?

@jorisvandenbossche I am only talking about row-wise expansion, which is the obvious issue. column assignment of course is unaffected. It *possible* that this will still be ok (the issue is can performant ``.appends`` be done. https://github.com/pandas-dev/pandas2/issues/53), the main issue is the dtype inference / changes. So this would be an error.

```
In [1]: s = Series([1,2,3])

In [2]: s
Out[2]: 
0    1
1    2
2    3
dtype: int64

In [3]: s.loc[3] = 'foo'

In [4]: s
Out[4]: 
0      1
1      2
2      3
3    foo
dtype: object

```

> The question for me is more: can we justify such a breaking change? (as it will break people's code) Are there ways to ease a transition? (I think in principle we can first have a warning in case it will raise an error in a next release?)

I think a warning is possible.
> @jreback When you say "I would be ok with 1)", you are OK with loc and reindex no longer being equivalent? And also with the difference between getting (no enlargement) and setting (enlargement allowed)? As those would be the consequence of choosing for option 1.

yes I think moving back to a strict separation between ``.reindex`` and ``.loc`` is fine. setting enlargement should be unaffected (and can kick the can down the road about whether to allow it).
getting enlargement by-definition will now fail (as its essentially a ``.reindex`` with missing labels)


@jorisvandenbossche 

> I think this third option needs clarification, as you can still either drop missing labels or introduce NaNs (so in that regard maybe 4 options or a 3a and 3b).

how is 3b different from 2?

xref https://github.com/pandas-dev/pandas2/issues/32 proposing disallowing setting with (row) enlargement
And actually https://github.com/pandas-dev/pandas2/issues/21 is related to the proposal here (or at least to option 1)
I would certainly pick (1) if starting from scratch (I did for xarray), but it's a serious breaking change so it's probably best to save it for pandas 2.0.

We discussed this in other issues, but I think we need symmetric behavior for `.loc` with unrecognized keys (for both getitem and setitem). So if we pick (1), we should be sure we have matching behavior prohibiting setting and enlarging, too.

(3) seems like a strict improvement over what we have now, so I would definitely go for it.
> (3) seems like a strict improvement over what we have now, so I would definitely go for it.

@shoyer What do you mean with 3 ? (see my previous question regarding clarification of this option)

>  I think we need symmetric behavior for .loc with unrecognized keys (for both getitem and setitem)

I don't see how this is strictly possible. Assigning a new, non-existant column (`df['A']`) should work, getting a non-existant column with `df['A']` should still raise a KeyError (or would you let `.loc` differ here?)



> @shoyer What do you mean with 3 ? (see my previous question regarding clarification of this option)

I meant (3) with introducing NaNs. Although it does indeed get really messy, e.g., with MultiIndex, so arguably we aren't even doing this consistently already.

> > I think we need symmetric behavior for .loc with unrecognized keys (for both getitem and setitem)
> I don't see how this is strictly possible. Assigning a new, non-existant column (df['A']) should work, getting a non-existant column with df['A'] should still raise a KeyError (or would you let .loc differ here?)

Sorry, yes, I meant all row indexing, and probably most but not all column indexing. We will need an exception to the rules for setting new columns, at least one at a time.
One possibility is to limit this discussion to only *getting*. 
For the setting, there is clearly still a lot more discussion needed (which kind of row / column enlargment would we like to allow? Which not?), and also a lot more code that will be broken by this (things as `s[..]` and `df.loc[..]` are widely used for enlarging series/frames I think). This will not so easily be achieved, and certainly needs to wait until pandas 2.

If we only speak about getting, eg

```
In [4]: df = pd.DataFrame({'A':[1,2], 'B':[3,4]})

In [5]: df.loc[:,['A', 'C']]
Out[5]: 
   A   C
0  1 NaN
1  2 NaN
```
I think something like that is a lot less common to be intended, and more likely to be an user error. 
Furthermore, for getting you already have the alternative of `reindex` if you want this behaviour. 
And thus (with a good deprecation strategy, I also think a warning should be possible, although maybe complicated to implement) more realistic to tackle on the shorter term?
@toobaz see the referenced issues (https://github.com/pandas-dev/pandas2/issues/32 and https://github.com/pandas-dev/pandas2/issues/21)
> for getting you already have the alternative of reindex

(Replying in general to comparisons of ``loc`` and ``reindex``, also in other comments): I can understand that on the dev side the two are similar, and that past discussions were based on this comparison... but I think for the typical user they are completely different beasts both conceptually (one is an indexer, the other is a modifier that returns a copy) and in the docs.

That said, I agree to keep the setting discussion separated. And I agree with @jorisvandenbossche that with columns it is even more true that missing labels are usually symptoms of an error.

> I meant (3) with introducing NaNs

If we go for (3), is there any real downside in just dropping the missing labels?! Because the downside of introducing NaNs is, as you (@shoyer) mention, that we don't know how to extend to partial indexing of MultiIndex, and also the undesired type changes when for instance working with ints or bools. And also that, again, one does not expect a getter indexer to expand the index - that is, to "get" something which is not there.
> how is 3b different from 2?

2 raises if asked a list of all missing labels, 3b would return an all-NaN object.
> 2 would raise if asked a list of all missing labels, 3b would return an all-NaN object.

so what is 3a then?
@toobaz you can feel free to actually edit the top-section (of options)
> so what is 3a then?

Drop the missing labels silently, without introducing NaNs (which is actually what MultiIndex does when indexing certain levels, as consequence of not being able to introduce NaNs)
(Updated the description)

Resolving patch:
diff --git a/pandas/core/indexing.py b/pandas/core/indexing.py
--- a/pandas/core/indexing.py
+++ b/pandas/core/indexing.py
@@ -1419,13 +1419,33 @@ def _has_valid_type(self, key, axis):
             if isinstance(key, tuple) and isinstance(ax, MultiIndex):
                 return True
 
-            # TODO: don't check the entire key unless necessary
-            if (not is_iterator(key) and len(key) and
-                    np.all(ax.get_indexer_for(key) < 0)):
+            if not is_iterator(key) and len(key):
 
-                raise KeyError(u"None of [{key}] are in the [{axis}]"
-                               .format(key=key,
-                                       axis=self.obj._get_axis_name(axis)))
+                # True indicates missing values
+                missing = ax.get_indexer_for(key) < 0
+
+                if np.any(missing):
+                    if len(key) == 1 or np.all(missing):
+                        raise KeyError(
+                            u"None of [{key}] are in the [{axis}]".format(
+                                key=key, axis=self.obj._get_axis_name(axis)))
+                    else:
+
+                        # we skip the warning on Categorical/Interval
+                        # as this check is actually done (check for
+                        # non-missing values), but a bit later in the
+                        # code, so we want to avoid warning & then
+                        # just raising
+                        _missing_key_warning = textwrap.dedent("""
+                        Passing list-likes to .loc or [] with any missing label will raise
+                        KeyError in the future, you can use .reindex() as an alternative.
+
+                        See the documentation here:
+                        http://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike""")  # noqa
+
+                        if not (ax.is_categorical() or ax.is_interval()):
+                            warnings.warn(_missing_key_warning,
+                                          FutureWarning, stacklevel=5)
 
             return True
 
diff --git a/pandas/core/series.py b/pandas/core/series.py
--- a/pandas/core/series.py
+++ b/pandas/core/series.py
@@ -691,7 +691,7 @@ def _get_with(self, key):
 
             if key_type == 'integer':
                 if self.index.is_integer() or self.index.is_floating():
-                    return self.reindex(key)
+                    return self.loc[key]
                 else:
                     return self._get_values(key)
             elif key_type == 'boolean':
diff --git a/pandas/io/formats/excel.py b/pandas/io/formats/excel.py
--- a/pandas/io/formats/excel.py
+++ b/pandas/io/formats/excel.py
@@ -356,7 +356,21 @@ def __init__(self, df, na_rep='', float_format=None, cols=None,
             self.styler = None
         self.df = df
         if cols is not None:
-            self.df = df.loc[:, cols]
+
+            # all missing, raise
+            if not len(Index(cols) & df.columns):
+                raise KeyError(
+                    "passes columns are not ALL present dataframe")
+
+            # deprecatedin gh-17295
+            # 1 missing is ok (for now)
+            if len(Index(cols) & df.columns) != len(cols):
+                warnings.warn(
+                    "Not all names specified in 'columns' are found; "
+                    "this will raise a KeyError in the future",
+                    FutureWarning)
+
+            self.df = df.reindex(columns=cols)
         self.columns = self.df.columns
         self.float_format = float_format
         self.index = index


Test patch:
diff --git a/pandas/tests/indexing/test_categorical.py b/pandas/tests/indexing/test_categorical.py
--- a/pandas/tests/indexing/test_categorical.py
+++ b/pandas/tests/indexing/test_categorical.py
@@ -111,7 +111,8 @@ def test_loc_listlike(self):
         assert_frame_equal(result, expected, check_index_type=True)
 
         # not all labels in the categories
-        pytest.raises(KeyError, lambda: self.df2.loc[['a', 'd']])
+        with pytest.raises(KeyError):
+            self.df2.loc[['a', 'd']]
 
     def test_loc_listlike_dtypes(self):
         # GH 11586
diff --git a/pandas/tests/indexing/test_datetime.py b/pandas/tests/indexing/test_datetime.py
--- a/pandas/tests/indexing/test_datetime.py
+++ b/pandas/tests/indexing/test_datetime.py
@@ -223,7 +223,9 @@ def test_series_partial_set_datetime(self):
                 Timestamp('2011-01-03')]
         exp = Series([np.nan, 0.2, np.nan],
                      index=pd.DatetimeIndex(keys, name='idx'), name='s')
-        tm.assert_series_equal(ser.loc[keys], exp, check_index_type=True)
+        with tm.assert_produces_warning(FutureWarning,
+                                        check_stacklevel=False):
+            tm.assert_series_equal(ser.loc[keys], exp, check_index_type=True)
 
     def test_series_partial_set_period(self):
         # GH 11497
@@ -248,5 +250,7 @@ def test_series_partial_set_period(self):
                 pd.Period('2011-01-03', freq='D')]
         exp = Series([np.nan, 0.2, np.nan],
                      index=pd.PeriodIndex(keys, name='idx'), name='s')
-        result = ser.loc[keys]
+        with tm.assert_produces_warning(FutureWarning,
+                                        check_stacklevel=False):
+            result = ser.loc[keys]
         tm.assert_series_equal(result, exp)
diff --git a/pandas/tests/indexing/test_iloc.py b/pandas/tests/indexing/test_iloc.py
--- a/pandas/tests/indexing/test_iloc.py
+++ b/pandas/tests/indexing/test_iloc.py
@@ -617,7 +617,8 @@ def test_iloc_non_unique_indexing(self):
         expected = DataFrame(new_list)
         expected = pd.concat([expected, DataFrame(index=idx[idx > sidx.max()])
                               ])
-        result = df2.loc[idx]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = df2.loc[idx]
         tm.assert_frame_equal(result, expected, check_index_type=False)
 
     def test_iloc_empty_list_indexer_is_ok(self):
diff --git a/pandas/tests/indexing/test_indexing.py b/pandas/tests/indexing/test_indexing.py
--- a/pandas/tests/indexing/test_indexing.py
+++ b/pandas/tests/indexing/test_indexing.py
@@ -176,7 +176,8 @@ def test_dups_fancy_indexing(self):
              'test1': [7., 6, np.nan],
              'other': ['d', 'c', np.nan]}, index=rows)
 
-        result = df.loc[rows]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = df.loc[rows]
         tm.assert_frame_equal(result, expected)
 
         # see GH5553, make sure we use the right indexer
@@ -186,7 +187,8 @@ def test_dups_fancy_indexing(self):
                               'other': [np.nan, np.nan, np.nan,
                                         'd', 'c', np.nan]},
                              index=rows)
-        result = df.loc[rows]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = df.loc[rows]
         tm.assert_frame_equal(result, expected)
 
         # inconsistent returns for unique/duplicate indices when values are
@@ -203,12 +205,14 @@ def test_dups_fancy_indexing(self):
 
         # GH 4619; duplicate indexer with missing label
         df = DataFrame({"A": [0, 1, 2]})
-        result = df.loc[[0, 8, 0]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = df.loc[[0, 8, 0]]
         expected = DataFrame({"A": [0, np.nan, 0]}, index=[0, 8, 0])
         tm.assert_frame_equal(result, expected, check_index_type=False)
 
         df = DataFrame({"A": list('abc')})
-        result = df.loc[[0, 8, 0]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = df.loc[[0, 8, 0]]
         expected = DataFrame({"A": ['a', np.nan, 'a']}, index=[0, 8, 0])
         tm.assert_frame_equal(result, expected, check_index_type=False)
 
@@ -216,7 +220,8 @@ def test_dups_fancy_indexing(self):
         df = DataFrame({'test': [5, 7, 9, 11]}, index=['A', 'A', 'B', 'C'])
         expected = DataFrame(
             {'test': [5, 7, 5, 7, np.nan]}, index=['A', 'A', 'A', 'A', 'E'])
-        result = df.loc[['A', 'A', 'E']]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = df.loc[['A', 'A', 'E']]
         tm.assert_frame_equal(result, expected)
 
         # GH 5835
@@ -227,7 +232,8 @@ def test_dups_fancy_indexing(self):
         expected = pd.concat(
             [df.loc[:, ['A', 'B']], DataFrame(np.nan, columns=['C'],
                                               index=df.index)], axis=1)
-        result = df.loc[:, ['A', 'B', 'C']]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = df.loc[:, ['A', 'B', 'C']]
         tm.assert_frame_equal(result, expected)
 
         # GH 6504, multi-axis indexing
diff --git a/pandas/tests/indexing/test_loc.py b/pandas/tests/indexing/test_loc.py
--- a/pandas/tests/indexing/test_loc.py
+++ b/pandas/tests/indexing/test_loc.py
@@ -152,15 +152,29 @@ def test_loc_getitem_label_list(self):
                           [Timestamp('20130102'), Timestamp('20130103')],
                           typs=['ts'], axes=0)
 
+    def test_loc_getitem_label_list_with_missing(self):
         self.check_result('list lbl', 'loc', [0, 1, 2], 'indexer', [0, 1, 2],
                           typs=['empty'], fails=KeyError)
-        self.check_result('list lbl', 'loc', [0, 2, 3], 'ix', [0, 2, 3],
-                          typs=['ints', 'uints'], axes=0, fails=KeyError)
-        self.check_result('list lbl', 'loc', [3, 6, 7], 'ix', [3, 6, 7],
-                          typs=['ints', 'uints'], axes=1, fails=KeyError)
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            self.check_result('list lbl', 'loc', [0, 2, 3], 'ix', [0, 2, 3],
+                              typs=['ints', 'uints'], axes=0, fails=KeyError)
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            self.check_result('list lbl', 'loc', [3, 6, 7], 'ix', [3, 6, 7],
+                              typs=['ints', 'uints'], axes=1, fails=KeyError)
         self.check_result('list lbl', 'loc', [4, 8, 10], 'ix', [4, 8, 10],
                           typs=['ints', 'uints'], axes=2, fails=KeyError)
 
+    def test_getitem_label_list_with_missing(self):
+        s = pd.Series(range(3), index=['a', 'b', 'c'])
+
+        # consistency
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            s[['a', 'd']]
+
+        s = pd.Series(range(3))
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            s[[0, 3]]
+
     def test_loc_getitem_label_list_fails(self):
         # fails
         self.check_result('list lbl', 'loc', [20, 30, 40], 'ix', [20, 30, 40],
@@ -249,7 +263,9 @@ def test_loc_to_fail(self):
         pytest.raises(KeyError, lambda: s.loc[['4']])
 
         s.loc[-1] = 3
-        result = s.loc[[-1, -2]]
+        with tm.assert_produces_warning(FutureWarning,
+                                        check_stacklevel=False):
+            result = s.loc[[-1, -2]]
         expected = Series([3, np.nan], index=[-1, -2])
         tm.assert_series_equal(result, expected)
 
@@ -277,6 +293,23 @@ def f():
 
         pytest.raises(KeyError, f)
 
+    def test_loc_getitem_list_with_fail(self):
+        # 15747
+        # should KeyError if *any* missing labels
+
+        s = Series([1, 2, 3])
+
+        s.loc[[2]]
+
+        with pytest.raises(KeyError):
+            s.loc[[3]]
+
+        # a non-match and a match
+        with tm.assert_produces_warning(FutureWarning):
+            expected = s.loc[[2, 3]]
+        result = s.reindex([2, 3])
+        tm.assert_series_equal(result, expected)
+
     def test_loc_getitem_label_slice(self):
 
         # label slices (with ints)
diff --git a/pandas/tests/indexing/test_partial.py b/pandas/tests/indexing/test_partial.py
--- a/pandas/tests/indexing/test_partial.py
+++ b/pandas/tests/indexing/test_partial.py
@@ -222,13 +222,21 @@ def test_series_partial_set(self):
         # Regression from GH4825
         ser = Series([0.1, 0.2], index=[1, 2])
 
-        # loc
+        # loc equiv to .reindex
         expected = Series([np.nan, 0.2, np.nan], index=[3, 2, 3])
-        result = ser.loc[[3, 2, 3]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = ser.loc[[3, 2, 3]]
+        tm.assert_series_equal(result, expected, check_index_type=True)
+
+        result = ser.reindex([3, 2, 3])
         tm.assert_series_equal(result, expected, check_index_type=True)
 
         expected = Series([np.nan, 0.2, np.nan, np.nan], index=[3, 2, 3, 'x'])
-        result = ser.loc[[3, 2, 3, 'x']]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = ser.loc[[3, 2, 3, 'x']]
+        tm.assert_series_equal(result, expected, check_index_type=True)
+
+        result = ser.reindex([3, 2, 3, 'x'])
         tm.assert_series_equal(result, expected, check_index_type=True)
 
         expected = Series([0.2, 0.2, 0.1], index=[2, 2, 1])
@@ -236,38 +244,71 @@ def test_series_partial_set(self):
         tm.assert_series_equal(result, expected, check_index_type=True)
 
         expected = Series([0.2, 0.2, np.nan, 0.1], index=[2, 2, 'x', 1])
-        result = ser.loc[[2, 2, 'x', 1]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = ser.loc[[2, 2, 'x', 1]]
+        tm.assert_series_equal(result, expected, check_index_type=True)
+
+        result = ser.reindex([2, 2, 'x', 1])
         tm.assert_series_equal(result, expected, check_index_type=True)
 
         # raises as nothing in in the index
         pytest.raises(KeyError, lambda: ser.loc[[3, 3, 3]])
 
         expected = Series([0.2, 0.2, np.nan], index=[2, 2, 3])
-        result = ser.loc[[2, 2, 3]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = ser.loc[[2, 2, 3]]
         tm.assert_series_equal(result, expected, check_index_type=True)
 
+        result = ser.reindex([2, 2, 3])
+        tm.assert_series_equal(result, expected, check_index_type=True)
+
+        s = Series([0.1, 0.2, 0.3], index=[1, 2, 3])
         expected = Series([0.3, np.nan, np.nan], index=[3, 4, 4])
-        result = Series([0.1, 0.2, 0.3], index=[1, 2, 3]).loc[[3, 4, 4]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = s.loc[[3, 4, 4]]
         tm.assert_series_equal(result, expected, check_index_type=True)
 
+        result = s.reindex([3, 4, 4])
+        tm.assert_series_equal(result, expected, check_index_type=True)
+
+        s = Series([0.1, 0.2, 0.3, 0.4],
+                   index=[1, 2, 3, 4])
         expected = Series([np.nan, 0.3, 0.3], index=[5, 3, 3])
-        result = Series([0.1, 0.2, 0.3, 0.4],
-                        index=[1, 2, 3, 4]).loc[[5, 3, 3]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = s.loc[[5, 3, 3]]
+        tm.assert_series_equal(result, expected, check_index_type=True)
+
+        result = s.reindex([5, 3, 3])
         tm.assert_series_equal(result, expected, check_index_type=True)
 
+        s = Series([0.1, 0.2, 0.3, 0.4],
+                   index=[1, 2, 3, 4])
         expected = Series([np.nan, 0.4, 0.4], index=[5, 4, 4])
-        result = Series([0.1, 0.2, 0.3, 0.4],
-                        index=[1, 2, 3, 4]).loc[[5, 4, 4]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = s.loc[[5, 4, 4]]
+        tm.assert_series_equal(result, expected, check_index_type=True)
+
+        result = s.reindex([5, 4, 4])
         tm.assert_series_equal(result, expected, check_index_type=True)
 
+        s = Series([0.1, 0.2, 0.3, 0.4],
+                   index=[4, 5, 6, 7])
         expected = Series([0.4, np.nan, np.nan], index=[7, 2, 2])
-        result = Series([0.1, 0.2, 0.3, 0.4],
-                        index=[4, 5, 6, 7]).loc[[7, 2, 2]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = s.loc[[7, 2, 2]]
         tm.assert_series_equal(result, expected, check_index_type=True)
 
+        result = s.reindex([7, 2, 2])
+        tm.assert_series_equal(result, expected, check_index_type=True)
+
+        s = Series([0.1, 0.2, 0.3, 0.4],
+                   index=[1, 2, 3, 4])
         expected = Series([0.4, np.nan, np.nan], index=[4, 5, 5])
-        result = Series([0.1, 0.2, 0.3, 0.4],
-                        index=[1, 2, 3, 4]).loc[[4, 5, 5]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = s.loc[[4, 5, 5]]
+        tm.assert_series_equal(result, expected, check_index_type=True)
+
+        result = s.reindex([4, 5, 5])
         tm.assert_series_equal(result, expected, check_index_type=True)
 
         # iloc
@@ -284,13 +325,15 @@ def test_series_partial_set_with_name(self):
         # loc
         exp_idx = Index([3, 2, 3], dtype='int64', name='idx')
         expected = Series([np.nan, 0.2, np.nan], index=exp_idx, name='s')
-        result = ser.loc[[3, 2, 3]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = ser.loc[[3, 2, 3]]
         tm.assert_series_equal(result, expected, check_index_type=True)
 
         exp_idx = Index([3, 2, 3, 'x'], dtype='object', name='idx')
         expected = Series([np.nan, 0.2, np.nan, np.nan], index=exp_idx,
                           name='s')
-        result = ser.loc[[3, 2, 3, 'x']]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = ser.loc[[3, 2, 3, 'x']]
         tm.assert_series_equal(result, expected, check_index_type=True)
 
         exp_idx = Index([2, 2, 1], dtype='int64', name='idx')
@@ -300,7 +343,8 @@ def test_series_partial_set_with_name(self):
 
         exp_idx = Index([2, 2, 'x', 1], dtype='object', name='idx')
         expected = Series([0.2, 0.2, np.nan, 0.1], index=exp_idx, name='s')
-        result = ser.loc[[2, 2, 'x', 1]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = ser.loc[[2, 2, 'x', 1]]
         tm.assert_series_equal(result, expected, check_index_type=True)
 
         # raises as nothing in in the index
@@ -308,41 +352,49 @@ def test_series_partial_set_with_name(self):
 
         exp_idx = Index([2, 2, 3], dtype='int64', name='idx')
         expected = Series([0.2, 0.2, np.nan], index=exp_idx, name='s')
-        result = ser.loc[[2, 2, 3]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = ser.loc[[2, 2, 3]]
         tm.assert_series_equal(result, expected, check_index_type=True)
 
         exp_idx = Index([3, 4, 4], dtype='int64', name='idx')
         expected = Series([0.3, np.nan, np.nan], index=exp_idx, name='s')
         idx = Index([1, 2, 3], dtype='int64', name='idx')
-        result = Series([0.1, 0.2, 0.3], index=idx, name='s').loc[[3, 4, 4]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = Series([0.1, 0.2, 0.3],
+                            index=idx,
+                            name='s').loc[[3, 4, 4]]
         tm.assert_series_equal(result, expected, check_index_type=True)
 
         exp_idx = Index([5, 3, 3], dtype='int64', name='idx')
         expected = Series([np.nan, 0.3, 0.3], index=exp_idx, name='s')
         idx = Index([1, 2, 3, 4], dtype='int64', name='idx')
-        result = Series([0.1, 0.2, 0.3, 0.4], index=idx,
-                        name='s').loc[[5, 3, 3]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = Series([0.1, 0.2, 0.3, 0.4], index=idx,
+                            name='s').loc[[5, 3, 3]]
         tm.assert_series_equal(result, expected, check_index_type=True)
 
         exp_idx = Index([5, 4, 4], dtype='int64', name='idx')
         expected = Series([np.nan, 0.4, 0.4], index=exp_idx, name='s')
         idx = Index([1, 2, 3, 4], dtype='int64', name='idx')
-        result = Series([0.1, 0.2, 0.3, 0.4], index=idx,
-                        name='s').loc[[5, 4, 4]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = Series([0.1, 0.2, 0.3, 0.4], index=idx,
+                            name='s').loc[[5, 4, 4]]
         tm.assert_series_equal(result, expected, check_index_type=True)
 
         exp_idx = Index([7, 2, 2], dtype='int64', name='idx')
         expected = Series([0.4, np.nan, np.nan], index=exp_idx, name='s')
         idx = Index([4, 5, 6, 7], dtype='int64', name='idx')
-        result = Series([0.1, 0.2, 0.3, 0.4], index=idx,
-                        name='s').loc[[7, 2, 2]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = Series([0.1, 0.2, 0.3, 0.4], index=idx,
+                            name='s').loc[[7, 2, 2]]
         tm.assert_series_equal(result, expected, check_index_type=True)
 
         exp_idx = Index([4, 5, 5], dtype='int64', name='idx')
         expected = Series([0.4, np.nan, np.nan], index=exp_idx, name='s')
         idx = Index([1, 2, 3, 4], dtype='int64', name='idx')
-        result = Series([0.1, 0.2, 0.3, 0.4], index=idx,
-                        name='s').loc[[4, 5, 5]]
+        with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
+            result = Series([0.1, 0.2, 0.3, 0.4], index=idx,
+                            name='s').loc[[4, 5, 5]]
         tm.assert_series_equal(result, expected, check_index_type=True)
 
         # iloc
diff --git a/pandas/tests/io/test_excel.py b/pandas/tests/io/test_excel.py
--- a/pandas/tests/io/test_excel.py
+++ b/pandas/tests/io/test_excel.py
@@ -1808,8 +1808,10 @@ def test_invalid_columns(self):
             write_frame = DataFrame({'A': [1, 1, 1],
                                      'B': [2, 2, 2]})
 
-            write_frame.to_excel(path, 'test1', columns=['B', 'C'])
-            expected = write_frame.loc[:, ['B', 'C']]
+            with tm.assert_produces_warning(FutureWarning,
+                                            check_stacklevel=False):
+                write_frame.to_excel(path, 'test1', columns=['B', 'C'])
+            expected = write_frame.reindex(columns=['B', 'C'])
             read_frame = read_excel(path, 'test1')
             tm.assert_frame_equal(expected, read_frame)
 
diff --git a/pandas/tests/reshape/test_concat.py b/pandas/tests/reshape/test_concat.py
--- a/pandas/tests/reshape/test_concat.py
+++ b/pandas/tests/reshape/test_concat.py
@@ -1222,7 +1222,7 @@ def test_handle_empty_objects(self):
         frames = [baz, empty, empty, df[5:]]
         concatted = concat(frames, axis=0)
 
-        expected = df.loc[:, ['a', 'b', 'c', 'd', 'foo']]
+        expected = df.reindex(columns=['a', 'b', 'c', 'd', 'foo'])
         expected['foo'] = expected['foo'].astype('O')
         expected.loc[0:4, 'foo'] = 'bar'
 
diff --git a/pandas/tests/series/test_indexing.py b/pandas/tests/series/test_indexing.py
--- a/pandas/tests/series/test_indexing.py
+++ b/pandas/tests/series/test_indexing.py
@@ -590,8 +590,13 @@ def test_getitem_dups_with_missing(self):
         # breaks reindex, so need to use .loc internally
         # GH 4246
         s = Series([1, 2, 3, 4], ['foo', 'bar', 'foo', 'bah'])
-        expected = s.loc[['foo', 'bar', 'bah', 'bam']]
-        result = s[['foo', 'bar', 'bah', 'bam']]
+        with tm.assert_produces_warning(FutureWarning,
+                                        check_stacklevel=False):
+            expected = s.loc[['foo', 'bar', 'bah', 'bam']]
+
+        with tm.assert_produces_warning(FutureWarning,
+                                        check_stacklevel=False):
+            result = s[['foo', 'bar', 'bah', 'bam']]
         assert_series_equal(result, expected)
 
     def test_getitem_dups(self):
diff --git a/pandas/tests/sparse/test_indexing.py b/pandas/tests/sparse/test_indexing.py
--- a/pandas/tests/sparse/test_indexing.py
+++ b/pandas/tests/sparse/test_indexing.py
@@ -121,8 +121,8 @@ def test_loc(self):
         tm.assert_sp_series_equal(result, exp)
 
         # exceeds the bounds
-        result = sparse.loc[[1, 3, 4, 5]]
-        exp = orig.loc[[1, 3, 4, 5]].to_sparse()
+        result = sparse.reindex([1, 3, 4, 5])
+        exp = orig.reindex([1, 3, 4, 5]).to_sparse()
         tm.assert_sp_series_equal(result, exp)
         # padded with NaN
         assert np.isnan(result[-1])
@@ -677,8 +677,8 @@ def test_loc(self):
         tm.assert_sp_frame_equal(result, exp)
 
         # exceeds the bounds
-        result = sparse.loc[[1, 3, 4, 5]]
-        exp = orig.loc[[1, 3, 4, 5]].to_sparse()
+        result = sparse.reindex([1, 3, 4, 5])
+        exp = orig.reindex([1, 3, 4, 5]).to_sparse()
         tm.assert_sp_frame_equal(result, exp)
 
         # dense array