Repository: pandas-dev/pandas
Pull request: ENH: Add errors parameter to DataFrame.rename
- [X] closes #13473
- [x] tests added / passed
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Problem statement:
ENH: add errors='raise' option to rename
Is there a reason that `.rename` doesn't raise if keys are supplied that don't exist? This caught me out a couple of times:

``` python
In [14]: import pandas as pd

In [15]: df=pd.DataFrame({'a': [1,2], 'b': [3,4]})

In [16]: df
Out[16]: 
   a  b
0  1  3
1  2  4

In [17]: df.rename(columns={'a': 'c'})
Out[17]: 
   c  b
0  1  3
1  2  4
# ok

In [18]: df.rename(columns={'d': 'c'})
Out[18]: 
   a  b
0  1  3
1  2  4

# this should raise?
```

Hints from issue discussion:
I suspect it is intentional. Either way, the behavior should be clarified in the docs. Missing keys (in columns but not in the dict) are explicitly documented.

I know I've used this in the past. For example a bunch of CSVs with variations on the same items. I'll make one dictionary with all the mappings

``` python
renamer = {'zip': 'zip_code', 'postal code': 'zip_code', ...}`
```

And use that same `renamer` on many dataframes.

OK. You know this better than me - do you think this is too much vs the increase in explicitness?

``` python
df.rename(columns={col: renamer[col] for col in renamer if col in df.columns})
```

For `DataFrame.drop`, the `errors` keyword was added some time ago:

```
errors : {'ignore', 'raise'}, default 'raise'
    If 'ignore', suppress error and existing labels are dropped.

    .. versionadded:: 0.16.1
```

We could maybe add this to rename as well to have similar functionality? (although the default value would be different for rename)

Having a keyword to control this would be great. 

Another option is to revisit the "tell people to use `relabel` discussion". If we implemented it there we could have the default be to raise. 

> On Jun 30, 2016, at 18:02, Joris Van den Bossche notifications@github.com wrote:
> 
> For DataFrame.drop, the errors keyword was added some time ago:
> 
> errors : {'ignore', 'raise'}, default 'raise'
>     If 'ignore', suppress error and existing labels are dropped.
> 
> ```
> .. versionadded:: 0.16.1
> ```
> 
> We could maybe add this to rename as well to have similar functionality? (although the default value would be different for rename)
> 
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or mute the thread.

Is there a particular reason why the default value would be "ignore" for `rename`? If I'm renaming keys in a mapping but one isn't there, I normally anticipate a `KeyError`. A deprecation period would be necessary, of course -- I think a similar change has occurred recently with missing columns for `loc`.
Historical reasons. I think raising would be preferable though correct would require deprecation cycle
To be clear, I don't think there's a consensus for deprecating the current behavior. Just adding the option to raise when not present.

Commit message: ENH: Implement feedback (#13473)

Patch to review:
diff --git a/doc/source/whatsnew/v0.25.0.rst b/doc/source/whatsnew/v0.25.0.rst
index 124ec8f4ab92c..67474918159d5 100644
--- a/doc/source/whatsnew/v0.25.0.rst
+++ b/doc/source/whatsnew/v0.25.0.rst
@@ -25,7 +25,7 @@ Other Enhancements
 - ``Series.str`` has gained :meth:`Series.str.casefold` method to removes all case distinctions present in a string (:issue:`25405`)
 - :meth:`DataFrame.set_index` now works for instances of ``abc.Iterator``, provided their output is of the same length as the calling frame (:issue:`22484`, :issue:`24984`)
 - :meth:`DatetimeIndex.union` now supports the ``sort`` argument. The behaviour of the sort parameter matches that of :meth:`Index.union` (:issue:`24994`)
--
+- :meth:`DataFrame.rename` now supports the ``errors`` argument to raise errors when attempting to rename nonexistent keys (:issue:`13473`)
 
 .. _whatsnew_0250.api_breaking:
 
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
index 6b4d95055d06d..ea06116e7dd9c 100644
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -3911,7 +3911,8 @@ def drop(self, labels=None, axis=0, index=None, columns=None,
 
     @rewrite_axis_style_signature('mapper', [('copy', True),
                                              ('inplace', False),
-                                             ('level', None)])
+                                             ('level', None),
+                                             ('errors', 'ignore')])
     def rename(self, *args, **kwargs):
         """
         Alter axes labels.
@@ -3924,30 +3925,46 @@ def rename(self, *args, **kwargs):
 
         Parameters
         ----------
-        mapper, index, columns : dict-like or function, optional
-            dict-like or functions transformations to apply to
+        mapper : dict-like or function
+            Dict-like or functions transformations to apply to
             that axis' values. Use either ``mapper`` and ``axis`` to
             specify the axis to target with ``mapper``, or ``index`` and
             ``columns``.
-        axis : int or str, optional
+        index : dict-like or function
+            Alternative to specifying axis (``mapper, axis=0``
+            is equivalent to ``index=mapper``).
+        columns : dict-like or function
+            Alternative to specifying axis (``mapper, axis=1``
+            is equivalent to ``columns=mapper``).
+        axis : int or str
             Axis to target with ``mapper``. Can be either the axis name
             ('index', 'columns') or number (0, 1). The default is 'index'.
-        copy : boolean, default True
-            Also copy underlying data
-        inplace : boolean, default False
+        copy : bool, default True
+            Also copy underlying data.
+        inplace : bool, default False
             Whether to return a new DataFrame. If True then value of copy is
             ignored.
         level : int or level name, default None
             In case of a MultiIndex, only rename labels in the specified
             level.
+        errors : {'ignore', 'raise'}, default 'ignore'
+            If 'ignore', suppress error and existing labels are renamed.
 
         Returns
         -------
         DataFrame
+            DataFrame with the renamed axis labels.
+
+        Raises
+        ------
+        KeyError
+            If any of the labels is not found in the selected axis and
+            "errors='raise'".
 
         See Also
         --------
-        DataFrame.rename_axis
+        DataFrame.rename_axis : Set the name of the axis for the index or
+         columns.
 
         Examples
         --------
@@ -3973,6 +3990,10 @@ def rename(self, *args, **kwargs):
         1  2  5
         2  3  6
 
+        >>> df.rename(index=str, columns={"A": "a", "C": "c"}, errors="raise")
+        Traceback (most recent call last):
+        KeyError: ['C'] not found in axis
+
         Using axis-style parameters
 
         >>> df.rename(str.lower, axis='columns')
diff --git a/pandas/core/generic.py b/pandas/core/generic.py
index ee8f9cba951b3..e54fec9e14525 100644
--- a/pandas/core/generic.py
+++ b/pandas/core/generic.py
@@ -981,11 +981,19 @@ def rename(self, *args, **kwargs):
         level : int or level name, default None
             In case of a MultiIndex, only rename labels in the specified
             level.
+        errors : {'ignore', 'raise'}, default 'ignore'
+            If 'ignore', suppress error and existing labels are renamed.
 
         Returns
         -------
         renamed : %(klass)s (new object)
 
+        Raises
+        ------
+        KeyError
+            If any of the labels is not found in the selected axis and
+            "errors='raise'".
+
         See Also
         --------
         NDFrame.rename_axis
@@ -1065,6 +1073,7 @@ def rename(self, *args, **kwargs):
         inplace = kwargs.pop('inplace', False)
         level = kwargs.pop('level', None)
         axis = kwargs.pop('axis', None)
+        errors = kwargs.pop('errors', 'ignore')
         if axis is not None:
             # Validate the axis
             self._get_axis_number(axis)
@@ -1085,10 +1094,19 @@ def rename(self, *args, **kwargs):
             if v is None:
                 continue
             f = com._get_rename_function(v)
-
             baxis = self._get_block_manager_axis(axis)
             if level is not None:
                 level = self.axes[axis]._get_level_number(level)
+
+            # GH 13473
+            if not callable(v):
+                indexer = self.axes[axis].get_indexer_for(v)
+                missing_labels = [label for index, label in enumerate(v)
+                                  if indexer[index] == -1]
+                if errors == 'raise' and len(missing_labels) > 0:
+                    raise KeyError('{} not found in axis'
+                                   .format(missing_labels))
+
             result._data = result._data.rename_axis(f, axis=baxis, copy=copy,
                                                     level=level)
             result._clear_item_cache()
diff --git a/pandas/tests/frame/test_alter_axes.py b/pandas/tests/frame/test_alter_axes.py
index a25e893e08900..571455515e2a9 100644
--- a/pandas/tests/frame/test_alter_axes.py
+++ b/pandas/tests/frame/test_alter_axes.py
@@ -871,6 +871,23 @@ def test_rename_bug2(self):
                              columns=["a"])
         tm.assert_frame_equal(df, expected)
 
+    def test_rename_errors_raises(self):
+        df = DataFrame(columns=['A', 'B', 'C', 'D'])
+        with pytest.raises(KeyError, match='\'E\'] not found in axis'):
+            df.rename(columns={'A': 'a', 'E': 'e'}, errors='raise')
+
+    @pytest.mark.parametrize('mapper, errors, expected_columns', [
+        ({'A': 'a', 'E': 'e'}, 'ignore', ['a', 'B', 'C', 'D']),
+        ({'A': 'a'}, 'raise', ['a', 'B', 'C', 'D']),
+        (str.lower, 'raise', ['a', 'b', 'c', 'd'])])
+    def test_rename_errors(self, mapper, errors, expected_columns):
+        # GH 13473
+        # rename now works with errors parameter
+        df = DataFrame(columns=['A', 'B', 'C', 'D'])
+        renamed = df.rename(columns=mapper, errors=errors)
+        expected = DataFrame(columns=expected_columns)
+        tm.assert_frame_equal(renamed, expected)
+
     def test_reorder_levels(self):
         index = MultiIndex(levels=[['bar'], ['one', 'two', 'three'], [0, 1]],
                            codes=[[0, 0, 0, 0, 0, 0],
@@ -1328,7 +1345,7 @@ def test_rename_signature(self):
         sig = inspect.signature(DataFrame.rename)
         parameters = set(sig.parameters)
         assert parameters == {"self", "mapper", "index", "columns", "axis",
-                              "inplace", "copy", "level"}
+                              "inplace", "copy", "level", "errors"}
 
     @pytest.mark.skipif(PY2, reason="inspect.signature")
     def test_reindex_signature(self):


Code review comments:
diff_hunk: @@ -3924,30 +3925,46 @@ def rename(self, *args, **kwargs):
 
         Parameters
         ----------
-        mapper, index, columns : dict-like or function, optional
-            dict-like or functions transformations to apply to
+        mapper : dict-like or function
+            Dict-like or functions transformations to apply to
             that axis' values. Use either ``mapper`` and ``axis`` to
             specify the axis to target with ``mapper``, or ``index`` and
             ``columns``.
-        axis : int or str, optional
+        index : dict-like or function
+            Alternative to specifying axis (``mapper, axis=0``
+            is equivalent to ``index=mapper``).
+        columns : dict-like or function
+            Alternative to specifying axis (``mapper, axis=1``
+            is equivalent to ``columns=mapper``).
+        axis : int or str
             Axis to target with ``mapper``. Can be either the axis name
             ('index', 'columns') or number (0, 1). The default is 'index'.
-        copy : boolean, default True
-            Also copy underlying data
-        inplace : boolean, default False
+        copy : bool, default True
+            Also copy underlying data.
+        inplace : bool, default False
             Whether to return a new DataFrame. If True then value of copy is
             ignored.
         level : int or level name, default None
             In case of a MultiIndex, only rename labels in the specified
             level.
+        errors : {'ignore', 'raise'}, default 'ignore'
+            If 'ignore', suppress error and existing labels are renamed.
 
         Returns
         -------
         DataFrame
+            DataFrame with the renamed axis labels.
+
+        Raises
+        ------
+        KeyError
+            If any of the labels is not found in the selected axis and
+            "errors='raise'".
 
         See Also
         --------
-        DataFrame.rename_axis
+        DataFrame.rename_axis : Set the name of the axis for the index or
+         columns.
path: pandas/core/frame.py
text: @user1:
Check the doc build log for a warning. This may need to be one line. Maybe shorten to "Set the name of the axis"?

@author:
Although I do not see an error or warning while building the documentation, I will shorten the description.

diff_hunk: @@ -1085,10 +1094,19 @@ def rename(self, *args, **kwargs):
             if v is None:
                 continue
             f = com._get_rename_function(v)
-
             baxis = self._get_block_manager_axis(axis)
             if level is not None:
                 level = self.axes[axis]._get_level_number(level)
+
+            # GH 13473
+            if not callable(v):
+                indexer = self.axes[axis].get_indexer_for(v)
+                missing_labels = [label for index, label in enumerate(v)
path: pandas/core/generic.py
text: @user1:
I'm a tad concerned about perf here. Can you instead check first if there are any missing labels with something like

```python
missing = indexer[indexer == -1]
if len(missing):
    ...
```

I'd like to avoid the iteration over `v`, which may be large.

but maybe measure the relative perf of the two approaches.

@author:
I understand what you mean; I introduced `missing_labels` to generate a nice error message, where only the labels which are missing are in the error message.

I see two options:
- Remove `missing_labels` altogether and just place `v` inside the error message.
- Move `missing_labels` into the if-statement, so that the enumeration is only done in the case of an KeyError.

I'd personally prefer the latter, what do you think is best?

@user1:
Sorry if I wasn't clear. Including the missing labels in the error message is nice. Moving it under the `if` statement is best.