Repository: pandas-dev/pandas

Issue:
Surprising behavior of DataFrame.replace
This seems a bit surprising:

``` python
>>> from string import ascii_lowercase as letters
>>> from pandas import DataFrame
>>> df = DataFrame(list(letters[:4]), columns=['a'])
>>> df
   a
0  a
1  b
2  c
3  d
>>> df.replace({'a': 'b'})
   a
0  a
1  a
2  c
3  d
>>> df.replace({'a': 'c'})
   a
0  a
1  b
2  b
3  d
```

Does this have to do with padding?



Discussion:
could be a bug

also no doc for the first parameter of this method, will submit a pr for that...

also if u have time:

I think we need clarification in the main docs on usage of when to use

filter, replace, select, update, lookup

I also don't think there is an example of filter anywhere

np, not sure if will be done 2day, trying to work out the regex replace on frames and just wanted to note this strangeness when i was looking at one of the simplest possible cases of replacement in an object block

@jreback A quick glance at `indexing.rst` shows docs for `select` and `lookup` (they could be fleshed out a bit), but nothing for the other 3. Will add the 3 and flesh the other 2 out.

@cpcloud I think replace is somewhere
a complete example

ah yes, `grin '\.replace' doc` shows it

Wow, this was worrying, but fortunately `df.a.replace({'a': 'b'})` and `df.replace({"a":{"a":"b"}})` works as intended...

@jreback seems the magic is already in `_interpolate`, I will leave it out in `replace` though.

@jreback  The "bug" here is that when you call

``` python
df.replace({'a': 'b'})
```

the column `'a'` replaces the value at index `'b'` with the previous non-null value because the fill method default is pad. This seems a bit weird. Can fix if u want. I would change to "search the whole frame for occurrences of `'a'` and replace with `'b'`.

you want this equivalent to

df.replace('a','b')

?

yes, is that ok?

let me take a look a little later

was always fuzzy on this anyhow

are there tests for this usage anyhow ?
do they break?

There are tests. However, the `replace` method doesn't actually do anything so they don't break. Here's a pared down example from the `test_replace_interpolate` method of `TestDataFrame` from `pandas/tests/test_frame.py`:

``` python
# get the test frame
from pandas.tests.test_frame import _tsframe as df
from numpy.testing import assert_array_equal
res = df.replace({'A': nan}, method='pad', axis=1)
assert_array_equal(df, res)  # replace == identity function here
```

I think it makes sense to not have automatic interpolation by default

so, if say `df.replace({ 'a' : 'b' })` would be translated to `df.replace('a','b')`, IOW, only allow None for the value if `to_replace` is a dict/Series (right now this will interpolate)

then make `_interpolate` an explicit method `interpolate`. Even though this is an API change, I doubt this is expected behavor

@wesm, @y-p any thoughts?

@cpcloud why don't you put up the PR when you are ready (as this would be an easy change anyhow)

okay. kicking tires a bit right now (added 12 new tests so far); adding the regex functionality is proving to be quite a bit more involved than i thought, but i have a much better understanding (although not complete) of `DataFrame` internals.

@jreback @y-p Can I address this issue in the regex pr? I think it will create fewer merge conflicts if I just do this all at once. 

yes, I would

just realized the `axis` parameter in replace never gets used if the call to `_interpolate` is nixed...remove the parameter?

@cploud close this too?

Not just yet. Need to submit 0.12 pr. 


Resolving patch:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -3506,7 +3506,7 @@ def replace(self, to_replace=None, value=None, method='pad', axis=0,
 
         See also
         --------
-        reindex, asfreq, fillna, interpolate
+        reindex, asfreq, fillna
 
         Returns
         -------
@@ -3678,6 +3678,10 @@ def interpolate(self, to_replace, method='pad', axis=0, inplace=False,
         --------
         reindex, replace, fillna
         """
+        from warnings import warn
+        warn('DataFrame.interpolate will be removed in v0.12, please use '
+             'either DataFrame.fillna or DataFrame.replace instead',
+             FutureWarning)
         if self._is_mixed_type and axis == 1:
             return self.T.replace(to_replace, method=method, limit=limit).T
 


Test patch: