Repository: pandas-dev/pandas

Issue:
replace() alters unrelated values
I don't know if it is the same bug than #7126 (looks similar, but I didn't check the implementation), but

```
In [1]: from pandas import DataFrame

In [2]: import numpy as np

In [3]: df = DataFrame(index=range(2))

In [4]: df['a'] = True

In [5]: df.a

Out[5]: 
0    True
1    True
Name: a, dtype: bool

In [6]: df.replace([np.inf, -np.inf], np.nan).a

Out[6]: 
0   NaN
1   NaN
Name: a, dtype: float64
```

Notice the operation of doing such a replace on boolean variables may seem stupid... but it is less so if you consider that "replace" (differently from i.e. dropna) does not support a "subset" argument.



Discussion:
seems to be replacing "truthy" values ... again the dtype issues arise :)

The docs on replace suggest that, if it's a list, it should be a list of _strings_... so I'm not sure this is a bug per se. wow, [replace](http://pandas.pydata.org/pandas-docs/version/0.13.1/generated/pandas.Series.replace.html) really does do a **lot**!

I would definitely consider using one of these instead:

```
df.where(~df.isin([np.inf, -np.inf]))
df.where(np.isfinite(df.values))
```

_Note: These work with both series and dataframes._

@hayd : I interpret "list of str, regex, or numeric" as "list of str, list of regex, or list of numeric"... after all,

```
df['b'] = range(2)
df.replace([-1, 0], np.nan)
```

works as expected (and there is no theoretical reason why it shouldn't).

Moreover, "where" does work on dataframes... but I don't see what I am supposed to do with its result, in order to get the result of "replace" (but this may be me temporarily blind).

I do understand that a "replace" on a Series of bools may want to filter based on the truthy value, but I'm pretty sure that I want a quick way to replace any numeric value in a DataFrame, without touching booleans. So maybe one possible solution would be to clarify the problem in the docs, and add an argument "subset" as for dropna. Although a cleaner solution would be to add an argument "dtype" which determines the dtype to use for the comparison (i.e. convert booleans to float, not vice versa), and which would have the default value of the series' dtype for a Series, and of the item to be replaced's dtype for a DataFrame with non-homogeneous dtypes.

Notice the same problem arises with

```
df['c'] = False
df.replace([None], np.nan)
```

Good point, that seems like a reasonable interpretation!

@toobaz while adding those things (kwarg, subset) might make this use case a bit more clear, replace already tries to do too much. In fact we're considering deprecating the replace method (over at least a couple of releases since it's fairly old) into a namespace like we do with `str` for vectorized string methods. See https://github.com/pydata/pandas/issues/5541#issuecomment-39671216 and that issue (#5541). 

@toobaz all that said, this IS a bug which will be fixed by the next minor release (0.14.1)


Resolving patch:
diff --git a/pandas/core/internals.py b/pandas/core/internals.py
--- a/pandas/core/internals.py
+++ b/pandas/core/internals.py
@@ -3,18 +3,17 @@
 import re
 import operator
 from datetime import datetime, timedelta
-from collections import defaultdict, deque
+from collections import defaultdict
 
 import numpy as np
 from pandas.core.base import PandasObject
 
-from pandas.hashtable import Factorizer
-from pandas.core.common import (_possibly_downcast_to_dtype, isnull, notnull,
+from pandas.core.common import (_possibly_downcast_to_dtype, isnull,
                                 _NS_DTYPE, _TD_DTYPE, ABCSeries, is_list_like,
                                 ABCSparseSeries, _infer_dtype_from_scalar,
                                 _is_null_datelike_scalar,
                                 is_timedelta64_dtype, is_datetime64_dtype,)
-from pandas.core.index import Index, Int64Index, MultiIndex, _ensure_index
+from pandas.core.index import Index, MultiIndex, _ensure_index
 from pandas.core.indexing import (_maybe_convert_indices, _length_of_indexer)
 import pandas.core.common as com
 from pandas.sparse.array import _maybe_to_sparse, SparseArray
@@ -25,12 +24,10 @@
 
 from pandas.tslib import Timestamp
 from pandas import compat
-from pandas.compat import (range, lrange, lmap, callable, map, zip, u,
-                           OrderedDict)
+from pandas.compat import range, map, zip, u
 from pandas.tseries.timedeltas import _coerce_scalar_to_timedelta_type
 
 
-
 from pandas.lib import BlockPlacement
 
 
@@ -1020,6 +1017,7 @@ def equals(self, other):
         left, right = self.values, other.values
         return ((left == right) | (np.isnan(left) & np.isnan(right))).all()
 
+
 class FloatBlock(FloatOrComplexBlock):
     __slots__ = ()
     is_float = True
@@ -1212,7 +1210,7 @@ def replace(self, to_replace, value, inplace=False, filter=None,
                 regex=False):
         to_replace_values = np.atleast_1d(to_replace)
         if not np.can_cast(to_replace_values, bool):
-            to_replace = to_replace_values
+            return self
         return super(BoolBlock, self).replace(to_replace, value,
                                               inplace=inplace, filter=filter,
                                               regex=regex)


Test patch:
diff --git a/pandas/tests/test_frame.py b/pandas/tests/test_frame.py
--- a/pandas/tests/test_frame.py
+++ b/pandas/tests/test_frame.py
@@ -8351,6 +8351,12 @@ def test_replace_with_dict_with_bool_keys(self):
         with tm.assertRaisesRegexp(TypeError, 'Cannot compare types .+'):
             df.replace({'asdf': 'asdb', True: 'yes'})
 
+    def test_replace_truthy(self):
+        df = DataFrame({'a': [True, True]})
+        r = df.replace([np.inf, -np.inf], np.nan)
+        e = df
+        tm.assert_frame_equal(r, e)
+
     def test_replace_int_to_int_chain(self):
         df = DataFrame({'a': lrange(1, 5)})
         with tm.assertRaisesRegexp(ValueError, "Replacement not allowed .+"):
diff --git a/pandas/tests/test_internals.py b/pandas/tests/test_internals.py
--- a/pandas/tests/test_internals.py
+++ b/pandas/tests/test_internals.py
@@ -4,7 +4,7 @@
 import numpy as np
 
 from pandas import Index, MultiIndex, DataFrame, Series
-from pandas.compat import OrderedDict
+from pandas.compat import OrderedDict, lrange
 from pandas.sparse.array import SparseArray
 from pandas.core.internals import *
 import pandas.core.internals as internals
@@ -835,8 +835,6 @@ def assert_reindex_indexer_is_ok(mgr, axis, new_labels, indexer,
     # reindex_indexer(new_labels, indexer, axis)
 
 
-
-
 class TestBlockPlacement(tm.TestCase):
     _multiprocess_can_split_ = True