Repository: pandas-dev/pandas
Pull request: TST: add test for df comparing strings to numbers raises ValueError
- [x] closes #11565
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Problem statement:
Comparing Strings to Numbers: No ValueError
I noticed this quirk today: When you do a DataFrame-wide comparison (excluding `==`) using a number, it doesn't raise a ValueError (which you'd expect in Python 3); instead it always returns True.

``` python
>>> from pandas import DataFrame
>>> df = DataFrame(x: {'x': 'foo', 'y': 'bar', 'z': 'baz'} for x in ['a', 'b', 'c']})
>>> df
     a    b    c
x  foo  foo  foo
y  bar  bar  bar
z  baz  baz  baz
>>> df < 0
      a     b     c
x  True  True  True
y  True  True  True
z  True  True  True
>>> df > 0
      a     b     c
x  True  True  True
y  True  True  True
z  True  True  True
```

However, when you compare a Series of strings to a number, you get the expected ValueError:

``` python
>>> df.a < 0
TypeError: unorderable types: str() < int()
```

Is this a bug or a feature?

Python: 3.4.3
Pandas: 0.17.0
OS: Mac OSX 10.11

Hints from issue discussion:
yeh, I think this is  an api inconsistency. we _should_ make this work I think for `df.a < 0` as it works for everything else, and raising an error is not useful here.

Note that we _do_ have code to do this comparision even in py3, see `core/algorithms.py/factorize` so I think that could be incorporated into the comparison routines (in ops.py)

Oh, really? I think the error is helpful for Series. What I find problematic is that strings always return `True` on a DataFrame-wide comparison. I'd expect `None` or at the very least `False`.

What I originally wanted to do was replace all negative values in the DataFrame with `np.nan` and then `df.dropna()`. Under this structure I can't, because the strings will be replaced as well, leaving me with a rather empty DataFrame. Of course, if I wanted to replace all the positive values `I still couldn't!

Thoughts?

This looks to raise correctly on master. Could use a test.

```
In [68]: df < 0
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
TypeError: '>' not supported between instances of 'str' and 'int'

In [71]: pd.__version__
Out[71]: '0.26.0.dev0+533.gd8f9be7e3'
```

Commit message: add parameterize over all operators

Patch to review:
diff --git a/pandas/tests/frame/test_operators.py b/pandas/tests/frame/test_operators.py
index 19d91241d6a6b..e16b8260709fc 100644
--- a/pandas/tests/frame/test_operators.py
+++ b/pandas/tests/frame/test_operators.py
@@ -530,6 +530,34 @@ def test_comp(func):
         test_comp(operator.ge)
         test_comp(operator.le)
 
+    @pytest.mark.parametrize("op", ["lt", "le", "gt", "ge", "eq", "ne"])
+    def test_strings_to_numbers_comparisons(self, op):
+        # GH 11565
+        df = DataFrame(
+            {x: {"x": "foo", "y": "bar", "z": "baz"} for x in ["a", "b", "c"]}
+        )
+
+        f = getattr(operator, op)
+        if op == "eq":
+            result = f(df, 0)
+            expected = DataFrame(
+                {x: {y: False for y in ["x", "y", "z"]} for x in ["a", "b", "c"]}
+            )
+
+            tm.assert_frame_equal(result, expected)
+
+        elif op == "ne":
+            result = f(df, 0)
+            expected = DataFrame(
+                {x: {y: True for y in ["x", "y", "z"]} for x in ["a", "b", "c"]}
+            )
+
+            tm.assert_frame_equal(result, expected)
+
+        else:
+            with pytest.raises(TypeError):
+                f(df, 0)
+
     def test_comparison_protected_from_errstate(self):
         missing_df = tm.makeDataFrame()
         missing_df.iloc[0]["A"] = np.nan


Code review comments:
diff_hunk: @@ -530,6 +530,34 @@ def test_comp(func):
         test_comp(operator.ge)
         test_comp(operator.le)
 
+    @pytest.mark.parametrize("op", ["lt", "le", "gt", "ge", "eq", "ne"])
path: pandas/tests/frame/test_operators.py
text: @user1:
Can you use the `compare_operators_no_eq_ne` pytest fixture instead?

@author:
Thanks, @user1 - that was neat! Now using `compare_operators_no_eq_ne`.