BUG: fix issue of boolean comparison on empty DataFrames (GH5808)
State: closed

closes #5808


Labels:

Comments:
@jtratner I'd like you to look at this

I actually think my 'fix' is wrong, as it patches an arithmetic methods, but
shouldn't a bool/flex method be called here?

maybe this is the case of that you have here:  https://github.com/pydata/pandas/blob/master/pandas/core/ops.py#L223

e.g. 'rand_' is called (I think because it is trying the reverse ops first?)

@jtratner if you have a chance...

@jreback sorry missed this I'll take a look now.

It's sort of self-documented (though not really well) which method does what in ops:

``` python
frame_special_funcs = dict(arith_method=_arith_method_FRAME,
                           radd_func=_radd_compat,
                           comp_method=_comp_method_FRAME,
                           bool_method=_arith_method_FRAME)
```

and frame flex funcs don't include the bool methods...

frame never had a separate bool method.  Ignoring TypeErrors there feels wrong - wouldn't it silence other actually useful TypeErrrors?

Why couldn't we just check whether either or both are empty and then skip the transformation op if so?

The reason that I popped the `r*` bool methods is because they didn't work (there's an issue about that), and I felt it was better to have a NotImplemented rather than a weird error.

@jtratner 

ok so for Series we have a `_bool_method_SERIES`.

for Frame I guess didn't exist before and is handled by arithmetic methods.

Biggest problem is I cannot find ANY tests here with boolean on frames.

I guess I can just handle the empties.

Problem is have all kinds of weirdness if these are treated as arithmetic operators

these should all raise, but an int and bool are ok. I think the dtypes have to be restricted for this to properly work.
IIRC we had some similar issues with Series

```
In [16]: DataFrame(True,index=[1],columns=['A']) & DataFrame(5,index=[1],columns=['A'])
Out[16]: 
   A
1  1

[1 rows x 1 columns]

In [17]: DataFrame(True,index=[1],columns=['A']) & DataFrame(5.0,index=[1],columns=['A'])
TypeError: ufunc 'bitwise_and' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

In [18]: DataFrame(1,index=[1],columns=['A']) & DataFrame(5.0,index=[1],columns=['A'])
TypeError: ufunc 'bitwise_and' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
```

```
In [19]: Series(1)&Series(2)
Out[19]: 
0    True
dtype: bool

In [20]: Series(1.)&Series(2)
TypeError: unsupported operand type(s) for &: 'float' and 'bool'

In [21]: Series(1)&Series(True)
Out[21]: 
0    True
dtype: bool
```

I agree, it's totally weird, but that's what we had previously. So do you want to just coerce it to bool first or are you going to restrict it to bool dtype only?

I think I need to add `bool_method_FRAME` otherwise too many ifs

that makes sense, that should simplify things a bit - though still need to mask it.

no perfect but preserves functionaility (and handles the empty case)

current format looks better, `np.prod` is relatively clear.