Make div use truediv instead of div
State: closed

This changes the `div()` method in pandas on Python 2.x to always do
`truediv` rather than integer division. Currently, pandas doesn't
respect  `from __future__ import division` (it's defined in a file that
doesn't start with the import, so it always does integer division).
`__div__`, `__rdiv__`, and `__idiv__` remain the same, so that the `/`
operator still works as expected. This PR also adds `truediv()` and
`floordiv()` methods to series, panel and frame, plus a test utility
to generate integer series.

This change makes `div()` behave the same way in Python 2 and in
Python 3. It also limits the surprise that comes from using the
future import with pandas in Python 2.

You can see the difference between Python 2 and 3 with the following
snippet:

```

from __future__ import division
import pandas
from pandas.util.testing import assert_frame_equal
df = pandas.DataFrame({"A": range(4)})
result = df.div(4)
expected = df / 4
assert_frame_equal(result, expected)

```

It's not easy (or particularly useful) to detect which division setting
is currently active. Plus, if you were to check it, the actual behavior
would depend on [which file imports `pandas`
first](http://stackoverflow.com/questions/16880552/dynamically-detecting-division-future-import).

There might be too many tests associated with this, so if you'd like me
to slim them down, I'd be happy to do so.

Another alternative would to this patch would be to follow the python
`operator` convention and remove `div()` altogether from Python 3, while
leaving `div()` as it previously was in Python 2; however, I think that
will continue to suprise users.


Labels:

Comments:
I like the alternative you present at the bottom. Why is that behavior surprising? I can see that if you're completely new to Python or you've only been using Python 3 and living in a bubble and for some reason you need to switch to Python 2, but if you're not new and you don't live in a bubble then this would be the most consistent with the language. I think maintaining _that_ invariant is important until the entire scientific community moves to Python 3 which may be a long time. Have `div` and `truediv` methods just like the `operator` module. What do you mean by pandas doesn't respect `from __future__ import division`? If you import it in an ipython session it works fine.

@cpcloud What I mean by surprising is that, if you run `from __future__ import division` and then try to use the `div` method, it only does integer division. (try running the code snippet I posted).

I'd argue that if you know nothing about special methods like `__div__` and `__truediv__`, it's counter-intuitive that `div` doesn't change to doing true division when you use `from __future__ import division`. The `/` operator feels synonymous to division.

That said, it's a pretty quick error to notice, so I guess just adding `floordiv`, `truediv` in general and removing `div` from Python 3 would work.  Is there a way to add something to the docstring for `div` that indicates that it will only do integer division?

i c what u mean. the reason why `div` doesn't work even when you import division is because that's how the `operator` module works `operator.div(1, 2)` is invariant to the division import. i'm guessing u know that already though. i don't agree that div should change to true division (except of course in python 3), but others may feel different. the fewer things that conflict with python's conventions the better. not necessarily because i think python's conventions are the "best" (whatever that means), but because they are so widely used.

- python 2 `div` and `truediv` maybe `floordiv` since the syntax `a // b` is valid, just make `div` an alias for `floordiv` or vice versa
- python 3 `floordiv` and `div`

@cpcloud well, it's actually because the `lambda x, y: x / y` is executed in the context of `pandas.core.frame`, which doesn't have the `from __future__ import division` enabled.

`operator.div` is gone in Python 3, so under that scenario it should be gone from pandas too, right?

oh ur right i didn't acutally look at the function my bad. i assumed pandas just passed the operator module funcs

and it does now that i look except for a few

@cpcloud I'm not sure why that is, actually. Maybe it was an attempt to handle the div / truediv difference? The test cases didn't explicitly handle integer division, so I'm unclear. I left it as close to the original as I could.

it looks like that is what's happening, but i think there's a scoping issue here since the import is not respected

not sure why `lambda x, y: x / y` doesn't respect the import. doesn't look like there's an easy way to add notes to a docstring appender. what u could do is add a default string interpolating function, although passing parameters to a function that formats strings is really annoying. but it might be ok for this case

Since `from __future__ import division` only affects the current file, so you'd need to do a bunch of workarounds to make sure this happens. It's also slightly problematic throughout, because any function that uses the div method anywhere will run differently in Python 3 and Python 2.

That said, I realize I missed adding truediv and floordiv to other containers (sparse versions, etc.), so I think it makes more sense to close this, split out the commits to add `truediv` and `floordiv` to all the containers in one PR and then write up another that suggests changing the behavior of `div` and removing `div` from the Python 3. 

@cpcloud okay, further update - this probably doesn't work correctly with numexpr, since it would use the string representation of `/` and probably do the wrong thing at that point. So, definitely more to think over with changing div and truediv (and probably a good idea to add a test case that explicitly checks that, with numexpr installed, it does `div` and `truediv` appropriately).

indeed numexpr enforces the numpy convention on everything so it will be hard to get around that without a bunch of casts i think.

why don't we do something like evaluate division always wih astype(float) then cast back if we can (we do this is other cases as well)?

as a side note - does numpy respect div / truediv? 
numpy was buggy for mod wih integers should be straightforward to do something similar here I think