BUG: Fix numpy boolean subtraction error in Series.diff
State: closed

This fixes #27755, for more than three years now NumPy has not allowed the subtraction of boolean series. 

TypeError                                 Traceback (most recent call last)
<ipython-input-46-3da3b949c6bd> in <module>
      1 data = pd.Series([0,-1,-2,-3,-4,-3,-2,-1,0,-1,-1,0,-1,-2,-3,-2,0])
      2 filtered = data.between(-2,0, inclusive = True)
----> 3 filtered.diff()
      4 print(filtered)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\series.py in diff(self, periods)
   2191         dtype: float64
   2192         """
-> 2193         result = algorithms.diff(com.values_from_object(self), periods)
   2194         return self._constructor(result, index=self.index).__finalize__(self)
   2195 

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\algorithms.py in diff(arr, n, axis)
   1817             out_arr[res_indexer] = result
   1818         else:
-> 1819             out_arr[res_indexer] = arr[res_indexer] - arr[lag_indexer]
   1820 
   1821     if is_timedelta:

TypeError: numpy boolean subtract, the `-` operator, is deprecated, use the bitwise_xor, the `^` operator, or the logical_xor function instead.

- [x] closes #17294
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:

Comments:
Hello @Unprocessable! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-08-10 23:56:48 UTC
Are there other tests for diff?  If so they should be collected here or this should join them.
I cannot get the import order fixed correctly, on my PC isort does not show any changes, but in the checks it does show up.
For now I think it's fine to just add the new test to pandas.tests.series.test_analytics.TestSeriesAnalytics. There's a `test_npdiff` there that's related.

Then you can revert the other test changes, and it's easier to see what's new.
@Unprocessable do you have time to update with the recommendations from https://github.com/pandas-dev/pandas/pull/27755#issuecomment-519275906?
Apparently there are some problems with tslibs in pandas' \_\_init\_\_.py. Apart from that it should be fine. 

@jbrockmendel @TomAugspurger Feel free to comment on it again or merge it. I might have some time tomorrow for some quick fixes if required. I will not get around to fixing larger bugs in the next few weeks and will definitely not re-factor a whole code base to use functions/fixtures.
Just checked it, everyone seems to have that problem with pandas-master at the moment, so it is not due to my code. Please fix the master and rerun the tests if you're unsure, otherwise feel free to merge. Let's finally fix this three year old bug!
Hopefully fixed in https://github.com/pandas-dev/pandas/pull/27845.

If you can merge master & re-push it should be fixed.

On Sun, Aug 11, 2019 at 5:50 PM Unprocessable <notifications@github.com>
wrote:

> Just checked it, everyone seems to have that problem with pandas-master at
> the moment, so it is not due to my code. Please fix the master and rerun
> the tests if you're unsure, otherwise feel free to merge. Let's finally fix
> the three year old bug!
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/27755?email_source=notifications&email_token=AAKAOITLPRNNCVJZNYX77TLQECJT5A5CNFSM4IJKX662YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4BKQDQ#issuecomment-520267790>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAKAOIQMFBEF6BK3MRISATTQECJT5ANCNFSM4IJKX66Q>
> .
>

> 
> Hopefully fixed in #27845. If you can merge master & re-push it should be fixed.
> 
> I might have some time tomorrow for some quick fixes if required. 


No, I hate to say this, but I guess you are in a different time zone and tomorrow means something different. Sorry, this will have to wait a few weeks or it will have to go as is.

No rush, thanks.

On Mon, Aug 12, 2019 at 2:02 AM Unprocessable <notifications@github.com>
wrote:

> Hopefully fixed in #27845
> <https://github.com/pandas-dev/pandas/pull/27845>. If you can merge
> master & re-push it should be fixed.
>
> I might have some time tomorrow for some quick fixes if required.
> No, I hate to say this, but I guess you are in a different time zone and
> tomorrow means something different. Sorry, this will have to wait a few
> weeks or it will have to go as is.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/27755?email_source=notifications&email_token=AAKAOISFOX3RZMRGYZ3CXSLQEEDJ7A5CNFSM4IJKX662YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4BWSIY#issuecomment-520317219>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAKAOIV5XK24JOIGOGGTKI3QEEDJ7ANCNFSM4IJKX66Q>
> .
>

superseded by #28251