DEPR: raise deprecation warning in numpy ufuncs on DataFrames if not aligned + fallback to <1.2.0 behaviour
State: closed

Closes #39184

This is obviously a last-minute change, but if people agree on the deprecation, I think we should try to include it in v1.2.1. I *think* my patch is relatively safe, since converting the input to numpy arrays (what I am doing now manually as fallback) is what happened before adding `DataFrame.__array_ufunc__` as well. 

It adds quite some lines of code, but it's mostly some simple checking of the exact case which is a bit verbose.

The specific tests I added were verified to pass on pandas 1.1.5, so they codify the previous behaviour (minus the warnings).

cc @TomAugspurger 




Labels:
name: Blocker
description: Blocking issue or pull request for an upcoming release
name: Deprecate
description: Functionality to remove in pandas

Comments:
@jreback could you at least read what it is about and give your opinion about that? Because it is reverting behavior that was changed in 1.2.0, it exactly *is* timely to do it for 1.2.1
it is not timely to do things at the last minute

we have had several last minute changes in the past which have been a disaster

-1000 on merging this now
Thanks for the feedback @simonjayhawkins, pushed an update
we are pretty much on top of the regressions from 1.2, so if a short delay enables to clear the list, it may be worth considering.

but agreed should not be rushed
FWIW I'm leaning to prefer keeping the breaking change from the consistency with Series argument and avoiding the code changes by users to avoid the warnings.
> See #39184

this would close?
Yes, this closes that issue, will update the top post.
> we are pretty much on top of the regressions from 1.2, so if a short delay enables to clear the list, it may be worth considering.

If delaying the release with 1 or 2 days helps getting this merged, I think that is worth it.
Maybe add something in 1.2 release notes after `Calling a binary-input NumPy ufunc on multiple DataFrame objects now aligns, matching the behavior of binary operations and ufuncs on Series (:issue:23743).` like: This change has been reverted and deprecated instead in pandas 1.2.1, see :doc:`v1.2.1` " or link direct to sub section
> If delaying the release with 1 or 2 days helps getting this merged, I think that is worth it.

i'll add the blocker tag here until there is consensus to re-open up the 1.2.1 milestone for new issues/PRs e.g. #39253 that need not block, but could potentially be completed before this. 
I think this should go in the next release. Worth blocking for a day or two.

This code is very similar to a deprecation we did for Series. The reviews there help lower the burden for DataFrame.
@jorisvandenbossche did you see https://github.com/pandas-dev/pandas/pull/39239#issuecomment-762444475

(also since this PR is the blocker could maybe also update the release date in the notes to maybe save an extra ci/backport cycle)
@simonjayhawkins yeah, I saw that, thanks for the reminder, as I still need to do that. 
I was maybe thinking to also add a subsection (or mainly the title + short summary) to the deprecations section, linking to the 1.2.1 page. Although the deprecation was not in 1.2.0 itself, people wanting to know what changed in 1.2.x in general will still mostly look at the 1.2.0 page. 

> (also since this PR is the blocker could maybe also update the release date in the notes to maybe save an extra ci/backport cycle)

Good idea. What's your current idea about the timeline? (eg try to merge this PR this evening, and start release process tomorrow morning? in which case I pick the date of tomorrow)
Actaully, we don't have subsections yet in the deprecations section in v1.2.0.rst, so just did the clarification of the original whatsnew note as you suggested @simonjayhawkins 
I normally like to start the release nearer to the start of the day, but could get going on the final pre-release checks https://github.com/pandas-dev/pandas/issues/38721#issuecomment-762226060 as soon as this is backported.

I think the date should match the tag (and we discussed templating this https://github.com/pandas-dev/pandas/issues/21050#issuecomment-687082687) which may not match the github release if the release spans a couple of days.

If we're not sure, maybe best to leave out of this PR and I could expedite the change by not waiting for ci to complete (i normally wait for ci to complete which can add a couple of hours to the release process, for the change to master and again for the backport PR)

AFAIK the remaining comment is a doc comment on the whatsnew notes, and since this is a somewhat subjective style discussion / not critical IMO, I am going to take the liberty to merge this, so @simonjayhawkins can start the release process early in the day once this is backported and builds have passed. 
(but, Jeff, I am happy to further discuss it and do a follow-up PR, we can still update those docs in the 1.2.x cycle and those will already be out in a few weeks)

@simonjayhawkins I also updated the date in the release notes here.
@MeeseeksDev backport to 1.2.x
@jorisvandenbossche pls following up and fix the docs


it's not a style issue rather this is completely inconsistent with the current docs 

we NEVER use the style - always ipython docs style