PDEP-6: Ban upcasting in setitem-like operations
State: closed

Original discussion: #39584

Turning into a PDEP as I think the `int -> float` warrants some care

Labels:
name: PDEP
description: pandas enhancement proposal

Comments:
I like it!

2 questions

1. You have mentioned int vs float upcast/downcast as being an open item. What about upcasts within ints - for example if an out of bounds integer was set in an int8 series. (Currrently this upcasts to int16 / int32 etc. as appropriate)

2. Similarly - What about setting pd.NA or np.nan to an int series (non-nullable).  Currently this upcasts to float
Thanks @asishm for taking a look! Setting an out-of-bounds value in an `int8` Series, or `np.nan` in any `'int'` (non-nullable) Series, would also raise
Thanks for your reviews!

Right, I think it's time for yet another @pandas-dev/pandas-core @pandas-dev/pandas-triage ping...would love to hear people's thoughts on this
I think you should be more complete on all the operations that would be affected.  `s.loc[1] = 1.5` would also raise under this proposal.  In addition, setting values of a column in a `DataFrame` would also raise.

Also, we have discussed the possibility of eliminating `inplace = True`.  If we did that first, wouldn't then the only places where this PDEP would apply be in `Series.__setitem__()` and `Series.loc.__setitem__()`  (and the corresponding `DataFrame` places as well)?


I like the ideas here and would be in favor of the strict option for the float / int case. I think its easy enough to work around if that's really what you want and would enforce more consistent behavior within our own library
is the intention to follow for other routines which can change dtype (either other pdep or as a follow up)

eg fillna, where?
does this proposal include (or explicitly exclude) setitem-like operations with enlargement?

For instance, setting a float into a Series with nullable integer.

s = Series([1,2,3], dtype="Int64")

s[1] = 1.5 currently raises
s[3] = 1.5 upcasts to Float64

This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
> This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.

I'm intentionally waiting for 2.0 to be released before going to far with this one - looks like there's not long left, so I'll pick this up again soon
Thanks all for your comments! I'm picking this up again, since the 2.0.0 release candidates are out. Have updated with respect to comments
As per yesterday's call, I've marked this as 'under discussion'

Unless major discussion points come up, let's start the voting process in a couple of weeks
> As per yesterday's call, I've marked this as 'under discussion'
> 
> Unless major discussion points come up, let's start the voting process in a couple of weeks

Right, let's do this? Let's do this! I declare the voting period to be open!

[The voting process](https://docs.google.com/document/d/1QmV5fdzZ0i2cfpp1QkZyS1V8lWJlv5fh7yVw3qNlkPI/edit#heading=h.71c0n9ls3ixn) hasn't been formally agreed-upon, but as discussed on the March 22 2023 community meeting, let's try it out anyway so as to not leave this hanging around forever.

---

The process is:
1. @pandas-dev/pandas-core please comment:

  - 1: approve.
  - 0: abstain (reason: a one sentence reason is required)
  - -1: disapprove (reason: a one sentence reason is required)

2. voting will be open for 15 days. We'll need:

  - a quorum to be achieved (at least 11 votes - abstaining votes are counted)
  - 75% approval (excluding abstaining votes)
+1
+1
+1

Right, that's 12 upvotes, and no downvotes. Lot's of great discussion here, thanks all for the attention to detail, much appreciated!

Merging this today then