REGR: setting column with setitem should not modify existing array inplace
State: closed

Closes #33457

This changes `ExtensionBlock.setitem` to how it was before #32831: updating the Block itself inplace, but not the array. 

@jbrockmendel I know you won't like that this update the Block inplace (it's also not exactly what you suggested at https://github.com/pandas-dev/pandas/issues/33457#issuecomment-654367183). But this was the smallest change I could find to fix the issue. 

The alternative is probably changing `BlockManager.iset` to not use Block.set in case of an existing ExtensionBlock, but `BlockManager.iset` is quite complex ..


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
> The alternative is probably changing BlockManager.iset to not use Block.set in case of an existing ExtensionBlock, but BlockManager.iset is quite complex ..

@jbrockmendel quickly tried this, and so changing `BlockManager.iset` to create a new Block is not actually that complex. However, both `iloc` and `__setitem__` use this under the hood, so this then doesn't preserve the change for `iloc` you did in #32831. So not directly sure how to solve that.


```diff
diff --git a/pandas/core/internals/managers.py b/pandas/core/internals/managers.py
index c82670106..859e2b149 100644
--- a/pandas/core/internals/managers.py
+++ b/pandas/core/internals/managers.py
@@ -1055,11 +1055,16 @@ class BlockManager(PandasObject):
         unfit_mgr_locs = []
         unfit_val_locs = []
         removed_blknos = []
+        blocks = list(self.blocks)
         for blkno, val_locs in libinternals.get_blkno_placements(blknos, group=True):
-            blk = self.blocks[blkno]
+            blk = blocks[blkno]
             blk_locs = blklocs[val_locs.indexer]
             if blk.should_store(value):
-                blk.set(blk_locs, value_getitem(val_locs))
+                if blk.is_extension:
+                    new_blk = blk.make_block_same_class(value)
+                    blocks[blkno] = new_blk
+                else:
+                    blk.set(blk_locs, value_getitem(val_locs))
             else:
                 unfit_mgr_locs.append(blk.mgr_locs.as_array[blk_locs])
                 unfit_val_locs.append(val_locs)
@@ -1081,8 +1086,10 @@ class BlockManager(PandasObject):
             new_blknos[~is_deleted] = np.arange(self.nblocks - len(removed_blknos))
             self._blknos = new_blknos[self._blknos]
             self.blocks = tuple(
-                blk for i, blk in enumerate(self.blocks) if i not in set(removed_blknos)
+                blk for i, blk in enumerate(blocks) if i not in set(removed_blknos)
             )
+        else:
+            self.blocks = tuple(blocks)
```
@jbrockmendel @jreback I also opened an alternative PR to consider: https://github.com/pandas-dev/pandas/pull/35271, which is a more targeted revert of the original #32831 bug fix (simpler, but also not preserving the original fix that caused the regression).
Given there are clearly some side effects here (failing tests related to `replace`), @jreback @jbrockmendel can you take a look at the above-mentioned alternative PR (a more targetted revert): https://github.com/pandas-dev/pandas/pull/35254
I'll take fresh look at this.  As of now this is the only item on my todo list before 1.1.  LMK if there's anything else that needs my immediate attention.
Closing this. I don't think it'll be necessary after https://github.com/pandas-dev/pandas/pull/32831 and https://github.com/pandas-dev/pandas/pull/35417.