BUG: Bug in loc did not change dtype when complete column was assigned
State: closed

- [x] closes #20635
- [x] closes #20511
- [x] closes #27583
- [x] closes #33198
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Stale

Comments:
Hello @phofl! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:













There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-02-13 20:38:58 UTC
requests re test names/placement, others LGTM
Added code to fix a related issue
According to #27583 the test I have fixed behaved wrong
Does this fix #33198?
Yep, will add test. 

The failing test can't be fixed that easily. We now convert to int if an integer is set to a whole df. Numpy does not in the expected case
> The failing test can't be fixed that easily. We now convert to int if an integer is set to a whole df. Numpy does not in the expected case

Long day.  can you remind me which test you're referring to?
Forgot to push yesterday, ``test_partial_set``
The case described in #37593 fails now again, because split path can not handle Series assignment into a single cell. Will look into this further tomorrow
Puuuuh,

cc @jbrockmendel 
The case
```
df = DataFrame(columns=["a"], index=[0])
rhs = Series([1, 2, 3])
df.iloc[0, 0] = rhs
```

runs now through ``_setitem_with_indexer_split_path`` instead of ``_setitem_single_block``. This raises 
```
Traceback (most recent call last):
  File "/home/developer/.config/JetBrains/PyCharm2020.2/scratches/scratch_4.py", line 158, in <module>
    df.iloc[0, 0] = rhs
  File "/home/developer/PycharmProjects/pandas/pandas/core/indexing.py", line 689, in __setitem__
    iloc._setitem_with_indexer(indexer, value, self.name)
  File "/home/developer/PycharmProjects/pandas/pandas/core/indexing.py", line 1643, in _setitem_with_indexer
    self._setitem_with_indexer_split_path(indexer, value, name)
  File "/home/developer/PycharmProjects/pandas/pandas/core/indexing.py", line 1689, in _setitem_with_indexer_split_path
    raise ValueError(
ValueError: Must have equal len keys and value when setting with an iterable
```

The case is pretty weird with setting a Series into a single cell. Any idea how we could accomplish this within ``_setitem_with_indexer_split_path``?
> The case is pretty weird with setting a Series into a single cell. Any idea how we could accomplish this within _setitem_with_indexer_split_path?

so ive mentioned a couple of times an upcoming PR that will make all DataFrame cases go through split_path.  To make that work there are two kludgy checks I have to do near the top of the method (also non-kludges #37931 and #37932):

```
        info_idx = indexer[1]
        pi = indexer[0]

        if com.is_null_slice(info_idx) and is_scalar(value):
            # We can go directly through BlockManager.setitem without worrying
            #  about alignment.
            # TODO: do we need to do some kind of copy_with_setting check?
            self.obj._mgr = self.obj._mgr.setitem(indexer=indexer, value=value)
            return

        if is_integer(info_idx):
            if is_integer(pi):
                # We need to watch out for case where we are treating a listlike
                #  as a scalar, e.g. test_setitem_iloc_scalar_single for JSONArray

                mgr = self.obj._mgr
                blkno = mgr.blknos[info_idx]
                blkloc = mgr.blklocs[info_idx]
                blk = mgr.blocks[blkno]

                if blk._can_hold_element(value):
                    # NB: we are assuming here that _can_hold_element is accurate
                    # TODO: do we need to do some kind of copy_with_setting check?
                    self.obj._check_is_chained_assignment_possible()
                    blk.setitem_inplace((pi, blkloc), value)
                    self.obj._maybe_update_cacher(clear=True)
                    return
```

where I've defined `Block.setitem_inplace`:

```
    def setitem_inplace(self, indexer, value):
        """
        setitem but only inplace.

        Notes
        -----
        Assumes self is 2D and that indexer is a 2-tuple.
        """
        if lib.is_scalar(value) and not self.is_extension:
            # Convert timedelta/datetime to timedelta64/datetime64
            value = convert_scalar_for_putitemlike(value, self.dtype)

        pi = indexer[0]
        if self.is_extension:
            # TODO(EA2D): not needed with 2D EAs
            self.values[pi] = value
        else:
            blkloc = indexer[1]
            self.values[blkloc, pi] = value
```

Looks like you're dealing with roughly the same problem that drove me to put in the double is_integer checks.
It is exactly the same problem. If there is not elegant way, we could check if we have this case before setting split path to True. But this makes only sense if your pr does not arrive before 1.2 and this could go in
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@jbrockmendel Is your PR with always split path still planned?
status here?
Depends, this sends a few more code paths down the split path in indexing, but I don't know if @jbrockmendel has something other in mind here?
@phofl what's the status on this?
@phofl closing as stale. reopen when ready
@phofl is this worth reviving?  AFAICT you were waiting on me to get the always-split-path PR (#40380) in and that is stuck in limbo with a few tests failing locally.