Repository: pandas-dev/pandas

Issue:
DataFrame.iat will create new column if .iat is used to set None on int Series
#### Code Sample, a copy-pastable example if possible

```python
>>> df = pd.DataFrame({'a':[0,1],'b':[4,5]})
>>> df
   a  b
0  0  4
1  1  5
>>> df.iat[0, 0] = None
>>> df
   a  b   0
0  0  4 NaN
1  1  5 NaN
```
#### Problem description

This is problematic for multiple reasons.
 - inconsistency between iloc and iat. 
 - creation of a brand new column is almost surely not the intended/expected behavior
 - At the very least, I would expect it to simply bail on the operation with a warning about incompatible types.

This is likely related to the non-intuitive behavior of Series which has already been documented here:
https://github.com/pandas-dev/pandas/issues/20643#issuecomment-431244590

#### Expected Output
I would expect it to do what it does when using `.iloc`:
```python
>>> df = pd.DataFrame({'a':[0,1],'b':[4,5]})
>>> df
   a  b
0  0  4
1  1  5
>>> df.iloc[0, 0] = None
>>> df
     a  b
0  NaN  4
1  1.0  5
```

#### Output of ``pd.show_versions()``

<details>
INSTALLED VERSIONS
------------------
commit: None
python: 3.6.6.final.0
python-bits: 64
OS: Linux
OS-release: 3.13.0-24-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.utf8
LOCALE: en_US.UTF-8

pandas: 0.22.0
pytest: None
pip: 9.0.3
setuptools: 39.0.1
Cython: 0.27.3
numpy: 1.14.0
scipy: 1.0.0
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: 0.5.0
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: 0.1.2
fastparquet: None
pandas_gbq: None
pandas_datareader: None
</details>



Discussion:
I tried it with 0.23.4 and had the same output.
<details>
INSTALLED VERSIONS
------------------
commit: None
python: 3.6.6.final.0
python-bits: 64
OS: Linux
OS-release: 3.13.0-24-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.utf8
LOCALE: en_US.UTF-8

pandas: 0.23.4
pytest: None
pip: 9.0.3
setuptools: 39.0.1
Cython: 0.27.3
numpy: 1.14.0
scipy: 1.0.0
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: 0.5.0
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: 0.1.2
fastparquet: None
pandas_gbq: None
pandas_datareader: None
</details>
Thanks for the report. Investigations and PRs welcome!
 I think we can conclude that as df.iat is capable of creating and setting values,it will create index for 0 which is not present as column index(a,b).
We cannot access those values using iat if column index are char type.


`iat` indexes by integer position and not label, so it shouldn't matter if `0` is not in the columns; it should modify the value in row `0` column `A`

IIRC `iat` and `at` doesn't perform as many data validation checks, so this may be a "fallback" assignment and broadcasting.
I do not agree with @aditya0811 :
>I think we can conclude that as df.iat is capable of creating and setting values,it will create index for 0 which is not present as column index(a,b).

As explained by @mroeschke here:
> iat indexes by integer position and not label, so it shouldn't matter if 0 is not in the columns; it should modify the value in row 0 column A

Resolving patch:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -2716,7 +2716,10 @@ def _set_value(self, index, col, value, takeable=False):
         except (KeyError, TypeError):
 
             # set using a non-recursive method & reset the cache
-            self.loc[index, col] = value
+            if takeable:
+                self.iloc[index, col] = value
+            else:
+                self.loc[index, col] = value
             self._item_cache.pop(col, None)
 
             return self


Test patch:
diff --git a/pandas/tests/indexing/test_scalar.py b/pandas/tests/indexing/test_scalar.py
--- a/pandas/tests/indexing/test_scalar.py
+++ b/pandas/tests/indexing/test_scalar.py
@@ -198,3 +198,10 @@ def test_mixed_index_at_iat_loc_iloc_dataframe(self):
             df.at[0, 3]
         with pytest.raises(KeyError):
             df.loc[0, 3]
+
+    def test_iat_setter_incompatible_assignment(self):
+        # GH 23236
+        result = DataFrame({'a': [0, 1], 'b': [4, 5]})
+        result.iat[0, 0] = None
+        expected = DataFrame({"a": [None, 1], "b": [4, 5]})
+        tm.assert_frame_equal(result, expected)