Repository: pandas-dev/pandas

Issue:
BUG: scalar assignment of a tz-aware is object dtype
[3] should be a ``datetime64[ns, UTC]``

```
In [1]: df = pd.DataFrame({'A': [0, 1]})

In [3]: df['now'] = pd.Timestamp('20130101', tz='UTC')

In [4]: df
Out[4]: 
   A                        now
0  0  2013-01-01 00:00:00+00:00
1  1  2013-01-01 00:00:00+00:00

In [5]: df.dtypes
Out[5]: 
A       int64
now    object
dtype: object

In [6]: df['now2'] = pd.DatetimeIndex([pd.Timestamp('20130101', tz='UTC')]).repeat(len(df))

In [7]: df.dtypes
Out[7]: 
A                     int64
now                  object
now2    datetime64[ns, UTC]
dtype: object
```




Discussion:
I will try and fix this.
great!
Currently, ```infer_dtype_from_scalar``` (on datetimey/timestampy objects) returns a ```np.datetime64``` if no timezone is given, and defaults to ```np.object_``` on objects with timezones. Fixing this problem means returning something else, rather than ```np.object_```. 

Ideally return ```DatetimeTZDtypeType```. However, this crashes on ```np.empty(shape, dtype=dtype)``` in ```cast_scalar_to_array```. Seems like this should work, but it doesn't.

Quick fix is returning ```np.datetime64``` rather than ```np.object_```. You lose the timezone name, but numpy applies the correct offset before saving so the numbers are correct. This change doesn't break any tests, and results in the following behavior:

```
In [1]: df = pd.DataFrame({'A': [0, 1]})

In [3]: df['now'] = pd.Timestamp('20130101', tz='UTC')

In [5]: df.dtypes
Out[5]: 
A               int64
now    datetime64[ns]
dtype: object

In [6]: df['now2'] = pd.DatetimeIndex([pd.Timestamp('20130101', tz='UTC')]).repeat(len(df))

In [7]: df.dtypes
Out[7]: 
A                     int64
now          datetime64[ns]
now2    datetime64[ns, UTC]
dtype: object
```
Raises some inconsistencies, potentially problems with mixing in timezone-naive datetimes. Is the quick fix good enough?
@DylanDmitri you don't want to *ever* have numpy deal with timezones, they are completely wrong. ``infer_dtype_from_scalar`` has a ``pandas_dtype`` parameter that will make this work. We should actully just change this to do this by default (though this might break other things)
Been busy the last week, sorry.  Here's the problem code (from line 2874 of ```frame.py```)

```
# BEFORE
value = cast_scalar_to_array(len(self.index), value)
value = maybe_cast_to_datetime(value, value.dtype)
```

Main issue: ```cast_scalar_to_array``` defaults to dtype ```np.object_```, which is then ignored by ```maybe_cast_to_datetime```. Want to capture the real pandas dtype, and then pass that into ```maybe_cast_to_datetime```, which then works properly.

```
# AFTER
from pandas.core.dtypes.cast import infer_dtype_from_scalar
pandas_dtype, _ = infer_dtype_from_scalar(value, pandas_dtype=True)

value = cast_scalar_to_array(len(self.index), value)
value = maybe_cast_to_datetime(value, pandas_dtype)
```

This fixes the problem. Will check tests, and have a PR soon.

Resolving patch:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -27,6 +27,7 @@
     maybe_upcast,
     cast_scalar_to_array,
     construct_1d_arraylike_from_scalar,
+    infer_dtype_from_scalar,
     maybe_cast_to_datetime,
     maybe_infer_to_datetimelike,
     maybe_convert_platform,
@@ -3507,9 +3508,13 @@ def reindexer(value):
                 value = maybe_infer_to_datetimelike(value)
 
         else:
-            # upcast the scalar
+            # cast ignores pandas dtypes. so save the dtype first
+            infer_dtype, _ = infer_dtype_from_scalar(
+                value, pandas_dtype=True)
+
+            # upcast
             value = cast_scalar_to_array(len(self.index), value)
-            value = maybe_cast_to_datetime(value, value.dtype)
+            value = maybe_cast_to_datetime(value, infer_dtype)
 
         # return internal types directly
         if is_extension_type(value) or is_extension_array_dtype(value):


Test patch:
diff --git a/pandas/tests/frame/test_indexing.py b/pandas/tests/frame/test_indexing.py
--- a/pandas/tests/frame/test_indexing.py
+++ b/pandas/tests/frame/test_indexing.py
@@ -3135,6 +3135,14 @@ def test_transpose(self):
         expected.index = ['A', 'B']
         assert_frame_equal(result, expected)
 
+    def test_scalar_assignment(self):
+        # issue #19843
+        df = pd.DataFrame(index=(0, 1, 2))
+        df['now'] = pd.Timestamp('20130101', tz='UTC')
+        expected = pd.DataFrame(
+            {'now': pd.Timestamp('20130101', tz='UTC')}, index=[0, 1, 2])
+        tm.assert_frame_equal(df, expected)
+
 
 class TestDataFrameIndexingUInt64(TestData):