Repository: pandas-dev/pandas
Pull request: Assignment of column via .loc for numpy non-ns datetimes
- [x] closes #27395 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Problem statement:
Assignment of column via .loc
```python
# This code is failing
df = pd.DataFrame({'timestamp': [np.datetime64('2017-01-01 01:00:00'), np.datetime64('2017-01-01 02:00:00')],
                   'value': ['a', 'b']})
df.loc[:, 'day'] = df.loc[:, 'timestamp'].values.astype('datetime64[D]')
```
#### Error
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-91-7e5c52b6ed28> in <module>()
----> 1 df.loc[:, 'day'] = df.loc[:, 'timestamp'].values.astype('datetime64[D]')

/home/homola/env/venv/local/lib/python2.7/site-packages/pandas/core/indexing.pyc in __setitem__(self, key, value)
    188             key = com.apply_if_callable(key, self.obj)
    189         indexer = self._get_setitem_indexer(key)
--> 190         self._setitem_with_indexer(indexer, value)
    191 
    192     def _validate_key(self, key, axis):

/home/homola/env/venv/local/lib/python2.7/site-packages/pandas/core/indexing.pyc in _setitem_with_indexer(self, indexer, value)
    360 
    361                         # add a new item with the dtype setup
--> 362                         self.obj[key] = _infer_fill_value(value)
    363 
    364                         new_indexer = convert_from_missing_indexer_tuple(

/home/homola/env/venv/local/lib/python2.7/site-packages/pandas/core/frame.pyc in __setitem__(self, key, value)
   3368         else:
   3369             # set column
-> 3370             self._set_item(key, value)
   3371 
   3372     def _setitem_slice(self, key, value):

/home/homola/env/venv/local/lib/python2.7/site-packages/pandas/core/frame.pyc in _set_item(self, key, value)
   3443 
   3444         self._ensure_valid_index(value)
-> 3445         value = self._sanitize_column(key, value)
   3446         NDFrame._set_item(self, key, value)
   3447 

/home/homola/env/venv/local/lib/python2.7/site-packages/pandas/core/frame.pyc in _sanitize_column(self, key, value, broadcast)
   3652             # upcast
   3653             value = cast_scalar_to_array(len(self.index), value)
-> 3654             value = maybe_cast_to_datetime(value, infer_dtype)
   3655 
   3656         # return internal types directly

/home/homola/env/venv/local/lib/python2.7/site-packages/pandas/core/dtypes/cast.pyc in maybe_cast_to_datetime(value, dtype, errors)
    979                 else:
    980                     raise TypeError("cannot convert datetimelike to "
--> 981                                     "dtype [{dtype}]".format(dtype=dtype))
    982             elif is_datetime64tz:
    983 

TypeError: cannot convert datetimelike to dtype [datetime64[D]]
```

```python
# This code is ok
df = pd.DataFrame({'timestamp': [np.datetime64('2017-01-01 01:00:00'), np.datetime64('2017-01-01 02:00:00')],
                   'value': ['a', 'b']})
df['day'] = df.loc[:, 'timestamp'].values.astype('datetime64[D]')
```

#### Problem description

When assigning to a column and using pandas.DataFrame.loc, it fails due to 'TypeError: cannot convert datetimelike to dtype [datetime64[D]]'. When using simple assignment, it doesn't fail. 
If it is a bad practice what is done above in the first example, the error message should be different.

#### Expected Outcome
Assignment is successful for both below.
```python
df.loc[:, 'day'] = df.loc[:, 'timestamp'].values.astype('datetime64[D]')
df['day'] = df.loc[:, 'timestamp'].values.astype('datetime64[D]')
```

#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.12.final.0
python-bits: 64
OS: Linux
OS-release: 4.15.0-51-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: en_US.utf8
LANG: en_US.UTF-8
LOCALE: None.None

pandas: 0.24.2
numpy: 1.16.4
...
</details>

Hints from issue discussion:
Looks like a bug. Interested in debugging where things go wrong?
May I take a look?

I'm on sprint at PyCon KR 2019
go ahead! @inmoonlight

Commit message: Fix lint error

Patch to review:
diff --git a/doc/source/whatsnew/v0.25.1.rst b/doc/source/whatsnew/v0.25.1.rst
index 21f1fa7ddec1f..b975ab76fdcc8 100644
--- a/doc/source/whatsnew/v0.25.1.rst
+++ b/doc/source/whatsnew/v0.25.1.rst
@@ -32,7 +32,7 @@ Categorical
 Datetimelike
 ^^^^^^^^^^^^
 - Bug in :func:`to_datetime` where passing a timezone-naive :class:`DatetimeArray` or :class:`DatetimeIndex` and ``utc=True`` would incorrectly return a timezone-naive result (:issue:`27733`)
--
+- Bug in :func:`maybe_cast_to_datetime` where converting a `np.datetime64` to `datetime64[D]` raise `TypeError` (:issue: `27395`)
 -
 -
 
diff --git a/pandas/core/dtypes/cast.py b/pandas/core/dtypes/cast.py
index 4bb1deffd9524..29b07b35a3614 100644
--- a/pandas/core/dtypes/cast.py
+++ b/pandas/core/dtypes/cast.py
@@ -1026,7 +1026,7 @@ def maybe_cast_to_datetime(value, dtype, errors="raise"):
             )
 
             if is_datetime64 and not is_dtype_equal(dtype, _NS_DTYPE):
-                if dtype.name in ("datetime64", "datetime64[ns]"):
+                if dtype.name in ("datetime64", "datetime64[ns]", "datetime64[D]"):
                     if dtype.name == "datetime64":
                         raise ValueError(msg.format(dtype=dtype.name))
                     dtype = _NS_DTYPE
@@ -1044,7 +1044,7 @@ def maybe_cast_to_datetime(value, dtype, errors="raise"):
                     value = [value]
 
             elif is_timedelta64 and not is_dtype_equal(dtype, _TD_DTYPE):
-                if dtype.name in ("timedelta64", "timedelta64[ns]"):
+                if dtype.name in ("timedelta64", "timedelta64[ns]", "timedelta64[D]"):
                     if dtype.name == "timedelta64":
                         raise ValueError(msg.format(dtype=dtype.name))
                     dtype = _TD_DTYPE
diff --git a/pandas/tests/dtypes/cast/test_infer_dtype.py b/pandas/tests/dtypes/cast/test_infer_dtype.py
index 602b2f26eaa4a..a204242c5c98d 100644
--- a/pandas/tests/dtypes/cast/test_infer_dtype.py
+++ b/pandas/tests/dtypes/cast/test_infer_dtype.py
@@ -7,6 +7,7 @@
     cast_scalar_to_array,
     infer_dtype_from_array,
     infer_dtype_from_scalar,
+    maybe_cast_to_datetime,
 )
 from pandas.core.dtypes.common import is_dtype_equal
 
@@ -169,3 +170,14 @@ def test_cast_scalar_to_array(obj, dtype):
 
     arr = cast_scalar_to_array(shape, obj, dtype=dtype)
     tm.assert_numpy_array_equal(arr, exp)
+
+
+@pytest.mark.parametrize(
+    "obj,dtype",
+    [
+        (np.datetime64("2017-01-01 01:00:00"), "datetime64[D]"),
+        (np.datetime64("2017-01-01 02:00:00"), "datetime64[D]"),
+    ],
+)
+def test_maybe_cast_to_datetime(obj, dtype):
+    maybe_cast_to_datetime(obj, dtype)


Code review comments:
diff_hunk: @@ -32,7 +32,7 @@ Categorical
 Datetimelike
 ^^^^^^^^^^^^
 - Bug in :func:`to_datetime` where passing a timezone-naive :class:`DatetimeArray` or :class:`DatetimeIndex` and ``utc=True`` would incorrectly return a timezone-naive result (:issue:`27733`)
--
+- Bug in :func:`maybe_cast_to_datetime` where converting a `np.datetime64` to `datetime64[D]` raise `TypeError` (:issue: `27395`)
path: doc/source/whatsnew/v0.25.1.rst
text: @user1:
this is a private function, you would have to indicate the user facing experience here instead (e.g. the assignment)

diff_hunk: @@ -1026,7 +1026,7 @@ def maybe_cast_to_datetime(value, dtype, errors="raise"):
             )
 
             if is_datetime64 and not is_dtype_equal(dtype, _NS_DTYPE):
-                if dtype.name in ("datetime64", "datetime64[ns]"):
+                if dtype.name in ("datetime64", "datetime64[ns]", "datetime64[D]"):
path: pandas/core/dtypes/cast.py
text: @user1:
just use ```dtype.kind == 'M'```

@author:
Since `pandas` doesn't support to convert [`datetime[ps]`](https://github.com/pandas-dev/pandas/blob/master/pandas/tests/series/test_constructors.py#L1352), I used `(dtype.kind == 'M') and (dtype.name != 'datetime64[ps]')`

diff_hunk: @@ -1044,7 +1044,7 @@ def maybe_cast_to_datetime(value, dtype, errors="raise"):
                     value = [value]
 
             elif is_timedelta64 and not is_dtype_equal(dtype, _TD_DTYPE):
-                if dtype.name in ("timedelta64", "timedelta64[ns]"):
+                if dtype.name in ("timedelta64", "timedelta64[ns]", "timedelta64[D]"):
path: pandas/core/dtypes/cast.py
text: @user1:
```dtype.kind == 'm'```

@author:
Since `pandas` doesn't support to convert [`timedelta[ps]`](https://github.com/pandas-dev/pandas/blob/master/pandas/tests/series/test_constructors.py#L1352), I used `(dtype.kind == 'm') and (dtype.name != 'timedelta64[ps]')`