Repository: modin-project/modin

Issue:
astype category doesn't change the actual dtype to categorical

### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Linux Ubuntu 18.04
- **Modin installed from (source or binary)**: binary (pip)
- **Modin version**:  0.6.2
- **Python version**: 3.6.9
- **Exact command to reproduce**:
```import modin.pandas as pd
a = pd.Series(['A', 'A', 'B', 'B', 'A'])
a.dtype
Out[4]: dtype('O')
a = a.astype('category')
a.dtype
Out[6]: 'category'
```
Whereas in the original pandas:
```
a = pd_org.Series(['A', 'A', 'B', 'B', 'A'])
a = a.astype('category')
a.dtype
Out[10]: CategoricalDtype(categories=['A', 'B'], ordered=False)
```
### Describe the problem
'astype' function is only changing the name of the type without changing it to `CategoricalDtype` (as expected in Pandas). 
Using the following workaround, the expected behavior is achieved:
```
a = pd.Series(['A', 'A', 'B', 'B', 'A']) 
a = pd.Series(a.values, dtype='category') 
a.dtype 
Out[13]: CategoricalDtype(categories=['A', 'B'], ordered=False)
```

### Source code / logs
The above causing the following issues (when relying on `CategoricalDtype`):
```
a = pd.Series(['A', 'A', 'B', 'B', 'A'])
a = a.astype('category')
a_df = a.to_frame()
a_df.select_dtypes('category')
Traceback (most recent call last):
  File "/home/bar/.local/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3326, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-19-15c3ee7df6bb>", line 1, in <module>
    a_df.select_dtypes('category')
  File "/home/bar/.local/lib/python3.6/site-packages/modin/pandas/dataframe.py", line 1412, in select_dtypes
    is_dtype_instance_mapper, self.dtypes.iteritems()
  File "/home/bar/.local/lib/python3.6/site-packages/modin/pandas/dataframe.py", line 1409, in is_dtype_instance_mapper
    return column, functools.partial(issubclass, dtype.type)
AttributeError: 'str' object has no attribute 'type'
```



Discussion:
Hi @bareini, thanks for the report! It looks like our optimization is oversimplifying the `dtype` in the case of `Categorical`.

This should be relatively easy to fix, but in the meantime you can reset the dtypes with:

```python
a._query_compiler._modin_frame._dtypes = None
a.dtype                                                                                                                                                                       
Out[9]: CategoricalDtype(categories=['A', 'B'], ordered=False)
```

This will force Modin to recompute the dtypes as if we could not infer them. Thanks again for the report!

Resolving patch:
diff --git a/modin/engines/base/frame/data.py b/modin/engines/base/frame/data.py
--- a/modin/engines/base/frame/data.py
+++ b/modin/engines/base/frame/data.py
@@ -450,12 +450,15 @@ def astype(self, col_dtypes):
                     new_dtype = np.dtype(dtype)
                 except TypeError:
                     new_dtype = dtype
+
                 if dtype != np.int32 and new_dtype == np.int32:
-                    new_dtype = np.dtype("int64")
+                    new_dtypes[column] = np.dtype("int64")
                 elif dtype != np.float32 and new_dtype == np.float32:
-                    new_dtype = np.dtype("float64")
-                new_dtypes[column] = new_dtype
-        # Update partitions for each dtype that is updated
+                    new_dtypes[column] = np.dtype("float64")
+                # We cannot infer without computing the dtype if
+                elif isinstance(new_dtype, str) and new_dtype == "category":
+                    new_dtypes = None
+                    break
 
         def astype_builder(df):
             return df.astype({k: v for k, v in col_dtypes.items() if k in df})


Test patch:
diff --git a/modin/pandas/test/test_dataframe.py b/modin/pandas/test/test_dataframe.py
--- a/modin/pandas/test/test_dataframe.py
+++ b/modin/pandas/test/test_dataframe.py
@@ -1222,6 +1222,24 @@ def test_astype(self):
         with pytest.raises(KeyError):
             modin_df.astype({"not_exists": np.uint8})
 
+    def test_astype_category(self):
+        modin_df = pd.DataFrame(
+            {"col1": ["A", "A", "B", "B", "A"], "col2": [1, 2, 3, 4, 5]}
+        )
+        pandas_df = pandas.DataFrame(
+            {"col1": ["A", "A", "B", "B", "A"], "col2": [1, 2, 3, 4, 5]}
+        )
+
+        modin_result = modin_df.astype({"col1": "category"})
+        pandas_result = pandas_df.astype({"col1": "category"})
+        df_equals(modin_result, pandas_result)
+        assert modin_result.dtypes.equals(pandas_result.dtypes)
+
+        modin_result = modin_df.astype("category")
+        pandas_result = pandas_df.astype("category")
+        df_equals(modin_result, pandas_result)
+        assert modin_result.dtypes.equals(pandas_result.dtypes)
+
     def test_at_time(self):
         i = pd.date_range("2018-04-09", periods=4, freq="12H")
         ts = pd.DataFrame({"A": [1, 2, 3, 4]}, index=i)
diff --git a/modin/pandas/test/test_series.py b/modin/pandas/test/test_series.py
--- a/modin/pandas/test/test_series.py
+++ b/modin/pandas/test/test_series.py
@@ -794,6 +794,21 @@ def test_astype(data):
         df_equals(modin_series.astype(np.float64), pandas_result)
 
 
+def test_astype_categorical():
+    modin_df = pd.Series(["A", "A", "B", "B", "A"])
+    pandas_df = pandas.Series(["A", "A", "B", "B", "A"])
+
+    modin_result = modin_df.astype("category")
+    pandas_result = pandas_df.astype("category")
+    df_equals(modin_result, pandas_result)
+    assert modin_result.dtype == pandas_result.dtype
+
+    modin_df = pd.Series([1, 1, 2, 1, 2, 2, 3, 1, 2, 1, 2])
+    pandas_df = pandas.Series([1, 1, 2, 1, 2, 2, 3, 1, 2, 1, 2])
+    df_equals(modin_result, pandas_result)
+    assert modin_result.dtype == pandas_result.dtype
+
+
 @pytest.mark.parametrize("data", test_data_values, ids=test_data_keys)
 def test_at(data):
     modin_series, pandas_series = create_test_series(data)