Repository: modin-project/modin

Issue:
astype with "category" parameter throws ValueError
<!--
General questions should be asked on the mailing list modin-dev@googlegroups.com.

Before submitting an issue, please fill out the following form.
-->

### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: OSX
- **Modin installed from (source or binary)**: source
- **Modin version**: source
- **Python version**: 3.6
- **Exact command to reproduce**: `df.astype("categorical")`

<!--
You can obtain the Modin version with

python -c "import modin; print(modin.__version__)"
-->

### Describe the problem
Running `astype` with `"categorical"` throws an error because it is not a numpy dtype and the querycompiler is trying to cast it to a numpy dtype.
<!-- Describe the problem clearly here. -->

### Source code / logs
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/Downloads/kaggle3/kaggle3.py in <module>
    173 # conc_data_col
    174 data.dtypes
--> 175 data['Type 1'] = data['Type 1'].astype('category')
    176 data['Speed'] = data['Speed'].astype('float')
    177 data.dtypes

~/software_builds/modin/modin/pandas/base.py in astype(self, dtype, copy, errors, **kwargs)
    551                 col_dtypes[column] = dtype
    552 
--> 553         new_query_compiler = self._query_compiler.astype(col_dtypes, **kwargs)
    554         return self._create_or_update_from_compiler(new_query_compiler, not copy)
    555 

~/software_builds/modin/modin/backends/pandas/query_compiler.py in astype(self, col_dtypes, **kwargs)
   1111                     dtype_indices[dtype] = [numeric_indices[i]]
   1112                 # Update the new dtype series to the proper pandas dtype
-> 1113                 new_dtype = np.dtype(dtype)
   1114                 if dtype != np.int32 and new_dtype == np.int32:
   1115                     new_dtype = np.dtype("int64")

TypeError: data type "category" not understood
```
<!-- Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem. -->



Discussion:

Resolving patch:
diff --git a/modin/backends/pandas/query_compiler.py b/modin/backends/pandas/query_compiler.py
--- a/modin/backends/pandas/query_compiler.py
+++ b/modin/backends/pandas/query_compiler.py
@@ -1103,14 +1103,20 @@ def astype(self, col_dtypes, **kwargs):
         new_dtypes = self.dtypes.copy()
         for i, column in enumerate(columns):
             dtype = col_dtypes[column]
-            if dtype != self.dtypes[column]:
+            if (
+                not isinstance(dtype, type(self.dtypes[column]))
+                or dtype != self.dtypes[column]
+            ):
                 # Only add dtype only if different
                 if dtype in dtype_indices.keys():
                     dtype_indices[dtype].append(numeric_indices[i])
                 else:
                     dtype_indices[dtype] = [numeric_indices[i]]
                 # Update the new dtype series to the proper pandas dtype
-                new_dtype = np.dtype(dtype)
+                try:
+                    new_dtype = np.dtype(dtype)
+                except TypeError:
+                    new_dtype = dtype
                 if dtype != np.int32 and new_dtype == np.int32:
                     new_dtype = np.dtype("int64")
                 elif dtype != np.float32 and new_dtype == np.float32:


Test patch:
diff --git a/modin/pandas/test/test_dataframe.py b/modin/pandas/test/test_dataframe.py
--- a/modin/pandas/test/test_dataframe.py
+++ b/modin/pandas/test/test_dataframe.py
@@ -1063,6 +1063,10 @@ def test_astype(self):
         expected_df_casted = expected_df.astype(str)
         df_equals(modin_df_casted, expected_df_casted)
 
+        modin_df_casted = modin_df.astype("category")
+        expected_df_casted = expected_df.astype("category")
+        df_equals(modin_df_casted, expected_df_casted)
+
         dtype_dict = {"A": np.int32, "B": np.int64, "C": str}
         modin_df_casted = modin_df.astype(dtype_dict)
         expected_df_casted = expected_df.astype(dtype_dict)