Repository: modin-project/modin

Issue:
Incorrect dtype when DataFrame is distributed
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**:

Windows 10

- **Modin version** (`modin.__version__`):

0.7.2+85.gb9b791f.dirty

- **Python version**:

Python 3.7.7

- **Code we can use to reproduce**:

```python
if __name__ == "__main__":
    import numpy as np
    import modin.pandas as pd

    s1 = pd.DataFrame(np.arange(1, 10000, dtype=np.float32), columns=["s1"])
    print(s1.dtypes['s1'])
```

<!--
You can obtain the Modin version with

python -c "import modin; print(modin.__version__)"
-->

### Describe the problem
<!-- Describe the problem clearly here. -->

On Modin resulting dtype is `float64` while on Pandas it is `float32`.

### Source code / logs
<!-- Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem. -->

Incorrect dtype when DataFrame is distributed
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**:

Windows 10

- **Modin version** (`modin.__version__`):

0.7.2+85.gb9b791f.dirty

- **Python version**:

Python 3.7.7

- **Code we can use to reproduce**:

```python
if __name__ == "__main__":
    import numpy as np
    import modin.pandas as pd

    s1 = pd.DataFrame(np.arange(1, 10000, dtype=np.float32), columns=["s1"])
    print(s1.dtypes['s1'])
```

<!--
You can obtain the Modin version with

python -c "import modin; print(modin.__version__)"
-->

### Describe the problem
<!-- Describe the problem clearly here. -->

On Modin resulting dtype is `float64` while on Pandas it is `float32`.

### Source code / logs
<!-- Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem. -->



Discussion:
Interesting, it appears that it does not happen when data is loaded with `dtype` kwarg.
There is a bug in DataFrame constructor:
https://github.com/modin-project/modin/blob/4c772ab5a97276709ae9e035ead5198caaa7f2bf/modin/pandas/dataframe.py#L113-L116
That list comprehension expression will return a python `list` with no `dtype`, with elements of `data`.
Let's say our `data` is `np.ndarray` with `dtype=float32`, after above expression, we will get a list of array elements, and then, when we would like to create `pandas.DataFrame`:
https://github.com/modin-project/modin/blob/4c772ab5a97276709ae9e035ead5198caaa7f2bf/modin/pandas/dataframe.py#L124-L127
Pandas will automatically detect the `dtype` as `float64`, because it has no information about original data type.
Interesting, it appears that it does not happen when data is loaded with `dtype` kwarg.
There is a bug in DataFrame constructor:
https://github.com/modin-project/modin/blob/4c772ab5a97276709ae9e035ead5198caaa7f2bf/modin/pandas/dataframe.py#L113-L116
That list comprehension expression will return a python `list` with no `dtype`, with elements of `data`.
Let's say our `data` is `np.ndarray` with `dtype=float32`, after above expression, we will get a list of array elements, and then, when we would like to create `pandas.DataFrame`:
https://github.com/modin-project/modin/blob/4c772ab5a97276709ae9e035ead5198caaa7f2bf/modin/pandas/dataframe.py#L124-L127
Pandas will automatically detect the `dtype` as `float64`, because it has no information about original data type.

Resolving patch:
diff --git a/modin/pandas/dataframe.py b/modin/pandas/dataframe.py
--- a/modin/pandas/dataframe.py
+++ b/modin/pandas/dataframe.py
@@ -111,9 +111,17 @@ def __init__(
                 "Distributing {} object. This may take some time.".format(type(data))
             )
             if is_list_like(data) and not is_dict_like(data):
-                data = [
+                old_dtype = getattr(data, "dtype", None)
+                values = [
                     obj._to_pandas() if isinstance(obj, Series) else obj for obj in data
                 ]
+                if isinstance(data, np.ndarray):
+                    data = np.array(values, dtype=old_dtype)
+                else:
+                    try:
+                        data = type(data)(values, dtype=old_dtype)
+                    except TypeError:
+                        data = values
             elif is_dict_like(data) and not isinstance(
                 data, (pandas.Series, Series, pandas.DataFrame, DataFrame)
             ):


Test patch:
diff --git a/modin/pandas/test/test_dataframe.py b/modin/pandas/test/test_dataframe.py
--- a/modin/pandas/test/test_dataframe.py
+++ b/modin/pandas/test/test_dataframe.py
@@ -107,6 +107,10 @@ def eval_insert(modin_df, pandas_df, **kwargs):
     )
 
 
+def create_test_dfs(*args, **kwargs):
+    return pd.DataFrame(*args, **kwargs), pandas.DataFrame(*args, **kwargs)
+
+
 class TestDataFrameBinary:
     def inter_df_math_helper(self, modin_df, pandas_df, op):
         # Test dataframe to datframe
@@ -5418,6 +5422,22 @@ def test_constructor(self, data):
         modin_df = pd.DataFrame({k: pd.Series(v) for k, v in data.items()})
         df_equals(pandas_df, modin_df)
 
+    @pytest.mark.parametrize(
+        "data",
+        [
+            np.arange(1, 10000, dtype=np.float32),
+            [
+                pd.Series([1, 2, 3], dtype="int32"),
+                pandas.Series([4, 5, 6], dtype="int64"),
+                np.array([7, 8, 9], dtype=np.float32),
+            ],
+            pandas.Categorical([1, 2, 3, 4, 5]),
+        ],
+    )
+    def test_constructor_dtypes(self, data):
+        md_df, pd_df = create_test_dfs(data)
+        df_equals(md_df, pd_df)
+
     def test_constructor_columns_and_index(self):
         modin_df = pd.DataFrame(
             [[1, 1, 10], [2, 4, 20], [3, 7, 30]],