Repository: modin-project/modin

Issue:
BUG: Creating a DF from a large numpy array takes a very long time
### Modin version checks

- [X] I have checked that this issue has not already been reported.

- [X] I have confirmed this bug exists on the latest released version of Modin.

- [X] I have confirmed this bug exists on the main branch of Modin. (In order to do this you can follow [this guide](https://modin.readthedocs.io/en/stable/getting_started/installation.html#installing-from-the-github-master-branch).)


### Reproducible Example

```python
import numpy as np
arr = np.ones((100_000_000, 3))
import modin.pandas as pd
df = pd.DataFrame(arr)
```


### Issue Description

Our `__init__` for DataFrame's is very slow on big NumPy arrays. I've narrowed it down to a couple of chokepoints:
1. [This list comprehension](https://github.com/modin-project/modin/blob/a4504a0e298f6036dc9925bdee9a6898f4d64140/modin/pandas/dataframe.py#L177):
```python
                values = [
                    obj._to_pandas() if isinstance(obj, Series) else obj for obj in data
                ]
```
This list comprehension ensures that we don't have any distributed objects in our list before we try and make a pandas DF; however, if the object is a numpy array, it shouldn't be possible for us to have modin distributed objects internally:
```python

In [1]: import modin.pandas as pd
import UserWarning: The pandas version installed (1.5.2) does not match the supported pandas version in Modin (1.5.3). This may cause undesired side effects!
n
In [2]: import numpy as np

In [3]: arr = np.array([pd.Series([1]), pd.Series([2])])
UserWarning: Ray execution environment not yet initialized. Initializing...
To remove this warning, run the following python code before doing dataframe operations:

    import ray
    ray.init(runtime_env={'env_vars': {'__MODIN_AUTOIMPORT_PANDAS__': '1'}})

UserWarning: Distributing <class 'list'> object. This may take some time.
UserWarning: Distributing <class 'list'> object. This may take some time.

In [4]: arr
Out[4]:
array([[1],
       [2]])

In [5]: arr.dtype
Out[5]: dtype('int64')

In [6]: arr[0]
Out[6]: array([1])

In [7]: type(arr[0])
Out[7]: numpy.ndarray

In [8]: arr[0] = pd.Series([2])
UserWarning: Distributing <class 'list'> object. This may take some time.

In [9]: arr
Out[9]:
array([[2],
       [2]])

In [10]: type(arr[0])
Out[10]: numpy.ndarray
```
so we can skip this list comprehension if we know our input is an np array. @labanyamukhopadhyay is running tests with an np array that has 100 million rows, which means that this list comprehension takes a very long time, and effectively does nothing.
2. [This line](https://github.com/modin-project/modin/blob/a4504a0e298f6036dc9925bdee9a6898f4d64140/modin/pandas/dataframe.py#L181) that makes the result of a list comprehension back into a numpy array - since the list comprehension doesn't actually change numpy arrays, it's unnecessary, and on really big lists, can take some time.

In addition, we convert our object to pandas and use our QC's from_pandas to get a modin df, but this can take quite some time - I propose a longer term solution of adding a `from_numpy` on our QC's since 2D numpy objects can be distributed fairly similarly to pandas DF's.

I think we should at least rewrite the list comp + np.array lines above to include a fast path for numpy arrays.


### Expected Behavior

Labanya found a 100M row took about 4 minutes to distribute. I found that the list comprehension part took at least 30 seconds to compute. We could shave off about a minute by adding the fast path.

### Error Logs

<details>

```python-traceback

Replace this line with the error backtrace (if applicable).

```

</details>


### Installed Versions

<details>

Replace this line with the output of pd.show_versions()

</details>



Discussion:
With this fix: https://github.com/RehanSD/modin/tree/rehan/sklean_benching

The time to distribute the numpy array went from: 12.0 minutes and 31.1 seconds to: 51.5 seconds on @labanyamukhopadhyay's benchmarking data.
> With this fix: https://github.com/RehanSD/modin/tree/rehan/sklean_benching
> 
> The time to distribute the numpy array went from: 12.0 minutes and 31.1 seconds to: 51.5 seconds on @labanyamukhopadhyay's benchmarking data.

@RehanSD Your find is impressive. Do you plan to make a pull request with these changes in the near future?

Resolving patch:
diff --git a/modin/pandas/dataframe.py b/modin/pandas/dataframe.py
--- a/modin/pandas/dataframe.py
+++ b/modin/pandas/dataframe.py
@@ -173,18 +173,19 @@ def __init__(
             )
             if isinstance(data, pandas.Index):
                 pass
-            elif is_list_like(data) and not is_dict_like(data):
+            elif (
+                is_list_like(data)
+                and not is_dict_like(data)
+                and not isinstance(data, np.ndarray)
+            ):
                 old_dtype = getattr(data, "dtype", None)
                 values = [
                     obj._to_pandas() if isinstance(obj, Series) else obj for obj in data
                 ]
-                if isinstance(data, np.ndarray):
-                    data = np.array(values, dtype=old_dtype)
-                else:
-                    try:
-                        data = type(data)(values, dtype=old_dtype)
-                    except TypeError:
-                        data = values
+                try:
+                    data = type(data)(values, dtype=old_dtype)
+                except TypeError:
+                    data = values
             elif is_dict_like(data) and not isinstance(
                 data, (pandas.Series, Series, pandas.DataFrame, DataFrame)
             ):


Test patch: