Repository: modin-project/modin
Pull request: FIX-#3896: Change Series.values to default to to_numpy().
Signed-off-by: jeffreykennethli <jkli@ponder.io>

<!--
Thank you for your contribution!
Please review the contributing docs: https://modin.readthedocs.io/en/latest/CONTRIBUTING.html
if you have questions about contributing.
-->

## What do these changes do?

Modify Series.values to default to Series.to_numpy(), which conveniently handles the empty dataframe/series case from #3896.

In [pandas/base.py](https://github.com/modin-project/modin/blob/master/modin/pandas/base.py#L3075-L3087), notice that if an attr is callable and self.empty, we default to pandas like in [Series.to_numpy()](https://github.com/modin-project/modin/blob/master/modin/pandas/series.py#L1863-L1877) which would fix our empty Series case from #3896. 

Because to_numpy() and .values are supposed to have identical behavior (see [pandas.Series.values](https://pandas.pydata.org/docs/reference/api/pandas.Series.values.html) and [this PR](https://github.com/pandas-dev/pandas/pull/23623)), we can safely make Series.values call Series.to_numpy().

- [ ] commit message follows format outlined [here](https://modin.readthedocs.io/en/latest/contributing.html)
- [ ] passes `flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py`
- [ ] passes `black --check modin/ asv_bench/benchmarks scripts/doc_checker.py`
- [ ] signed commit with `git commit -s` <!-- you can amend your commit with a signature via `git commit -amend -s` -->
- [ ] Resolves #3896 <!-- issue must be created for each patch -->
- [ ] tests added and passing
- [ ] module layout described at `docs/developer/architecture.rst` is up-to-date <!-- if you have added, renamed or removed files or directories please update the documentation accordingly -->


Problem statement:
IndexError when calling modin.pandas.DataFrame.values.all()
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Microsoft Windows 10 Entreprise 10.0.18363 N/A build 18363
- **Modin version** (`modin.__version__`): 0.12.1
- **Python version**: Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54)
- **Code we can use to reproduce**:

```python
from pyreadline import Readline
readline = Readline()
import numpy as np
import ray
import modin.pandas as pd

ray.init()

df_movment_subset = pd.DataFrame([], columns=['Date transaction', 'Reference','Statut','MSISDN','Num Correspondant', 'Type', 'Movments.final.movmentAmount'])

if df_movment_subset['Type'].values.all() == 'Depot':
    print("Depot")
elif df_movment_subset['Type'].values.all() == 'Retrait':
    print("Retrait")
else:
    print("None")
```

### Describe the problem

An IndexError: tuple index out of range is thrown
The same code with native pandas library will return 'None'

### Source code / logs

```
UserWarning: `DataFrame.__getitem__` for empty DataFrame defaulting to pandas implementation.
To request implementation, send an email to feature_requests@modin.org.
UserWarning: Distributing <class 'pandas.core.series.Series'> object. This may take some time.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\TNTZ2660\AppData\Local\Programs\Python\Python38\lib\site-packages\modin\pandas\base.py", line 3076, in __getattribute__
    attr = super().__getattribute__(item)
  File "C:\Users\TNTZ2660\AppData\Local\Programs\Python\Python38\lib\site-packages\modin\pandas\series.py", line 546, in values
    return super(Series, self).to_numpy().flatten()
  File "C:\Users\TNTZ2660\AppData\Local\Programs\Python\Python38\lib\site-packages\modin\pandas\base.py", line 2694, in to_numpy
    return self._query_compiler.to_numpy(
  File "C:\Users\TNTZ2660\AppData\Local\Programs\Python\Python38\lib\site-packages\modin\core\storage_formats\pandas\query_compiler.py", line 338, in to_numpy
    arr = self._modin_frame.to_numpy(**kwargs)
  File "C:\Users\TNTZ2660\AppData\Local\Programs\Python\Python38\lib\site-packages\modin\core\dataframe\pandas\dataframe\dataframe.py", line 2282, in to_numpy
    return self._partition_mgr_cls.to_numpy(self._partitions, **kwargs)
  File "C:\Users\TNTZ2660\AppData\Local\Programs\Python\Python38\lib\site-packages\modin\core\execution\ray\generic\partitioning\partition_manager.py", line 51, in to_numpy
    n = partitions.shape[1]
IndexError: tuple index out of range

```

Hints from issue discussion:
Thanks for bringing this up! I was able to reproduce this on my Windows machine am looking into the cause.
Can also confirm that this fails on OSX too - the pyreadline module isn't used and removing it from the code block doesn't affect the result on OSX.

Commit message: Change Series.values to default to to_numpy() and add unit test

Signed-off-by: jeffreykennethli <jkli@ponder.io>

Patch to review:
diff --git a/modin/pandas/series.py b/modin/pandas/series.py
index b970e0718cc..f6a868d8281 100644
--- a/modin/pandas/series.py
+++ b/modin/pandas/series.py
@@ -543,7 +543,7 @@ def values(self):  # noqa: RT01, D200
         """
         Return Series as ndarray or ndarray-like depending on the dtype.
         """
-        return super(Series, self).to_numpy().flatten()
+        return self.to_numpy()
 
     def add(self, other, level=None, fill_value=None, axis=0):  # noqa: PR01, RT01, D200
         """
diff --git a/modin/pandas/test/test_series.py b/modin/pandas/test/test_series.py
index f63aab63f47..7c6476aa239 100644
--- a/modin/pandas/test/test_series.py
+++ b/modin/pandas/test/test_series.py
@@ -3204,6 +3204,21 @@ def test_to_period():
 )
 def test_to_numpy(data):
     modin_series, pandas_series = create_test_series(data)
+    assert_array_equal(modin_series.to_numpy(), pandas_series.to_numpy())
+
+
+@pytest.mark.parametrize(
+    "data",
+    test_data_values + test_data_large_categorical_series_values,
+    ids=test_data_keys + test_data_large_categorical_series_keys,
+)
+def test_series_values(data):
+    modin_series, pandas_series = create_test_series(data)
+    assert_array_equal(modin_series.values, pandas_series.values)
+
+
+def test_series_empty_values():
+    modin_series = pandas_series = pd.Series([])
     assert_array_equal(modin_series.values, pandas_series.values)
 
 


Code review comments:
diff_hunk: @@ -3204,6 +3204,21 @@ def test_to_period():
 )
 def test_to_numpy(data):
     modin_series, pandas_series = create_test_series(data)
+    assert_array_equal(modin_series.to_numpy(), pandas_series.to_numpy())
+
+
+@pytest.mark.parametrize(
+    "data",
+    test_data_values + test_data_large_categorical_series_values,
+    ids=test_data_keys + test_data_large_categorical_series_keys,
+)
+def test_series_values(data):
+    modin_series, pandas_series = create_test_series(data)
+    assert_array_equal(modin_series.values, pandas_series.values)
+
+
+def test_series_empty_values():
+    modin_series = pandas_series = pd.Series([])
path: modin/pandas/test/test_series.py
text: @user1:
won't this make the `pandas_series` equal to `modin.pandas.Series`? I don't think this test actually tests that modin handles an empty series the same as pandas.

@author:
done! ty for catching this.