Repository: modin-project/modin

Issue:
Internal and external indices on axis 0 do not match when running rolling(window=...)
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: MacOS 11.6
- **Modin version** (`modin.__version__`): 0.11.0
- **Python version**: 3.8.11
- **Code we can use to reproduce**:

```python
import numpy as np
import modin.pandas as pd

l = np.random.rand(129)
modin_df = pd.DataFrame(l)
modin_df[0:33].rolling(window = 21).mean()
```

### Describe the problem
When trying to calculate a moving average by doing .rolling(window=...) the code fails with a message "_Exception: Internal Error. Please email bug_reports@modin.org with the traceback and command that caused this error.
Internal and external indices on axis 0 do not match._"

It happens with both Ray and Dask.

Interesting issue here is that, instead of generating 129 random numbers, if I only generate 128, it works. Similarly, if I slice the dataframe up to row 32 (instead of 33), it works.

### Source code / logs
```python
UserWarning: Distributing <class 'numpy.ndarray'> object. This may take some time.

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
~/miniconda3/lib/python3.8/site-packages/IPython/core/formatters.py in __call__(self, obj)
    700                 type_pprinters=self.type_printers,
    701                 deferred_pprinters=self.deferred_printers)
--> 702             printer.pretty(obj)
    703             printer.flush()
    704             return stream.getvalue()

~/miniconda3/lib/python3.8/site-packages/IPython/lib/pretty.py in pretty(self, obj)
    392                         if cls is not object \
    393                                 and callable(cls.__dict__.get('__repr__')):
--> 394                             return _repr_pprint(obj, self, cycle)
    395 
    396             return _default_pprint(obj, self, cycle)

~/miniconda3/lib/python3.8/site-packages/IPython/lib/pretty.py in _repr_pprint(obj, p, cycle)
    698     """A pprint that just redirects to the normal repr function."""
    699     # Find newlines and replace them with p.break_()
--> 700     output = repr(obj)
    701     lines = output.splitlines()
    702     with p.group():

~/miniconda3/lib/python3.8/site-packages/modin/pandas/dataframe.py in __repr__(self)
    208 
    209             num_cols += len(self.columns) - i
--> 210         result = repr(self._build_repr_df(num_rows, num_cols))
    211         if len(self.index) > num_rows or len(self.columns) > num_cols:
    212             # The split here is so that we don't repr pandas row lengths.

~/miniconda3/lib/python3.8/site-packages/modin/pandas/base.py in _build_repr_df(self, num_rows, num_cols)
    195         else:
    196             indexer = row_indexer
--> 197         return self.iloc[indexer]._query_compiler.to_pandas()
    198 
    199     def _update_inplace(self, new_query_compiler):

~/miniconda3/lib/python3.8/site-packages/modin/backends/pandas/query_compiler.py in to_pandas(self)
    253 
    254     def to_pandas(self):
--> 255         return self._modin_frame.to_pandas()
    256 
    257     @classmethod

~/miniconda3/lib/python3.8/site-packages/modin/engines/base/frame/data.py in to_pandas(self)
   2230         else:
   2231             for axis in [0, 1]:
-> 2232                 ErrorMessage.catch_bugs_and_request_email(
   2233                     not df.axes[axis].equals(self.axes[axis]),
   2234                     f"Internal and external indices on axis {axis} do not match.",

~/miniconda3/lib/python3.8/site-packages/modin/error_message.py in catch_bugs_and_request_email(cls, failure_condition, extra_log)
     58     def catch_bugs_and_request_email(cls, failure_condition, extra_log=""):
     59         if failure_condition:
---> 60             raise Exception(
     61                 "Internal Error. "
     62                 "Please email bug_reports@modin.org with the traceback and command that"

Exception: Internal Error. Please email bug_reports@modin.org with the traceback and command that caused this error.
Internal and external indices on axis 0 do not match.
```



Discussion:
Thanks @bbergua for the report! I appreciate the detail and the small reproducible code block with explanation of what you tested. It's very helpful, thanks!

@amyskov This appears to be an issue with Rolling, would you be able to take a look? As @bbergua said, the bug is interesting because the number of rows seems to directly cause the bug.
@devin-petersohn sure, i will take a look!

Resolving patch:
diff --git a/modin/core/dataframe/pandas/dataframe/dataframe.py b/modin/core/dataframe/pandas/dataframe/dataframe.py
--- a/modin/core/dataframe/pandas/dataframe/dataframe.py
+++ b/modin/core/dataframe/pandas/dataframe/dataframe.py
@@ -1313,7 +1313,7 @@ def fold(self, axis, func):
         The data shape is not changed (length and width of the table).
         """
         new_partitions = self._partition_mgr_cls.map_axis_partitions(
-            axis, self._partitions, func
+            axis, self._partitions, func, keep_partitioning=True
         )
         return self.__constructor__(
             new_partitions,


Test patch:
diff --git a/modin/pandas/test/test_rolling.py b/modin/pandas/test/test_rolling.py
--- a/modin/pandas/test/test_rolling.py
+++ b/modin/pandas/test/test_rolling.py
@@ -212,3 +212,14 @@ def test_api_indexer():
     pandas_rolled = pandas_df.rolling(window=indexer)
     modin_rolled = modin_df.rolling(window=indexer)
     df_equals(modin_rolled.sum(), pandas_rolled.sum())
+
+
+def test_issue_3512():
+    data = np.random.rand(129)
+    modin_df = pd.DataFrame(data)
+    pandas_df = pandas.DataFrame(data)
+
+    modin_ans = modin_df[0:33].rolling(window=21).mean()
+    pandas_ans = pandas_df[0:33].rolling(window=21).mean()
+
+    df_equals(modin_ans, pandas_ans)