Repository: modin-project/modin

Issue:
Assigning Index object to DataFrame.index which has an "index" column replaces column contents
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**:

Windows 10, Ubuntu 19.10

- **Modin version** (`modin.__version__`):

0.7.2+72.g6961c50.dirty

- **Python version**:

Python 3.7.7

- **Code we can use to reproduce**:

```python
import numpy as np
import modin.pandas as pd

df1 = pd.DataFrame([['ccc', 'xxx', 2], ['bbb', 'yyy', 5], ['aaa', 'zzz', 8]],
                   columns=['index', 'whatever', 'shield'])
print(df1)

df2 = df1.rename({'index': 'name'}, axis='columns')
print(df2)
```

<!--
You can obtain the Modin version with

python -c "import modin; print(modin.__version__)"
-->

### Describe the problem
<!-- Describe the problem clearly here. -->

If frame contains a column named `index` (which happens often after `reset_index` is used with `drop=False` (by default)), then some weird thing happens. In addition to renaming column label Modin inserts index contents into that column:
```
  index whatever  shield
0   ccc      xxx       2
1   bbb      yyy       5
2   aaa      zzz       8
   name whatever  shield
0     0      xxx       2
1     1      yyy       5
2     2      zzz       8
```
while pandas does only renaming:
```
  index whatever  shield
0   ccc      xxx       2
1   bbb      yyy       5
2   aaa      zzz       8
  name whatever  shield
0  ccc      xxx       2
1  bbb      yyy       5
2  aaa      zzz       8
```

### Source code / logs
<!-- Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem. -->



Discussion:
The problem is in this like in `dataframe.py`:
```python
        obj.index = new_index
```
which assigns an index to a new dataframe object. But when it goes inside of DataFrame.__setitem__ it assigns these values to a **column** named `index` instead of object **index**.
Here is a reproducer without `rename`
```python
import modin.pandas as pd

df = pd.DataFrame([['ccc', 2], ['bbb', 5], ['aaa', 8]],
                   columns=['index', 'shield'])
print(df)
idx = pd.Index(['111', '222', '333'])
df.index = idx
print(df)
```
This is patch to tests to test this behavior
```patch
diff --git a/modin/pandas/test/utils.py b/modin/pandas/test/utils.py
index 1c517a6..3986cad 100644
--- a/modin/pandas/test/utils.py
+++ b/modin/pandas/test/utils.py
@@ -62,6 +62,12 @@ test_data = {
         ]
         for i in range(NCOLS)
     },
+    "with_index_column": {
+        "index": [1, 2, 3, 4],
+        "col4": [4, 5, 6, 7],
+        "col1": [8.0, 9.4, 10.1, 11.3],
+        "col2": ["a", "b", "c", "d"],
+    },
     # "int_float_object_data": {
     #     "col3": [1, 2, 3, 4],
     #     "col4": [4, 5, 6, 7],
@@ -154,6 +160,7 @@ numeric_dfs = [
     "float_data",
     "sparse_nan_data",
     "dense_nan_data",
+    "with_index_column",
     "100x100",
 ]
```
> The problem is in this like in `dataframe.py`:
> 
> ```python
>         obj.index = new_index
> ```
> 
> which assigns an index to a new dataframe object. But when it goes inside of DataFrame.**setitem** it assigns these values to a **column** named `index` instead of object **index**.

Well quick fix for that is
```python
    obj._set_index(new_index)
```
now it's directly call the function that change indexes, without going to `dataframe.py: __setattr__` which and cause the problem and looks like this 
```python
def __setattr__(self, key, value):
        if key in ["_query_compiler"] or key in self.__dict__:
            pass
        elif key in self:
            self.__setitem__(key, value)
        elif isinstance(value, pandas.Series):
            ...
        object.__setattr__(self, key, value) 
```
so, if `key` named like some of `DataFrame` attribute (like `index`, `columns` etc) and the DataFrame itself contains a column with that name, then (going by that if-statement):
```python
    elif key in self:
        self.__setitem__(key, value)
```
we'll try to assign `value` to a column, instead of calling `object.__setattr__(self, key, value)` and changing the attribute itself

It means, that we probably should add a check that `key` isn't an attribute name (`self.__dict__` returns dict with only one field `'_query_compiler'` so this is not that type of check, maybe changing it to `dir(self)` is the solution?)
@dchigarev yes, but will that change make it possible to still write `df["index"] = some_new_value` or will all such assignments assign to `df.index` instead?
@gshimansky with changing `self.__dict__` to `dir(self)` related code works as expected:
```python
    import modin.pandas as pd

    df1 = pd.DataFrame([['ccc', 'xxx', 2], ['bbb', 'yyy', 5], ['aaa', 'zzz', 8]],
                       columns=['index', 'whatever', 'shield'])

    df1["index"] = [1,2,3]
    df2 = df1.rename({'index': 'name'}, axis='columns')
    
    print(df2)
```

Output:
```
   name whatever  shield
0     1      xxx       2
1     2      yyy       5
2     3      zzz       8
```

That's because assigning like `df["index"] = ...` directly call `dataframe.py: __setitem__` without hitting any checks in `dataframe.py: __setattr__`

Resolving patch:
diff --git a/modin/engines/base/frame/partition_manager.py b/modin/engines/base/frame/partition_manager.py
--- a/modin/engines/base/frame/partition_manager.py
+++ b/modin/engines/base/frame/partition_manager.py
@@ -111,7 +111,10 @@ def broadcast_apply(cls, axis, apply_func, left, right):
         Returns:
             A new `np.array` of partition objects.
         """
-        right_parts = np.squeeze(right)
+        if right.shape == (1, 1):
+            right_parts = right[0]
+        else:
+            right_parts = np.squeeze(right)
 
         [obj.drain_call_queue() for obj in right_parts]
         return np.array(
diff --git a/modin/pandas/dataframe.py b/modin/pandas/dataframe.py
--- a/modin/pandas/dataframe.py
+++ b/modin/pandas/dataframe.py
@@ -2048,7 +2048,7 @@ def __setattr__(self, key, value):
         # _query_compiler before we check if the key is in self
         if key in ["_query_compiler"] or key in self.__dict__:
             pass
-        elif key in self:
+        elif key in self and key not in dir(self):
             self.__setitem__(key, value)
         elif isinstance(value, pandas.Series):
             warnings.warn(
diff --git a/modin/pandas/series.py b/modin/pandas/series.py
--- a/modin/pandas/series.py
+++ b/modin/pandas/series.py
@@ -1258,7 +1258,7 @@ def is_monotonic(df):
     def is_monotonic_decreasing(self):
         # We cannot default to pandas without a named function to call.
         def is_monotonic_decreasing(df):
-            return df.is_monotonic
+            return df.is_monotonic_decreasing
 
         return self._default_to_pandas(is_monotonic_decreasing)
 
@@ -1266,7 +1266,7 @@ def is_monotonic_decreasing(df):
     def is_monotonic_increasing(self):
         # We cannot default to pandas without a named function to call.
         def is_monotonic_increasing(df):
-            return df.is_monotonic
+            return df.is_monotonic_increasing
 
         return self._default_to_pandas(is_monotonic_increasing)
 


Test patch:
diff --git a/modin/pandas/test/test_dataframe.py b/modin/pandas/test/test_dataframe.py
--- a/modin/pandas/test/test_dataframe.py
+++ b/modin/pandas/test/test_dataframe.py
@@ -2596,7 +2596,14 @@ def test_to_records(self, request, data):
 
         # Skips nan because only difference is nan instead of NaN
         if not name_contains(request.node.name, ["nan"]):
-            assert np.array_equal(modin_df.to_records(), pandas_df.to_records())
+            try:
+                pandas_result = pandas_df.to_records()
+            except Exception as e:
+                with pytest.raises(type(e)):
+                    modin_df.to_records()
+            else:
+                modin_result = modin_df.to_records()
+                assert np.array_equal(modin_result, pandas_result)
 
     def test_to_datetime(self):
         modin_df = pd.DataFrame({"year": [2015, 2016], "month": [2, 3], "day": [4, 5]})
diff --git a/modin/pandas/test/test_series.py b/modin/pandas/test/test_series.py
--- a/modin/pandas/test/test_series.py
+++ b/modin/pandas/test/test_series.py
@@ -2176,14 +2176,24 @@ def test_rtruediv(data):
 @pytest.mark.parametrize("data", test_data_values, ids=test_data_keys)
 def test_sample(data):
     modin_series, pandas_series = create_test_series(data)
-    df_equals(
-        modin_series.sample(frac=0.5, random_state=21019),
-        pandas_series.sample(frac=0.5, random_state=21019),
-    )
-    df_equals(
-        modin_series.sample(n=12, random_state=21019),
-        pandas_series.sample(n=12, random_state=21019),
-    )
+    try:
+        pandas_result = pandas_series.sample(frac=0.5, random_state=21019)
+    except Exception as e:
+        with pytest.raises(type(e)):
+            modin_series.sample(frac=0.5, random_state=21019)
+    else:
+        modin_result = modin_series.sample(frac=0.5, random_state=21019)
+        df_equals(pandas_result, modin_result)
+
+    try:
+        pandas_result = pandas_series.sample(n=12, random_state=21019)
+    except Exception as e:
+        with pytest.raises(type(e)):
+            modin_series.sample(n=12, random_state=21019)
+    else:
+        modin_result = modin_series.sample(n=12, random_state=21019)
+        df_equals(pandas_result, modin_result)
+
     with pytest.warns(UserWarning):
         df_equals(
             modin_series.sample(n=0, random_state=21019),
diff --git a/modin/pandas/test/utils.py b/modin/pandas/test/utils.py
--- a/modin/pandas/test/utils.py
+++ b/modin/pandas/test/utils.py
@@ -11,6 +11,7 @@
 # ANY KIND, either express or implied. See the License for the specific language
 # governing permissions and limitations under the License.
 
+import copy
 import numpy as np
 import pandas
 from pandas.util.testing import assert_almost_equal, assert_frame_equal
@@ -110,6 +111,14 @@
     },
 }
 
+# Create a dataframe based on integer dataframe but with one column called "index". Because of bug #1481 it cannot be
+# created in normal way and has to be copied from dataset that works.
+# TODO(gshimansky): when bug #1481 is fixed replace this dataframe initialization with ordinary one.
+test_data["with_index_column"] = copy.copy(test_data["int_data"])
+test_data["with_index_column"]["index"] = test_data["with_index_column"].pop(
+    "col{}".format(int(NCOLS / 2))
+)
+
 test_data_values = list(test_data.values())
 test_data_keys = list(test_data.keys())
 
@@ -155,6 +164,7 @@
     "float_data",
     "sparse_nan_data",
     "dense_nan_data",
+    "with_index_column",
     "100x100",
 ]