Repository: modin-project/modin

Issue:
get_dummies doesn't update columns variable
<!--
General questions should be asked on the mailing list modin-dev@googlegroups.com.

Before submitting an issue, please fill out the following form.
-->

### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Ubuntu 18.04
- **Modin installed from (source or binary)**: pypi
- **Modin version**: 0.6.2
- **Python version**: 3.6.8

### Describe the problem
```get_dummies``` doesn't update the dataframe columns variable. EDIT: Actually, it appears that it doesn't update anything in the dataframe, since the ```.shape``` of the returned dataframe stays the same.

With pandas:
```py
import pandas as pd

df = pd.DataFrame({'categories': ['one', 'two', 'three', 'four']})
df = pd.get_dummies(df)
df
```

Gives me:
```py
   categories_four  categories_one  categories_three  categories_two
0                0               1                 0               0
1                0               0                 0               1
2                0               0                 1               0
3                1               0                 0               0
```

and ```df.columns``` outputs:
```
Index(['categories_four', 'categories_one', 'categories_three',
       'categories_two'],
      dtype='object')
```

With modin:

```py
import modin.pandas as pd

df = pd.DataFrame({'categories': ['one', 'two', 'three', 'four']})
df = pd.get_dummies(df)
df
```

Gives

```py
   categories_four  categories_one  categories_three  categories_two
0                0               1                 0               0
1                0               0                 0               1
2                0               0                 1               0
3                1               0                 0               0
```

But ```df.columns``` gives

```py
Index(['categories'], dtype='object')
```



Discussion:
Hi @Rdlenke, thanks for posting!

This is a bug with the metadata calculation after the computation has finished. We keep the metadata separate for performance reasons, so this will be a quick fix. The computation is still correct, but if you do `df.columns` that value will not be correct. As you point out `shape` is also an issue, but `shape` is using the columns to determine the size. 

Thanks again for posting! I will get this fixed today.

Resolving patch:
diff --git a/modin/backends/pandas/query_compiler.py b/modin/backends/pandas/query_compiler.py
--- a/modin/backends/pandas/query_compiler.py
+++ b/modin/backends/pandas/query_compiler.py
@@ -1233,13 +1233,15 @@ def get_dummies(self, columns, **kwargs):
         # efficient if we are mapping over all of the data to do it this way
         # than it would be to reuse the code for specific columns.
         if len(columns) == len(self.columns):
-            new_modin_frame = self._modin_frame._fold(
-                0, lambda df: pandas.get_dummies(df, **kwargs)
+            new_modin_frame = self._modin_frame._apply_full_axis(
+                0, lambda df: pandas.get_dummies(df, **kwargs), new_index=self.index
             )
             untouched_frame = None
         else:
-            new_modin_frame = self._modin_frame.mask(col_indices=columns)._fold(
-                0, lambda df: pandas.get_dummies(df, **kwargs)
+            new_modin_frame = self._modin_frame.mask(
+                col_indices=columns
+            )._apply_full_axis(
+                0, lambda df: pandas.get_dummies(df, **kwargs), new_index=self.index
             )
             untouched_frame = self.drop(columns=columns)
         # If we mapped over all the data we are done. If not, we need to


Test patch:
diff --git a/modin/pandas/test/test_reshape.py b/modin/pandas/test/test_reshape.py
--- a/modin/pandas/test/test_reshape.py
+++ b/modin/pandas/test/test_reshape.py
@@ -25,12 +25,16 @@ def test_get_dummies():
     modin_result = pd.get_dummies(modin_df, prefix=["col1", "col2"])
     pandas_result = pandas.get_dummies(pandas_df, prefix=["col1", "col2"])
     df_equals(modin_result, pandas_result)
+    assert modin_result._to_pandas().columns.equals(pandas_result.columns)
+    assert modin_result.shape == pandas_result.shape
 
     modin_result = pd.get_dummies(pd.DataFrame(pd.Series(list("abcdeabac"))))
     pandas_result = pandas.get_dummies(
         pandas.DataFrame(pandas.Series(list("abcdeabac")))
     )
     df_equals(modin_result, pandas_result)
+    assert modin_result._to_pandas().columns.equals(pandas_result.columns)
+    assert modin_result.shape == pandas_result.shape
 
     with pytest.raises(NotImplementedError):
         pd.get_dummies(modin_df, prefix=["col1", "col2"], sparse=True)