Repository: databricks/koalas

Issue:
reset_index is super slow
Hello, I am trying to load a dataset that does not have a particular index and I would like to know if my approach is correct.

I read that due to koalas index issues we should always specify the index, so I did it like this:

`df = df_tmp.to_koalas(index_col = ['A', 'B'])` since A and B are the columns that make a value unique.

However I need to drop the index because I will need these columns further and I tried to reset_index():

`df = df.sort_values('Date').reset_index()`

But this opperation takes 32 minutes which is completely infeasible. 

My dataset comes from a csv file

`df_tmp = spark.read.option("header", "true").csv(data_lake_path+"test.csv")`

And I added this line already:

`ks.set_option('compute.default_index_type', 'distributed-sequence')`

Any idea what I am doing wrong?


Discussion:
I think the root cause is that we don't support the assignment of index to the columns. I will make a quick fix.

```
>>> kdf = koalas.range(1000)
>>> kdf["col"] = kdf.index
```

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/.../koalas/databricks/koalas/frame.py", line 10075, in __setitem__
    kdf = self._assign({key: value})
  File "/.../koalas/databricks/koalas/frame.py", line 4301, in _assign
    "Column assignment doesn't support type " "{0}".format(type(v).__name__)
TypeError: Column assignment doesn't support type Index
```

Resolving patch:
diff --git a/databricks/koalas/frame.py b/databricks/koalas/frame.py
--- a/databricks/koalas/frame.py
+++ b/databricks/koalas/frame.py
@@ -4241,7 +4241,7 @@ def assign(self, **kwargs):
 
         Parameters
         ----------
-        **kwargs : dict of {str: callable or Series}
+        **kwargs : dict of {str: callable, Series or Index}
             The column names are keywords. If the values are
             callable, they are computed on the DataFrame and
             assigned to the new columns. The callable must not
@@ -4276,11 +4276,12 @@ def assign(self, **kwargs):
         create multiple columns within the same assign.
 
         >>> assigned = df.assign(temp_f=df['temp_c'] * 9 / 5 + 32,
-        ...                      temp_k=df['temp_c'] + 273.15)
-        >>> assigned[['temp_c', 'temp_f', 'temp_k']]
-                  temp_c  temp_f  temp_k
-        Portland    17.0    62.6  290.15
-        Berkeley    25.0    77.0  298.15
+        ...                      temp_k=df['temp_c'] + 273.15,
+        ...                      temp_idx=df.index)
+        >>> assigned[['temp_c', 'temp_f', 'temp_k', 'temp_idx']]
+                  temp_c  temp_f  temp_k  temp_idx
+        Portland    17.0    62.6  290.15  Portland
+        Berkeley    25.0    77.0  298.15  Berkeley
 
         Notes
         -----
@@ -4293,10 +4294,14 @@ def assign(self, **kwargs):
 
     def _assign(self, kwargs):
         assert isinstance(kwargs, dict)
-        from databricks.koalas.series import Series
+        from databricks.koalas.indexes import MultiIndex
+        from databricks.koalas.series import IndexOpsMixin
 
         for k, v in kwargs.items():
-            if not (isinstance(v, (Series, spark.Column)) or callable(v) or is_scalar(v)):
+            is_invalid_assignee = (
+                not (isinstance(v, (IndexOpsMixin, spark.Column)) or callable(v) or is_scalar(v))
+            ) or isinstance(v, MultiIndex)
+            if is_invalid_assignee:
                 raise TypeError(
                     "Column assignment doesn't support type " "{0}".format(type(v).__name__)
                 )
@@ -4306,7 +4311,7 @@ def _assign(self, kwargs):
         pairs = {
             (k if isinstance(k, tuple) else (k,)): (
                 v.spark.column
-                if isinstance(v, Series)
+                if isinstance(v, IndexOpsMixin) and not isinstance(v, MultiIndex)
                 else v
                 if isinstance(v, spark.Column)
                 else F.lit(v)


Test patch: