Repository: databricks/koalas

Issue:
Rename changes logical plan
Please let me know if I am doing something wrong here, but I would not expect that the `rename()` operation would redact previous operations.


```python
df = ks.DataFrame([["2020-05-01", 1], ["2020-05-02", 2]], columns=["date", "value"])
df["date"] = F.to_date(F.col("date"))

df.spark.print_schema()
```
everything looks good, 

```
root
 |-- date: date (nullable = true)
 |-- value: long (nullable = false)
```

```python
df_add = df.copy()
df_add["date"] = F.add_months(F.col("date"), 12)
df_add.spark.print_schema()
```
```
== Physical Plan ==
*(1) Project [__index_level_0__#945L, add_months(cast(date#946 as date), 12) AS date#961, value#947L]
+- Scan ExistingRDD[__index_level_0__#945L,date#946,value#947L]
      
| date | value |
| ---- | ----- |
|2021-05-01 | 1 | 
| 2021-05-02 | 2| 
```

Once I rename the `value` column, however, it seems that the `add_months` no longer applies. 
```python
df_add = df_add.rename(columns={"value": "new_value"})
df_add.spark.print_schema()
```

```
== Physical Plan ==
*(1) Project [__index_level_0__#993L, date#994, value#995L AS new_value#1030L]
+- Scan ExistingRDD[__index_level_0__#993L,date#994,value#995L]
                    
| date | value |
| ---- | ----- |
|2020-05-01 | 1 | 
| 2020-05-02 | 2| 
```

EDIT: 
I receive the same result when trying to use the new `kss.spark.transform()` accessor. 
```python
df = ks.DataFrame([["2020-05-01", 1], ["2020-05-02", 2]], columns=["date", "value"])
df["date"] = df["date"].spark.transform(lambda col: F.to_date(col))

df_add = df.copy()
df_add["date"] = df_add["date"].spark.transform(lambda col: F.add_months(col, 12))

df_add = df_add.rename(columns={"value": "new_value"})
```


Discussion:
From my understanding, there is no Koalas API for the same functionality at this time?
@sebastianvermaas Thanks for the report. Seems like it's a bug. I'll submit a PR to fix soon.

Resolving patch:
diff --git a/databricks/koalas/frame.py b/databricks/koalas/frame.py
--- a/databricks/koalas/frame.py
+++ b/databricks/koalas/frame.py
@@ -9269,15 +9269,9 @@ def gen_new_column_labels_entry(column_labels_entry):
 
             new_column_labels = list(map(gen_new_column_labels_entry, internal.column_labels))
 
-            if internal.column_labels_level == 1:
-                new_data_columns = [col[0] for col in new_column_labels]
-            else:
-                new_data_columns = [str(col) for col in new_column_labels]
             new_data_scols = [
-                scol_for(internal.spark_frame, old_col_name).alias(new_col_name)
-                for old_col_name, new_col_name in zip(
-                    internal.data_spark_column_names, new_data_columns
-                )
+                scol.alias(name_like_string(new_label))
+                for scol, new_label in zip(internal.data_spark_columns, new_column_labels)
             ]
             internal = internal.with_new_columns(new_data_scols, column_labels=new_column_labels)
         if inplace:


Test patch:
diff --git a/databricks/koalas/tests/test_dataframe.py b/databricks/koalas/tests/test_dataframe.py
--- a/databricks/koalas/tests/test_dataframe.py
+++ b/databricks/koalas/tests/test_dataframe.py
@@ -539,6 +539,14 @@ def mul10(x) -> int:
         self.assert_eq(kdf3.rename(index=str_lower, level=0), pdf3.rename(index=str_lower, level=0))
         self.assert_eq(kdf3.rename(index=str_lower, level=1), pdf3.rename(index=str_lower, level=1))
 
+        pdf4 = pdf2 + 1
+        kdf4 = kdf2 + 1
+        self.assert_eq(kdf4.rename(columns=str_lower), pdf4.rename(columns=str_lower))
+
+        pdf5 = pdf3 + 1
+        kdf5 = kdf3 + 1
+        self.assert_eq(kdf5.rename(index=str_lower), pdf5.rename(index=str_lower))
+
     def test_dot_in_column_name(self):
         self.assert_eq(
             ks.DataFrame(ks.range(1)._internal.spark_frame.selectExpr("1 as `a.b`"))["a.b"],