Repository: databricks/koalas

Issue:
Support slice as `col_sel` for `loc` indexer.
We should support slice as `col_sel` for `loc` indexer.

```py
>>> pdf
   x     y     z
   a  b  c  d  e
0  1  2  3  4  5
>>> pdf.loc[:, 'y':]
   y     z
   c  d  e
0  3  4  5
>>> pdf.loc[:, :'y']
   x     y
   a  b  c  d
0  1  2  3  4
>>> pdf.loc[:, ('x','b'):]
   x  y     z
   b  c  d  e
0  2  3  4  5
>>> pdf.loc[:, :('y','c')]
   x     y
   a  b  c
0  1  2  3
>>> pdf.loc[:, ('x','b'):('y','c')]
   x  y
   b  c
0  2  3
>>> pdf.loc[:, 'x':('y','c')]
   x     y
   a  b  c
0  1  2  3
>>> pdf.loc[:, ('x','b'):'y']
   x  y
   b  c  d
0  2  3  4
```
[WIP] Support slice as `col_sel` for `loc` indexer
Resolves #1176 

```python
>>> kdf
   x     y     z
   a  b  c  d  e
0  1  2  3  4  5

>>> kdf.loc[:, 'y':]
   y     z
   c  d  e
0  3  4  5

>>> kdf.loc[:, :'y']
   x     y
   a  b  c  d
0  1  2  3  4

>>> kdf.loc[:, ('x', 'b'):]
   x  y     z
   b  c  d  e
0  2  3  4  5

>>> kdf.loc[:, :('y', 'c')]
   x     y
   a  b  c
0  1  2  3

>>> kdf.loc[:, ('x', 'b'):('y', 'c')]
   x  y
   b  c
0  2  3

>>> kdf.loc[:, 'x':('y', 'c')]
   x     y
   a  b  c
0  1  2  3

>>> kdf.loc[:, ('x', 'b'):'y']
   x  y
   b  c  d
0  2  3  4
```


Discussion:
@itholic are you interested in this?
yeah i'm happy to take it. and i'll take #1175 also
While i'm here i'm going to consider below things and add more codes for handling missing exception cases with proper way.

### the things we need to solve.

pandas doesn't support slice as `col_sel` for `loc` with `MultiIndex` when the first index is not ~~monotonically increasing.~~ sorted by ascending order.

for example

```python
>>> pdf
   z  y  x
   c  b  a
0  1  2  3

>>> pdf.loc[:, 'z':]
Traceback (most recent call last):
...
pandas.errors.UnsortedIndexError: 'Key length (1) was greater than MultiIndex lexsort depth (0)'
```

and doesn't support below way also

```python
>>> pdf.loc[:, ('z', 'c'):]
Traceback (most recent call last):
...
pandas.errors.UnsortedIndexError: 'Key length (2) was greater than MultiIndex lexsort depth (0)'
```

so i think we need #1183 for checking if the order of column is monotonic_increasing.
(of course, there are other ways to work around, but would we better to use API?)
# [Codecov](https://codecov.io/gh/databricks/koalas/pull/1188?src=pr&el=h1) Report
> Merging [#1188](https://codecov.io/gh/databricks/koalas/pull/1188?src=pr&el=desc) into [master](https://codecov.io/gh/databricks/koalas/commit/15a6406da7f92e0464639031eb818c40e874b8ad?src=pr&el=desc) will **increase** coverage by `0.04%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/databricks/koalas/pull/1188/graphs/tree.svg?width=650&token=EtnrpE8dVx&height=150&src=pr)](https://codecov.io/gh/databricks/koalas/pull/1188?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #1188      +/-   ##
==========================================
+ Coverage   95.24%   95.29%   +0.04%     
==========================================
  Files          35       35              
  Lines        7105     7180      +75     
==========================================
+ Hits         6767     6842      +75     
  Misses        338      338
```


| [Impacted Files](https://codecov.io/gh/databricks/koalas/pull/1188?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [databricks/koalas/indexing.py](https://codecov.io/gh/databricks/koalas/pull/1188/diff?src=pr&el=tree#diff-ZGF0YWJyaWNrcy9rb2FsYXMvaW5kZXhpbmcucHk=) | `96.55% <100%> (+0.58%)` | :arrow_up: |
| [databricks/koalas/indexes.py](https://codecov.io/gh/databricks/koalas/pull/1188/diff?src=pr&el=tree#diff-ZGF0YWJyaWNrcy9rb2FsYXMvaW5kZXhlcy5weQ==) | `96.41% <0%> (-0.48%)` | :arrow_down: |
| [databricks/koalas/frame.py](https://codecov.io/gh/databricks/koalas/pull/1188/diff?src=pr&el=tree#diff-ZGF0YWJyaWNrcy9rb2FsYXMvZnJhbWUucHk=) | `97% <0%> (-0.01%)` | :arrow_down: |
| [databricks/koalas/series.py](https://codecov.io/gh/databricks/koalas/pull/1188/diff?src=pr&el=tree#diff-ZGF0YWJyaWNrcy9rb2FsYXMvc2VyaWVzLnB5) | `96.46% <0%> (ø)` | :arrow_up: |
| [databricks/koalas/missing/groupby.py](https://codecov.io/gh/databricks/koalas/pull/1188/diff?src=pr&el=tree#diff-ZGF0YWJyaWNrcy9rb2FsYXMvbWlzc2luZy9ncm91cGJ5LnB5) | `100% <0%> (ø)` | :arrow_up: |
| [databricks/koalas/missing/indexes.py](https://codecov.io/gh/databricks/koalas/pull/1188/diff?src=pr&el=tree#diff-ZGF0YWJyaWNrcy9rb2FsYXMvbWlzc2luZy9pbmRleGVzLnB5) | `100% <0%> (ø)` | :arrow_up: |
| [databricks/koalas/groupby.py](https://codecov.io/gh/databricks/koalas/pull/1188/diff?src=pr&el=tree#diff-ZGF0YWJyaWNrcy9rb2FsYXMvZ3JvdXBieS5weQ==) | `91.9% <0%> (+0.3%)` | :arrow_up: |
| [databricks/koalas/internal.py](https://codecov.io/gh/databricks/koalas/pull/1188/diff?src=pr&el=tree#diff-ZGF0YWJyaWNrcy9rb2FsYXMvaW50ZXJuYWwucHk=) | `96.55% <0%> (+0.8%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/databricks/koalas/pull/1188?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/databricks/koalas/pull/1188?src=pr&el=footer). Last update [15a6406...6201ab2](https://codecov.io/gh/databricks/koalas/pull/1188?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

### Softagram Impact Report for pull/1188 (head commit: c51346233353)



### :warning: Copy paste found
:information_source: test_indexing.py: *Copy paste fragment* inside the same file on lines 595, 613:
```                        [False, True, True],
                        slice(0, 1)]:
            self.assert_eq(kdf.iloc[:, indexer], pdf.iloc[:, indexer])
            self.assert_eq(kdf.iloc[:1, indexe...(truncated 198 chars)
```

 _Now that you are on the file, it would be easier to pay back some tech. debt._


### :star: Change Overview


![Showing the changed files, dependency changes and the impact - click for full size](https://opensource.softagram.com/cdn/impact/dd6f973b-a9da-477d-8f74-1a29916573a5_koalas_1188_impact_change_graph_ZleeXt6E3aGUhhc4AyM0Q32ShpUdwW.png)
*(Open in [Softagram Desktop](https://softagram.com/desktop) for full details)*






### :page_facing_up: Full report

* Permalink: [Full report for pull/1188](https://opensource.softagram.com/dashboard/teams/7ff2e81c-61e0-4605-b4ac-d496bc4c37c7/projects/dd6f973b-a9da-477d-8f74-1a29916573a5/impact-reports/5c526117-14e3-488c-8e54-f700f6a32fa6)

#
[Impact Report explained.](http://help.softagram.com/en/articles/2847546-pull-merge-request-report-explained) Give feedback on this report to support@softagram.com

@itholic I'll take this over.

Resolving patch:
diff --git a/databricks/koalas/indexing.py b/databricks/koalas/indexing.py
--- a/databricks/koalas/indexing.py
+++ b/databricks/koalas/indexing.py
@@ -725,20 +725,19 @@ def _select_cols(self, cols_sel):
 
         returns_series = False
 
-        if isinstance(cols_sel, slice):
-            if cols_sel == slice(None):
-                cols_sel = None
-            else:
-                raise LocIndexer._raiseNotImplemented(
-                    "Can only select columns either by name or reference or all"
-                )
-        elif isinstance(cols_sel, (Series, spark.Column)):
+        if isinstance(cols_sel, (Series, spark.Column)):
             returns_series = True
             cols_sel = [cols_sel]
 
-        if cols_sel is None:
+        if cols_sel is None or cols_sel == slice(None):
             column_labels = self._internal.column_labels
             data_spark_columns = self._internal.data_spark_columns
+        elif isinstance(cols_sel, slice):
+            start, stop = self._kdf_or_kser.columns.slice_locs(
+                start=cols_sel.start, end=cols_sel.stop
+            )
+            column_labels = self._internal.column_labels[start:stop]
+            data_spark_columns = self._internal.data_spark_columns[start:stop]
         elif isinstance(cols_sel, (str, tuple)):
             if isinstance(cols_sel, str):
                 cols_sel = (cols_sel,)


Test patch:
diff --git a/databricks/koalas/tests/test_indexing.py b/databricks/koalas/tests/test_indexing.py
--- a/databricks/koalas/tests/test_indexing.py
+++ b/databricks/koalas/tests/test_indexing.py
@@ -478,6 +478,10 @@ def test_loc2d_multiindex(self):
         self.assert_eq(kdf.loc[:, "a"], pdf.loc[:, "a"])
         self.assert_eq(kdf.loc[5:5, "a"], pdf.loc[5:5, "a"])
 
+        self.assert_eq(kdf.loc[:, "a":"a"], pdf.loc[:, "a":"a"])
+        self.assert_eq(kdf.loc[:, "a":"c"], pdf.loc[:, "a":"c"])
+        self.assert_eq(kdf.loc[:, "b":"c"], pdf.loc[:, "b":"c"], almost=True)
+
     def test_loc2d(self):
         kdf = self.kdf
         pdf = self.pdf
@@ -509,6 +513,10 @@ def test_loc2d(self):
         self.assert_eq(kdf.loc[5, ["a"]], pdf.loc[5, ["a"]])
         self.assert_eq(kdf.loc[9, ["a"]], pdf.loc[9, ["a"]])
 
+        self.assert_eq(kdf.loc[:, "a":"a"], pdf.loc[:, "a":"a"])
+        self.assert_eq(kdf.loc[:, "a":"d"], pdf.loc[:, "a":"d"])
+        self.assert_eq(kdf.loc[:, "c":"d"], pdf.loc[:, "c":"d"], almost=True)
+
     def test_loc2d_multiindex_columns(self):
         arrays = [np.array(["bar", "bar", "baz", "baz"]), np.array(["one", "two", "one", "two"])]
 
@@ -518,6 +526,33 @@ def test_loc2d_multiindex_columns(self):
         self.assert_eq(kdf.loc["B":"B", "bar"], pdf.loc["B":"B", "bar"])
         self.assert_eq(kdf.loc["B":"B", ["bar"]], pdf.loc["B":"B", ["bar"]])
 
+        self.assert_eq(kdf.loc[:, "bar":"bar"], pdf.loc[:, "bar":"bar"])
+        self.assert_eq(kdf.loc[:, "bar":("baz", "one")], pdf.loc[:, "bar":("baz", "one")])
+        self.assert_eq(
+            kdf.loc[:, ("bar", "two"):("baz", "one")], pdf.loc[:, ("bar", "two"):("baz", "one")]
+        )
+        self.assert_eq(kdf.loc[:, ("bar", "two"):"bar"], pdf.loc[:, ("bar", "two"):"bar"])
+        self.assert_eq(kdf.loc[:, "a":"bax"], pdf.loc[:, "a":"bax"])
+        self.assert_eq(
+            kdf.loc[:, ("bar", "x"):("baz", "a")],
+            pdf.loc[:, ("bar", "x"):("baz", "a")],
+            almost=True,
+        )
+
+        pdf = pd.DataFrame(
+            np.random.randn(3, 4),
+            index=["A", "B", "C"],
+            columns=pd.MultiIndex.from_tuples(
+                [("bar", "two"), ("bar", "one"), ("baz", "one"), ("baz", "two")]
+            ),
+        )
+        kdf = ks.from_pandas(pdf)
+
+        self.assert_eq(kdf.loc[:, "bar":"baz"], pdf.loc[:, "bar":"baz"])
+
+        self.assertRaises(KeyError, lambda: kdf.loc[:, "bar":("baz", "one")])
+        self.assertRaises(KeyError, lambda: kdf.loc[:, ("bar", "two"):"bar"])
+
     def test_loc2d_with_known_divisions(self):
         pdf = pd.DataFrame(
             np.random.randn(20, 5), index=list("abcdefghijklmnopqrst"), columns=list("ABCDE")