ENH: #9847, adding a "same" and "expand" param to StringMethods.split()
State: closed

closes #9847


Labels:
name: API Design
name: Strings
description: String extension data type and string data

Comments:
I haven't read through all the issues, but has expanding an Index to a MultiIndex (if needed) been proposed? Analogous to `Series.str.*` -> `DataFrame`

@TomAugspurger ohh, that would be nice, but let's make another issue about that.

Did the `'index'` option ever make it into a release? I'm pretty sure it didn't, in which case we don't need to preserve it.

@TomAugspurger yes, I also thought about that! Looks like a very good idea to me, but indeed lets discuss in a separate issue (eg I think it should be in the main namespace, not in str)

Indeed, `'index'` should not be mentioned.

I am still slightly in favor of having a `expand=True/False` instead of `return_type='same'/'expand'`.

Keeping `return_type='series'/'frame'` as backcompat has to happen in both cases, and it is only there for one release.

Yes, `expand=True/False` seems like a slightly cleaner API than string options.

@sreejata can you update for the comments above?

I'm also slightly in favor of `expand=True/False`. It seems cleaner and more intuitive especially since we'd like to do a similar expansion for `Index.str` -> `MultiIndex` in `v0.17.0`

ok, let's go with `expand=True/False` then. This should be complmetely back-compat (use the `deprecate` decorator)

@sreejata can you update

I've implemented the `expand` wrapper in #9773. #9773 work as below

| Invoked by | `expand=False` | `expand=True (default)` |
| --- | --- | --- |
| `Series` | `Series` | `DataFrame` |
| `Index` | `Index (of tuples)` | `MultiIndex` |

I think the points are

#### `split` (this PR):
- Can follow the above behavior simply replacing `return_type` to `expand`. 
- Whether to change the defualt to `True` on this occasion.

#### `extract` (#9985)
- Expected `Series` and `Index` to work as the same. 
- Adding `extract` option breaks current `Series` behavior which implicitly decide returning type.

#### `get_dummies` (#9985)
- Expected `Series` and `Index` to work as the same. 
- Adding `extract` option breaks current `Series` behavior which always return `DataFrame`.

@sinhrks 

this needs to handle the backward compat issue (by deprecating the existing `return_type=series/frame`). The `index` is new in 0.16.1, so no need to handle that.

@sinhrks can you split out the `return_type` changes (and incorporate the deprecation warning), from #9773 (as that is for partition, yes?). Then can close this PR.

@jreback #9773 the logic is implemented in `_wrap_result_expand`, and it can be used in `str.split` as an alternative of `_wrap_result`. I think changes required in this PR are:
- adding `deprecate_kwargs` to `str_split`
- remove `return_type` logic from `str_split` (https://github.com/pydata/pandas/pull/9870/files#diff-4627140a96de4a6390883e22e983f1b2R675)
- Change `str.split` to use `_wrap_result_expand` rather than `_wrap_result`.

@sinhrks are you able to put in the deprecation of the existing `return_type` logic? (in favor or expand)?

@sreejata Can you update above?

@jreback If this needs in v0.16.1, I'll work on it.

ideally this could be fixed in next day or 2

replaced by #10085