Feature/get dotted columns by prefix
State: closed

 - [ ] closes #xxxx
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master --name-only -- '*.py' | flake8 --diff``
 - [ ] whatsnew entry

Assuming a DataFrame like this:
```
>> df
    foo  bar.foo  bar.bar
0  0.32     0.15     0.45
1  0.62     0.73     0.36
2  0.21     0.68     0.10
3  0.05     0.36     0.90
```

one can access to `df['foo']` with the shortcut `df.foo`.

This PR extends this idea to column names containing dots, so that `df.bar` yields:

```
>> df.bar
    foo   bar
0  0.15  0.45
1  0.73  0.36
2  0.68  0.10
3  0.36  0.90
```

I'm new to contributing in pandas: so I need a few help:

* Do I need to create an issue to *host* the PR ?
* Where in the docs should I host explainations about this ?

Thanks in advance (and pandas is awesome btw !)

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: API Design

Comments:
> Do I need to create an issue to host the PR ?

Not necessary, but typically we use issues to talk about whether a change is likely to be accepted, so that a contributor doesn't spend time on something that doesn't fit well in pandas, which might be relevant here :)

I'll let others chime in, but this might not receive support from the other maintainers. My initial reaction is -1 since it complicates when `.` access will work, and what will be returned if it does. Right now, it works when

- the column name is a valid python identifier
- the column name doesn't shadow an existing method

and always returns the same thing as `df[column_name]`.

I think this change would complicate that, `df[prefix]` is a KeyError, but `df.prefix` would potentially return some columns.
This is an interesting idea, but I'm slightly negative on this being accepted too.  I think a `MultiIndex` already handles this usecase reasonably well?

```
def make_col(s):
    pieces = s.split('.')
    if len(pieces) == 1:
        return tuple([pieces[0], ''])
    else:
        return tuple(pieces)

df.columns = pd.MultiIndex.from_tuples(df.columns.map(make_col))

df
Out[61]: 
    foo   bar      
          foo   bar
0  0.32  0.15  0.45
1  0.62  0.73  0.36
2  0.21  0.68  0.10
3  0.05  0.36  0.90

df['bar']
Out[62]: 
    foo   bar
0  0.15  0.45
1  0.73  0.36
2  0.68  0.10
3  0.36  0.90

df.bar
Out[63]: 
    foo   bar
0  0.15  0.45
1  0.73  0.36
2  0.68  0.10
3  0.36  0.90
```

@chris-b1 
>  I think a MultiIndex already handles this usecase reasonably well?

That's true. Your code works when you know the number of levels to be created. But this can handle the general case with few effort:

```
nb_level = df.columns.map(lambda x: len(x.split('.'))).max()
df.columns = df.columns.map(lambda x: tuple((x.split('.') + ['']*nb_level)[:nb_level]))
```

This being said, plus @TomAugspurger's remark on `df.prefix` being different from `df['prefix']`, I guess this PR can be closed.

Thanks for the review and comments.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16320?src=pr&el=h1) Report
> Merging [#16320](https://codecov.io/gh/pandas-dev/pandas/pull/16320?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/0607e03f71b52e131947d73880380796f51d72fb?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16320/graphs/tree.svg?width=650&height=150&src=pr&token=eZ4WkYLtcO)](https://codecov.io/gh/pandas-dev/pandas/pull/16320?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16320      +/-   ##
==========================================
- Coverage   90.37%   90.35%   -0.02%     
==========================================
  Files         161      161              
  Lines       50863    50873      +10     
==========================================
  Hits        45966    45966              
- Misses       4897     4907      +10
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.13% <100%> (ø)` | :arrow_up: |
| #single | `40.33% <80%> (-0.1%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16320?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/16320?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.6% <100%> (-0.09%)` | :arrow_down: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/16320?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/16320?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `91.88% <0%> (-0.06%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16320?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16320?src=pr&el=footer). Last update [0607e03...7031716](https://codecov.io/gh/pandas-dev/pandas/pull/16320?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

this is too much magic.