Change Panel.shift to use NDFrame.shift
State: closed

closes #4867

This brings `Panel.shift` in line with `DataFrame.shift`. The shifted data retains the same dimensions/indexes and doesn't drop the empty rows. So I have two concerns. 
1. This is an API change and could conceivably break existing code. 
2. The perf regresses quite a bit. 

Here is the vbench.

```
-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------
panel_shift                                  | 578.2867 |   0.0773 | 7478.4491 |
-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------
```

The previous implementation did not create a copy and thus had a constant time. The general shift scales linearly. 

I think the current implementation just defers work done in later alignments, but I can only speak to my own workflow.


Labels:
name: API Design
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
hmm. I bet the rolling stuff that makes a 2-d much faster makes this much slower.

I think you mnay have to roll this in a different way somehow.

Hm. You talking about #6404? I'm going to setup a test battery just to get an idea of what the timings are. I think that the shift isn't that much worse than a straight copy. The regression is going to look disgusting compared to grabbing a view :P 

#6404 was merged in, pls rebase and see about the perf tests and what if anything can do about them

With #6747 the panel shift speed is on par with pre-quick-shift. 

http://nbviewer.ipython.org/gist/dalejung/9731798/panel%20shift.ipynb
I monkey patched the old shift code onto the Panel to compare. 

@jreback ping on green. 

gr8 can you add a release note (referencing the issue)

and squash downto 1 commit

ping me whengreen

@jreback green

I thiought this was fixed?

```
panel_shift                                  | 263.2827 |   0.0746 | 3528.0756 |
-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------

Ratio < 1.0 means the target commit is faster then the baseline.
Seed used: 1234

Target [e84efe5] : Merge pull request #6825 from cpcloud/replace-dict-vbenches

BENCH: add vbench for issue 6697
Base   [d10a658] : RLS: set released to True. v0.13.1

```

https://github.com/pydata/pandas/issues/6826

That particularly perf hit won't be fixable if `Panel` uses the general shift. The 0.13.1 `Panel.shift` just creates a view, which is near instant and constant time. 

The quick shift fixes #6747 fixes speed for `shift` generally which included the `Panel` general shift case. 

So the timeline:
1. Panel.shift takes views on data. 0ms
2. General shift PR. Perf hit. 500ms
3. quick shift PR (np.roll) #6404. Panel.shift/F-ordered shift degrade 1.2s
4. perf fixes for quick shift #6746 Panel.shift back down to 500ms

Sorry, I think the fix from 3->4 was conflated with fixing the perf hit from 1->2. 

Let me know if I need to back out the PR.

ok

either back it out completely or
incorporate into the same shift logic in core/internals

so it's at least in 1 place (even if special cased)

moved discussion to #6826