Rolling window endpoints inclusion
State: closed

 - [x] closes #13965 
 - [x] time-based window tests added / passed
 - [x] fixed window tests added / passed
 - [x] passes ``git diff upstream/master --name-only -- '*.py' | flake8 --diff``
 - [x] whatsnew entry


Labels:
name: Datetime
description: Datetime data dtype
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: API Design

Comments:
Here is a quick example of what this feature looks like

```
df = pd.DataFrame({'x': [1]*5},
....:                 index = [pd.Timestamp('20130101 09:00:01'),
....:                          pd.Timestamp('20130101 09:00:02'),
....:                          pd.Timestamp('20130101 09:00:03'),
....:                          pd.Timestamp('20130101 09:00:04'),
....:                          pd.Timestamp('20130101 09:00:06')])

df["right"] = df.rolling('2s', closed='right').x.sum()   # current behavior
df["both"] = df.rolling('2s', closed='both').x.sum()
df["left"] = df.rolling('2s', closed='left').x.sum()
df["open"] = df.rolling('2s', closed='neither').x.sum()

print(df)
                     x  right  both  left  open
2013-01-01 09:00:01  1    1.0   1.0   NaN   NaN
2013-01-01 09:00:02  1    2.0   2.0   1.0   1.0
2013-01-01 09:00:03  1    2.0   3.0   2.0   1.0
2013-01-01 09:00:04  1    2.0   3.0   2.0   1.0
2013-01-01 09:00:06  1    1.0   2.0   1.0   NaN
```
@carlosdanielcsantos https://github.com/pandas-dev/pandas/pull/15795#issuecomment-288883027 are basically the docs (with a bit of prose) and nice formatting. Laying out the possibilities (in a sub-section of computation.rst). Note you also might want to have a 'real-world' uses case sentence for some of these. 
you can see pep errors: http://pandas-docs.github.io/pandas-docs-travis/contributing.html#python-pep8

use ``git diff master | flake8 --diff``
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=h1) Report
> Merging [#15795](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/9d3554c26aa85d66cbfe6f481464f1a357af5e12?src=pr&el=desc) will **decrease** coverage by `0.02%`.
> The diff coverage is `92.85%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/15795/graphs/tree.svg?width=650&src=pr&token=eZ4WkYLtcO&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #15795      +/-   ##
==========================================
- Coverage   91.01%   90.99%   -0.03%     
==========================================
  Files         143      143              
  Lines       49400    49404       +4     
==========================================
- Hits        44963    44956       -7     
- Misses       4437     4448      +11
```


| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `96.24% <100%> (ø)` | :arrow_up: |
| [pandas/core/window.py](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvd2luZG93LnB5) | `96.08% <92.3%> (-0.11%)` | :arrow_down: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/common.py](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tbW9uLnB5) | `90.96% <0%> (-0.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.86% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=footer). Last update [9d3554c...8533421](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=h1) Report
> Merging [#15795](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/838e09ccef896a1265b5447fb14f0969f7cd86d5?src=pr&el=desc) will **increase** coverage by `0.02%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/15795/graphs/tree.svg?token=eZ4WkYLtcO&width=650&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #15795      +/-   ##
==========================================
+ Coverage      91%   91.03%   +0.02%     
==========================================
  Files         145      145              
  Lines       49581    49592      +11     
==========================================
+ Hits        45122    45146      +24     
+ Misses       4459     4446      -13
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.8% <100%> (+0.02%)` | :arrow_up: |
| #single | `40.56% <0%> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `96.26% <ø> (ø)` | :arrow_up: |
| [pandas/core/window.py](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvd2luZG93LnB5) | `96.24% <100%> (+0.02%)` | :arrow_up: |
| [pandas/core/common.py](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tbW9uLnB5) | `90.68% <0%> (-0.35%)` | :arrow_down: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `81.13% <0%> (+0.18%)` | :arrow_up: |
| [pandas/tools/plotting.py](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=tree#diff-cGFuZGFzL3Rvb2xzL3Bsb3R0aW5nLnB5) | `72.47% <0%> (+0.71%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=footer). Last update [838e09c...aad97dc](https://codecov.io/gh/pandas-dev/pandas/pull/15795?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
@jreback How should I add this to computations.rst? A new section inside Window Functions or just add the examples inside the existing Window Functions -> Rolling Windows? 
@carlosdanielcsantos lgtm. some more doc-string nitpicks.
ping when all green.
@jreback I was going to start writing tests for the FixedWindowIndexer but there are no explicit rolling computing test going on in TestRolling (they are done in TestMoments, I think). Where should I add these new tests?
> @jreback How should I add this to computations.rst? A new section inside Window Functions or just add the examples inside the existing Window Functions -> Rolling Windows?

exactly, a new sub-section that pretty much shows the example with all 4 closed. maybe right before Centering Windows?
> @jreback I was going to start writing tests for the FixedWindowIndexer but there are no explicit rolling computing test going on in TestRolling (they are done in TestMoments, I think). Where should I add these new tests?

We don't have explicit tests for the cython routines, these are covered by the integration tests (top-level routines). Though certainly could add them if you want / can. 

You could add a new class in test_windows that explicity exercises these.
@jreback @chrisaycock 
Thinking deeper about the fixed window case, it seems that, intuitively, the default behavior right now better resembles the closed='both' case. Notice that when you use rolling(3).apply(f), the result for index i will be the result of f applied to `[i-2, i-1, i]`, not `]i-2, i-1, i]` (ISO 31-11 math set notation). One could argue that this is equivalent to having closed='right' with f applied to `]i-3, i-2, i-1, i]`. Indeed, but this probably adulterates the meaning of the window size.
> @jreback @chrisaycock
Thinking deeper about the fixed window case, it seems that, intuitively, the default behavior right now better resembles the closed='both' case. Notice that when you use rolling(3).apply(f), the result for index i will be the result of f applied to [i-2, i-1, i], not ]i-2, i-1, i] (ISO 31-11 math set notation). One could argue that this is equivalent to having closed='right' with f applied to ]i-3, i-2, i-1, i]. Indeed, but this probably adulterates the meaning of the window size.

@carlosdanielcsantos can you show a comparison of these 2, to show more what you mean?
@jreback 
```
pd.DataFrame([1] * 5).rolling(3, min_periods=1).sum()

     0
0  1.0
1  2.0
2  3.0
3  3.0
4  3.0
```

Look how these results come from the application of sum() to every element inside the valid window. This resembles `closed='both'`. `closed=right` would look like `[1.0, 1.0, 2.0, 2.0, 2.0]`. Maybe the confusion arises from the fact that in fixed windows we are using the size of the window and not an interval as in the variable window, so **maybe** the closed parameter doesn't quite make sense here. 

Nonetheless, I think that the possibility of ignoring the current index in computations is valuable (that was my initial problem, after all). Some solutions for this are:

1. Use `closed='both'` as default in Fixed windows and allow `closed = 'right', 'left', 'neither' `as well.
2. Drop the closed argument in Fixed windows but allow an argument to ignore the current index (same behavior as `closed='left'` in variable windows). The other cases equivalent to `closed='right', 'neither'` in variable windows can be obtained by varying window size: for example:` rolling(3, closed='right')` is the same as `rolling(2)` and `rolling(3, closed='neither')` is the same as `rolling(2, ignore_own_index=True)`
> Nonetheless, I think that the possibility of ignoring the current index in computations is valuable (that was my initial problem, after all). Some solutions for this are:

This a bit orthogonal though. We *always* want to actually use the index. Since using a fixed window size, it *looks* like we are ignoring the index :>.

So a possiblity is to actually check the index whether its equiv to a RangeIndex. e.g.

```
In [5]: df1 = pd.DataFrame([1] * 5)

In [6]: df1.rolling(3, min_periods=1).sum()
Out[6]: 
     0
0  1.0
1  2.0
2  3.0
3  3.0
4  3.0

In [8]: df2 = pd.DataFrame([1] * 5, index=[0, 1, 3, 4, 5])

In [9]: df2.rolling(3, min_periods=1).sum()
Out[9]: 
     0
0  1.0
1  2.0
3  3.0
4  3.0
5  3.0
```

```
In [11]: (np.diff(df1.index)==1).all()
Out[11]: True

In [12]: (np.diff(df2.index)==1).all()
Out[12]: False

```

Let's deal with this in a separate issue. 

A way to disambiguate on this ATM is to allow ``closed=None`` to be the default parameter. Then it cannot be anything but that for a FixedWindow. For Variable, if we see ``closed=None`` -> ``closed='right'`` IOW the default is the same. And then allow the rest of the closed intervals.

I think *we* should fix this for Fixed intervals, but I agree this is a problem because we essentially have ``closed='both'`` as the default now (its implicit though) (as you stated above).


@carlosdanielcsantos if you can update would be great (docs and such), ignoring the issue about fixed windows for the moment.
@carlosdanielcsantos can you update
@jreback I'll do that in the next hours. Should I just leave the fixed-window as it is?
I think make the ``closed=None`` the default, and document the possibilities (and default) for the variable sized window (and validate left,right,both,neither). For the fixed window document as well (you can say its closed='both', but ATM only allow None). and then open a new issue where we can deal with this.

We have to fix this, but this PR merely opens up the possibilities of change here. So I think ok to merge this.
@carlosdanielcsantos can you update
@jreback made some corrections but circleci is not passing in test_append_index - pandas.tests.test_multilevel.TestMultiLevel, but passing locally. Any thoughts?

try pushing again (make a modification)
looks like a fluke 
@jreback Still not passing. Could I have broken this with the changes in window?
@carlosdanielcsantos I rebased you against current master. This should pass cleanly. If you'd make those other changes and ping.
@jreback what changes? I believe I updated everything that you requested. Can you take a look at AppVeyor build that failed?
thanks @carlosdanielcsantos nice PR. as you can see, the code changes are the 'easy' part, its the docs & testing that are the details.!

keem em coming!