Allow DateOffset addition with Series
State: closed

Addresses #10699 - allowing relative offsets like `DateOffset(years=1)` to to be added to a `Series` containing datetimes for element-wise addition.

Previously only non-relative offsets that could be mapped to a time_delta (`Tick`, `Day`, etc) worked.

Some performance numbers are below - `Tick` based offsets should be unchanged, a subset of offsets (`DateOffset`, `YearEnd/Begin`, `MonthEnd/Begin`, `QuarterEnd/Begin`, `Week`, `BusinessDay`) will be sped up significantly; and other offsets will still fallback to a slow path, but now will raise a performance warning.
### Master

```
In [1]: idx = pd.date_range(start='20140101', freq='T', periods=100000)

In [2]: %timeit idx + pd.offsets.Day()
100 loops, best of 3: 5.72 ms per loop

In [3]: %timeit idx + pd.Timedelta(1, 'D')
100 loops, best of 3: 5.67 ms per loop

In [4]: %timeit idx + pd.offsets.DateOffset(years=1, months=2, days=1)
1 loops, best of 3: 1.79 s per loop

In [5]: %timeit idx + pd.offsets.WeekOfMonth(week=2, weekday=4)
1 loops, best of 3: 22.8 s per loop
```
### PR

```
In [3]: %timeit idx + pd.offsets.Day()
100 loops, best of 3: 5.8 ms per loop

In [4]: %timeit idx + pd.Timedelta(1, 'D')
100 loops, best of 3: 5.73 ms per loop

In [5]: %timeit idx + pd.offsets.DateOffset(years=1, months=2, days=1)
10 loops, best of 3: 86.9 ms per loop

In [6]: %timeit idx + pd.offsets.WeekOfMonth(week=2, weekday=4)
1 loops, best of 3: 23.4 s per loop
c:\users\chris\documents\python-dev\pandas\pandas\tseries\index.py:695: PerformanceWarning:     Non-vectorized DateOffset being applied to Series or DatetimeIndex  PerformanceWarning)
```


Labels:
name: Datetime
description: Datetime data dtype
name: Frequency
description: DateOffsets

Comments:
This is a nice idea, but it wasn't originally implemented because it has massive perf issues.
However, a divide & conquer strategy can fix most of this. 

```
In [61]: np.random.seed(1234)

In [62]: s = Series(pd.to_timedelta(np.random.randint(0,10000000,size=10000),unit='s') + Timestamp('20130101'))

In [63]: pd.set_option('max_rows',10)

In [66]: s
Out[66]: 
0      2013-01-06 15:03:11
1      2013-03-20 00:58:59
2      2013-02-20 10:57:42
3      2013-03-13 04:40:21
4      2013-01-12 07:42:04
               ...        
9995   2013-02-17 07:22:39
9996   2013-04-03 04:57:11
9997   2013-04-16 10:06:38
9998   2013-02-15 15:28:46
9999   2013-03-05 12:00:39
dtype: datetime64[ns]
```

Any offset that has a `_inc` (e.g. <= Day), is simply convertible to a `Timedelta`. (this _might_ work for Week as well as its a fixed interval).

Note that your PR is equivalent to item by item apply.
**note that I realized after I wrote this you are only dealing with non-Tick, but put this simple comparison here anyhow**

```
In [78]: %timeit s.apply(lambda x: x+pd.offsets.Day())
1 loops, best of 3: 347 ms per loop

In [79]: %timeit s+Timedelta('1D')
1000 loops, best of 3: 610 us per loop
```

Conversions to > Day are a bit tricky. Note that you need to convert using `pd.DatetimeIndex()` because the obvious `s.dt.to_period()` round-trips thru `object` which defeats the purpose.
This would do a MonthEnd conversion. I think some of this could be make easier with some additonal methods (prob to `DatetimeIndex`)

```
In [67]: i = pd.DatetimeIndex(s)

In [68]: base = (((i.to_period('M')+1).to_timestamp()).to_period('D')-1).to_timestamp()

In [69]: off = pd.TimedeltaIndex(i.asi8-i.to_period('D').to_timestamp().asi8)

In [70]: result = s.apply(lambda x: x+pd.offsets.MonthEnd())

In [71]: %timeit s.apply(lambda x: x+pd.offsets.MonthEnd())
1 loops, best of 3: 569 ms per loop
In [72]: result
Out[72]: 
0      2013-01-31 15:03:11
1      2013-03-31 00:58:59
2      2013-02-28 10:57:42
3      2013-03-31 04:40:21
4      2013-01-31 07:42:04
               ...        
9995   2013-02-28 07:22:39
9996   2013-04-30 04:57:11
9997   2013-04-30 10:06:38
9998   2013-02-28 15:28:46
9999   2013-03-31 12:00:39
dtype: datetime64[ns]

In [74]: Series(base+off)
Out[74]: 
0      2013-01-31 15:03:11
1      2013-03-31 00:58:59
2      2013-02-28 10:57:42
3      2013-03-31 04:40:21
4      2013-01-31 07:42:04
               ...        
9995   2013-02-28 07:22:39
9996   2013-04-30 04:57:11
9997   2013-04-30 10:06:38
9998   2013-02-28 15:28:46
9999   2013-03-31 12:00:39
dtype: datetime64[ns]

In [77]: %timeit Series((((i.to_period('M')+1).to_timestamp()).to_period('D')-1).to_timestamp() + pd.TimedeltaIndex(i.asi8-i.to_period('D').to_timestamp().asi8))
100 loops, best of 3: 7.74 ms per loop
```

Finally, you need to do Anchored offsets like `BusinessDay`, `W-THU` via `.apply` (.e.g. I am not sure these can be vectorized, though maybe for something like `W-THU` you could do it).

You should emit a `PerformanceWarning` if you accept these last ones (or just raise).

Note the above will fail if the freq cannot be inferred.

Furthermore this would need tests for each of the offsets as well as subtracting.

Performance tests as well.

this could be a property of a `DateimeIndex`, it is equivalent of `.time`, but returns a much more useful format `pd.TimedeltaIndex(i.asi8-i.to_period('D').to_timestamp().asi8)`, maybe `.timedelta` (though not that descsriptive)

```
In [8]: pd.TimedeltaIndex(i.asi8-i.to_period('D').to_timestamp().asi8),
Out[8]: 
(TimedeltaIndex(['15:03:11', '00:58:59', '10:57:42', '04:40:21', '07:42:04', '01:23:36', '21:41:13', '08:43:29', '08:03:53', '05:31:23', 
                 ...
                 '09:22:04', '14:21:48', '05:27:49', '17:05:16', '00:27:34', '07:22:39', '04:57:11', '10:06:38', '15:28:46', '12:00:39'],
                dtype='timedelta64[ns]', length=10000, freq=None),)

In [9]: i.time
Out[9]: 
array([datetime.time(15, 3, 11), datetime.time(0, 58, 59),
       datetime.time(10, 57, 42), ..., datetime.time(10, 6, 38),
       datetime.time(15, 28, 46), datetime.time(12, 0, 39)], dtype=object)

In [10]: %timeit i.time
100 loops, best of 3: 18.9 ms per loop

In [11]: %timeit pd.TimedeltaIndex(i.asi8-i.to_period('D').to_timestamp().asi8),
100 loops, best of 3: 2.55 ms per loop
```

Thanks.  For those offsets that can be vectorized, where would that logic sit?  Something like a `vectorized_apply` method that `DateOffset.__add__` calls to if the other is array-like?

you can prob put it in the same place (but maybe put in a separate method in `_Timeop`). Though some of the functions can put put in `DatetimeIndex` i think.

this won't call `DateOffset.__add__` at all.

@jreback can you take a look at the latest updates?  It might not be all the way there, but implements most of the obvious fastpaths I could see.

Instead of `.timedelta` like you showed  I added a `period_delta(freq)` method to `DatetimeIndex` which is the same idea, but handles other frequencies too ('M' and 'W' used).  Wasn't sure if this should be public.

@chris-b1 looks pretty good. Just a bit or reorg to put the code in the appropriate spots. 

also if you'd add some vbenches would be great (and show the results in the very top of the PR).

@jreback  - see the update,  I think I've got most everything addressed, thanks for all the notes.

I wasn't sure what the `DateOffset` method that adds to a `DatetimeIndex` should be called - you suggested `period_delta` - I think that may be confusing where it doesn't return a `Timedelta`?  I ended up using `apply_index` but would be glad to change it to something better.

I'm on Windows and haven't able to get vbench working quite right - I did add some new benchmarks, and put timeits at the top of the PR.

looks pretty good. couple of code comments (mostly minor, you did nice job on moving things around!).
couple of doc additions. 

@jreback  - here's another update - I addressed your comments, and expanded the tests, which highlighted a couple other changes needed.
- I had broke adding offsets to `Timedelta`s
- Handling normalized offsets
- `month` kwarg in `YearEnd/Begin`; matching rollforwards when date is on `End/Begin` point

I also added vectorized paths for the `BusinessDay`, `Week`, `Quarter` offsets - same of approach of manipulating their `Period` equivalent.

@chris-b1 looking pretty good. couple of doc changes / test changes. Almost there!

@jreback thanks, latest has the changes to doc / testing from your comments.

small doc adds. pls squash. ping when green.

@jreback green - thanks!

@chris-b1 great effort! sometimes these are involved issues!