Implement business_start/end cases for shift_months
State: closed

Unifies `apply_index` implementations for MonthEnd/MonthBegin, plus extends them to BMonthEnd and BMonthBegin.

Unifies `onOffset` implementations for QuarterEnd/BQuarterEnd, plus extends them to QuarterBegin/BQuarterBegin.

Implements a `cdef` version of `monthrange`.

Labels:
name: Performance
description: Memory or execution speed performance
name: Frequency
description: DateOffsets
name: Clean

Comments:
(will run asv shortly)
OK, I've addressed all the comments locally, am running benchmarks (several times) and fixed the travis flake problem.  Doing some extra cleanup and parametrization of the tests to make sure the affected offset methods are covered, will update later.
OK, just confirmed: the existing tests do _not_ hit the new `apply_index` methods, and the speedup is pretty enormous.
what is the perf issue here?
> what is the perf issue here?

I'm still running asvs to try to pin it down.  AFAICT something in ccalendar must be less performant than the version in src/datetime

```
taskset 4 asv continuous -f 1.1 -E virtualenv master HEAD -b offset -b timeseries -b period -b timedelta
[...]
       before           after         ratio
     [be66ef83]       [b244bab9]
+      2.50±0.3ms       4.81±0.4ms     1.93  period.Algorithms.time_drop_duplicates('series')
+     2.81±0.02ms      5.27±0.07ms     1.88  period.Algorithms.time_value_counts('series')
+        398±20ms          460±1ms     1.16  offset.SemiMonthOffset.time_begin_apply_index
+         401±1ms          461±2ms     1.15  offset.SemiMonthOffset.time_end_apply_index
+         406±6ms          458±2ms     1.13  offset.SemiMonthOffset.time_begin_incr_rng
-      22.8±0.5μs      20.4±0.06μs     0.89  timeseries.DatetimeAccessor.time_dt_accessor
-         126±6μs        112±0.3μs     0.89  timeseries.DatetimeIndex.time_unique
-           1.56s            1.38s     0.89  offset.ApplyIndex.time_apply_series(<BusinessYearBegin: month=1>)
-        68.0±3μs       60.1±0.2μs     0.88  period.PeriodUnaryMethods.time_now('min')
-      79.1±0.1μs       68.6±0.2μs     0.87  period.PeriodProperties.time_start_time('M')
-        25.7±2ms         22.0±0ms     0.86  timeseries.DatetimeIndex.time_to_pydatetime
-         180±5ns        154±0.6ns     0.85  timedelta.TimedeltaProperties.time_timedelta_nanoseconds
-           1.61s            1.37s     0.85  offset.ApplyIndex.time_apply_series(<BusinessYearEnd: month=12>)
-     11.3±0.07μs      9.55±0.08μs     0.85  offset.YearBegin.time_timeseries_year_apply
-        24.8±2μs       20.8±0.1μs     0.84  offset.SemiMonthOffset.time_begin_decr
-           25.2s            21.1s     0.84  gil.nogil_datetime_fields.time_period_to_datetime
-      26.9±0.2μs      22.1±0.09μs     0.82  period.PeriodUnaryMethods.time_now('M')
-           1.74s            1.42s     0.82  offset.ApplyIndex.time_apply_series(<BusinessQuarterEnd: startingMonth=3>)
-        1.04±0μs          848±2ns     0.81  period.PeriodProperties.time_minute('min')
-           9.62s            7.76s     0.81  gil.nogil_datetime_fields.time_datetime_to_period
-      9.17±0.3ms      7.31±0.09ms     0.80  timeseries.DatetimeIndex.time_dti_tz_factorize
-        15.3±1ms      12.2±0.06ms     0.80  timeseries.DatetimeIndex.time_infer_freq_none
-           1.46s       14.5±0.3ms     0.01  offset.ApplyIndex.time_apply_series(<BusinessMonthEnd>)
-           1.48s      13.8±0.08ms     0.01  offset.ApplyIndex.time_apply_series(<BusinessMonthBegin>)
-           1.50s      11.9±0.09ms     0.01  offset.ApplyIndex.time_apply_index(<BusinessMonthBegin>)
-           1.54s       12.3±0.2ms     0.01  offset.ApplyIndex.time_apply_index(<BusinessMonthEnd>)
```

b244bab9 is after updating all the cimports as suggested, except for one in `fields` that I reverted to try track down the regression.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=h1) Report
> Merging [#18489](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/d1010643fea058ba43c2c7124af75cc462ccf242?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `95.23%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18489/graphs/tree.svg?height=150&src=pr&token=eZ4WkYLtcO&width=650)](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18489      +/-   ##
==========================================
- Coverage   91.33%   91.32%   -0.02%     
==========================================
  Files         163      163              
  Lines       49801    49773      -28     
==========================================
- Hits        45487    45454      -33     
- Misses       4314     4319       +5
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.12% <95.23%> (-0.02%)` | :arrow_down: |
| #single | `40.72% <28.57%> (-0.05%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/tseries/offsets.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvb2Zmc2V0cy5weQ==) | `96.94% <95.23%> (-0.09%)` | :arrow_down: |
| [pandas/core/indexes/interval.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9pbnRlcnZhbC5weQ==) | `92.69% <0%> (-0.53%)` | :arrow_down: |
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `95.52% <0%> (-0.2%)` | :arrow_down: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `95.73% <0%> (-0.06%)` | :arrow_down: |
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `96.4% <0%> (-0.02%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.8% <0%> (-0.01%)` | :arrow_down: |
| [pandas/core/strings.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3RyaW5ncy5weQ==) | `98.46% <0%> (-0.01%)` | :arrow_down: |
| [pandas/core/internals.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzLnB5) | `94.47% <0%> (-0.01%)` | :arrow_down: |
| [pandas/core/panel.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcGFuZWwucHk=) | `97.14% <0%> (+0.28%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=footer). Last update [d101064...d3ff628](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=h1) Report
> Merging [#18489](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/d1010643fea058ba43c2c7124af75cc462ccf242?src=pr&el=desc) will **decrease** coverage by `50.56%`.
> The diff coverage is `28.57%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18489/graphs/tree.svg?width=650&src=pr&token=eZ4WkYLtcO&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree)

```diff
@@             Coverage Diff             @@
##           master   #18489       +/-   ##
===========================================
- Coverage   91.33%   40.77%   -50.57%     
===========================================
  Files         163      163               
  Lines       49801    49796        -5     
===========================================
- Hits        45487    20304    -25183     
- Misses       4314    29492    +25178
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `?` | |
| #single | `40.77% <28.57%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/tseries/offsets.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvb2Zmc2V0cy5weQ==) | `44.04% <28.57%> (-52.99%)` | :arrow_down: |
| [pandas/tools/hashing.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL3Rvb2xzL2hhc2hpbmcucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/sas/sas\_constants.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXNfY29uc3RhbnRzLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/formats/style.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvc3R5bGUucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/parser.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL3BhcnNlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/lib.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL2xpYi5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/json/json.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL2lvL2pzb24vanNvbi5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/types/concat.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL3R5cGVzL2NvbmNhdC5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/sas/sas7bdat.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXM3YmRhdC5weQ==) | `0% <0%> (-91.08%)` | :arrow_down: |
| [pandas/io/sas/sas\_xport.py](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXNfeHBvcnQucHk=) | `0% <0%> (-90.28%)` | :arrow_down: |
| ... and [112 more](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=footer). Last update [d101064...c503c53](https://codecov.io/gh/pandas-dev/pandas/pull/18489?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Separating the ccalendar stuff from the new apply_index and onOffset methods.  The newly-pushed commit should be an unambiguous win.

```
asv continuous -f 1.1 -E virtualenv master HEAD -b offset -b period -b timedelta -b timeseries
[...]
     [d1010643]       [d3ff6286]
+     27.5±0.09μs         36.5±2μs     1.33  offset.CBDayHolidays.time_custom_bday_cal_decr
+         499±1ms         612±20ms     1.23  offset.ApplyIndex.time_apply_series(<QuarterEnd: startingMonth=3>)
+     18.5±0.04μs       22.2±0.8μs     1.20  offset.Day.time_timeseries_day_apply
+         467±2ms          554±2ms     1.19  offset.SemiMonthOffset.time_end_apply_index
+      22.1±0.1μs       25.0±0.2μs     1.13  offset.SemiMonthOffset.time_end_decr_n
+      24.3±0.1μs       27.5±0.1μs     1.13  offset.CBDayHolidays.time_custom_bday_cal_incr_n
+         911±4ns      1.02±0.06μs     1.12  period.PeriodProperties.time_is_leap_year('M')
+     21.0±0.09μs         23.4±1μs     1.11  period.PeriodUnaryMethods.time_now('M')
+       107±0.2ns         118±10ns     1.10  timedelta.TimedeltaProperties.time_timedelta_seconds
-           1.73s            1.53s     0.89  timeseries.Iteration.time_iter_periodindex
-     3.93±0.04ms      3.47±0.02ms     0.88  period.PeriodIndexConstructor.time_from_pydatetime('D')
-     7.60±0.07μs      6.60±0.02μs     0.87  timeseries.DatetimeIndex.time_timestamp_tzinfo_cons
-           1.62s            1.41s     0.87  offset.ApplyIndex.time_apply_series(<BusinessYearEnd: month=12>)
-         530±9μs        459±0.6μs     0.87  period.Algorithms.time_value_counts('index')
-         536±7ms          458±3ms     0.85  offset.SemiMonthOffset.time_begin_apply_index
-      1.03±0.1μs          866±7ns     0.84  period.PeriodProperties.time_day('min')
-        133±10μs       111±0.05μs     0.83  timeseries.DatetimeIndex.time_unique
-     1.05±0.04μs          873±1ns     0.83  period.PeriodProperties.time_dayofyear('min')
-      20.1±0.3μs      16.4±0.09μs     0.82  timeseries.AsOf.time_asof_single_early
-      4.39±0.1ms      3.31±0.01ms     0.75  timeseries.ToDatetime.time_cache_true_with_unique_seconds_and_unit
-           24.0s            14.2s     0.59  gil.nogil_datetime_fields.time_datetime_to_period
-           1.46s       20.8±0.8ms     0.01  offset.ApplyIndex.time_apply_index(<BusinessMonthEnd>)
-           1.55s       20.9±0.1ms     0.01  offset.ApplyIndex.time_apply_series(<BusinessMonthBegin>)
-           1.61s       19.6±0.4ms     0.01  offset.ApplyIndex.time_apply_index(<BusinessMonthBegin>)
-           1.79s       21.6±0.2ms     0.01  offset.ApplyIndex.time_apply_series(<BusinessMonthEnd>)
-         668±2μs      6.93±0.02μs     0.01  offset.OnOffset.time_on_offset(<BusinessQuarterBegin: startingMonth=3>)
-         628±2μs      6.10±0.04μs     0.01  offset.OnOffset.time_on_offset(<QuarterBegin: startingMonth=3>)
```

Running again.  The 100x improvements are real.  The slowdowns I expect to be noise.
what path was BusinessMonthEnd for example taking before that made it a lot slower? all in python?
The base class applyindex raises, at which point it falls back to point wise addition
this needs a note for performance in whats for 0.22.0. ping when pushed as this is green already.
add an additional line (even though you have one for #18218). in general any PR that is perf related should get an entry (or you can include that PR number on an existing entry). cleaning/reorg PR's that don't touch the user don't need ones.
ping
thanks!