API: Add CalendarDay ('CD') offset
State: closed

- [x] closes #22274
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Adds `CalendarDay` and `'CD'` offset and alias respectively and changes `Day` (`'D'`) in `date_range` to respect calendar day. 

With regards to `CalendarDay` added tests for:
- `CalendarDay` arithmetic with itself, datetimes, and timedeltas (which should raise)
- `'CD'` with use in `date_range` and `timedelta_range` (which should raise)
- resampling with `'CD'`

Also refactored `_generate_range ` for `date_range` since we no longer have to check for `Day`


Labels:
name: API Design
name: Frequency
description: DateOffsets

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22288?src=pr&el=h1) Report
> Merging [#22288](https://codecov.io/gh/pandas-dev/pandas/pull/22288?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/e2e1a1051576a48f210ce17272fc24b90ebcf24a?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `93.02%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/22288/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/22288?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #22288      +/-   ##
==========================================
+ Coverage   92.04%   92.05%   +<.01%     
==========================================
  Files         169      169              
  Lines       50782    50787       +5     
==========================================
+ Hits        46742    46751       +9     
+ Misses       4040     4036       -4
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.46% <93.02%> (ø)` | :arrow_up: |
| #single | `42.29% <76.74%> (-0.02%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/22288?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/period.py](https://codecov.io/gh/pandas-dev/pandas/pull/22288/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9wZXJpb2QucHk=) | `93.5% <ø> (ø)` | :arrow_up: |
| [pandas/core/indexes/interval.py](https://codecov.io/gh/pandas-dev/pandas/pull/22288/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9pbnRlcnZhbC5weQ==) | `94.16% <ø> (ø)` | :arrow_up: |
| [pandas/core/indexes/timedeltas.py](https://codecov.io/gh/pandas-dev/pandas/pull/22288/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy90aW1lZGVsdGFzLnB5) | `91.16% <ø> (ø)` | :arrow_up: |
| [pandas/core/arrays/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/22288/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2RhdGV0aW1lcy5weQ==) | `96.6% <100%> (+1.07%)` | :arrow_up: |
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/22288/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `95.65% <50%> (+0.1%)` | :arrow_up: |
| [pandas/tseries/offsets.py](https://codecov.io/gh/pandas-dev/pandas/pull/22288/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvb2Zmc2V0cy5weQ==) | `97.03% <90.9%> (-0.12%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22288?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22288?src=pr&el=footer). Last update [e2e1a10...9ed681c](https://codecov.io/gh/pandas-dev/pandas/pull/22288?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Should be ready for a look @jbrockmendel 
Looks like I am running into a Windows/platform specific bug:

On OSX
```
In [3]: ts = date_range('1/1/2000', '4/1/2000', tz=tzlocal())

In [4]: ts.to_period()
Out[4]:
PeriodIndex(['2000-01-01', '2000-01-02', '2000-01-03', '2000-01-04',
             '2000-01-05', '2000-01-06', '2000-01-07', '2000-01-08',
             '2000-01-09', '2000-01-10', '2000-01-11', '2000-01-12',
             '2000-01-13', '2000-01-14', '2000-01-15', '2000-01-16',
             '2000-01-17', '2000-01-18', '2000-01-19', '2000-01-20',
             '2000-01-21', '2000-01-22', '2000-01-23', '2000-01-24',
             '2000-01-25', '2000-01-26', '2000-01-27', '2000-01-28',
             '2000-01-29', '2000-01-30', '2000-01-31', '2000-02-01',
             '2000-02-02', '2000-02-03', '2000-02-04', '2000-02-05',
             '2000-02-06', '2000-02-07', '2000-02-08', '2000-02-09',
             '2000-02-10', '2000-02-11', '2000-02-12', '2000-02-13',
             '2000-02-14', '2000-02-15', '2000-02-16', '2000-02-17',
             '2000-02-18', '2000-02-19', '2000-02-20', '2000-02-21',
             '2000-02-22', '2000-02-23', '2000-02-24', '2000-02-25',
             '2000-02-26', '2000-02-27', '2000-02-28', '2000-02-29',
             '2000-03-01', '2000-03-02', '2000-03-03', '2000-03-04',
             '2000-03-05', '2000-03-06', '2000-03-07', '2000-03-08',
             '2000-03-09', '2000-03-10', '2000-03-11', '2000-03-12',
             '2000-03-13', '2000-03-14', '2000-03-15', '2000-03-16',
             '2000-03-17', '2000-03-18', '2000-03-19', '2000-03-20',
             '2000-03-21', '2000-03-22', '2000-03-23', '2000-03-24',
             '2000-03-25', '2000-03-26', '2000-03-27', '2000-03-28',
             '2000-03-29', '2000-03-30', '2000-03-31', '2000-04-01'],
            dtype='period[D]', freq='D')
```

But on the AppVeyor test (Win32)

```
PeriodIndex(['2000-01-01', '2000-01-02', '2000-01-03', '2000-01-04',
                    '2000-01-05', '2000-01-06', '2000-01-07', '2000-01-08',
                    '2000-01-09', '2000-01-10', '2000-01-11', '2000-01-12',
                    '2000-01-13', '2000-01-14', '2000-01-15', '2000-01-16',
                    '2000-01-17', '2000-01-18', '2000-01-19', '2000-01-20',
                    '2000-01-21', '2000-01-22', '2000-01-23', '2000-01-24',
                    '2000-01-25', '2000-01-26', '2000-01-27', '2000-01-28',
                    '2000-01-29', '2000-01-30', '2000-01-31', '2000-02-01',
                    '2000-02-02', '2000-02-03', '2000-02-04', '2000-02-05',
                    '2000-02-06', '2000-02-07', '2000-02-08', '2000-02-09',
                    '2000-02-10', '2000-02-11', '2000-02-12', '2000-02-13',
                    '2000-02-14', '2000-02-15', '2000-02-16', '2000-02-17',
                    '2000-02-18', '2000-02-19', '2000-02-20', '2000-02-21',
                    '2000-02-22', '2000-02-23', '2000-02-24', '2000-02-25',
                    '2000-02-26', '2000-02-27', '2000-02-28', '2000-02-29',
                    '2000-03-01', '2000-03-02', '2000-03-03', '2000-03-04',
                    '2000-03-05', '2000-03-06', '2000-03-07', '2000-03-08',
                    '2000-03-09', '2000-03-10', '2000-03-11', '2000-03-12',
                    '2000-03-13', '2000-03-14', '2000-03-15', '2000-03-16',
                    '2000-03-17', '2000-03-18', '2000-03-19', '2000-03-20',
                    '2000-03-21', '2000-03-22', '2000-03-23', '2000-03-24',
                    '2000-03-25', '2000-03-26', '2000-03-27', '2000-03-28',
                    '2000-03-29', '2000-03-30', '2000-03-31'],
                   dtype='period[D]', freq='D')
```

Unfortunately I can't reproduce this locally. Guessing it may due to either:

1) `tzlocal`
2) Some calculation is Windows 32 bit specific.
This may be a hassle to check, but does `ts` match before the `to_period` call?
Hmm not sure I quite understand @jbrockmendel. Check that `ts` matches what before the `to_period` call? 
> Check that ts matches what before the to_period call?

The issue is that `ts.to_period('D')` on Appveyor does not `ts.to_period('D')` locally right?  So my question was if `ts` on Appveyor matches `ts` locally.
Okay so this looks like a `tzlocal` discrepancy:

the `date_range` with `tzlocal` on Appveyor passes a dst transition:

```
DatetimeIndex(['2000-01-01 00:00:00-08:00', '2000-01-02 00:00:00-08:00',
E                          '2000-01-03 00:00:00-08:00', '2000-01-04 00:00:00-08:00',
E                          '2000-01-05 00:00:00-08:00', '2000-01-06 00:00:00-08:00',
E                          '2000-01-07 00:00:00-08:00', '2000-01-08 00:00:00-08:00',
E                          '2000-01-09 00:00:00-08:00', '2000-01-10 00:00:00-08:00',
E                          '2000-01-11 00:00:00-08:00', '2000-01-12 00:00:00-08:00',
E                          '2000-01-13 00:00:00-08:00', '2000-01-14 00:00:00-08:00',
E                          '2000-01-15 00:00:00-08:00', '2000-01-16 00:00:00-08:00',
E                          '2000-01-17 00:00:00-08:00', '2000-01-18 00:00:00-08:00',
E                          '2000-01-19 00:00:00-08:00', '2000-01-20 00:00:00-08:00',
E                          '2000-01-21 00:00:00-08:00', '2000-01-22 00:00:00-08:00',
E                          '2000-01-23 00:00:00-08:00', '2000-01-24 00:00:00-08:00',
E                          '2000-01-25 00:00:00-08:00', '2000-01-26 00:00:00-08:00',
E                          '2000-01-27 00:00:00-08:00', '2000-01-28 00:00:00-08:00',
E                          '2000-01-29 00:00:00-08:00', '2000-01-30 00:00:00-08:00',
E                          '2000-01-31 00:00:00-08:00', '2000-02-01 00:00:00-08:00',
E                          '2000-02-02 00:00:00-08:00', '2000-02-03 00:00:00-08:00',
E                          '2000-02-04 00:00:00-08:00', '2000-02-05 00:00:00-08:00',
E                          '2000-02-06 00:00:00-08:00', '2000-02-07 00:00:00-08:00',
E                          '2000-02-08 00:00:00-08:00', '2000-02-09 00:00:00-08:00',
E                          '2000-02-10 00:00:00-08:00', '2000-02-11 00:00:00-08:00',
E                          '2000-02-12 00:00:00-08:00', '2000-02-13 00:00:00-08:00',
E                          '2000-02-14 00:00:00-08:00', '2000-02-15 00:00:00-08:00',
E                          '2000-02-16 00:00:00-08:00', '2000-02-17 00:00:00-08:00',
E                          '2000-02-18 00:00:00-08:00', '2000-02-19 00:00:00-08:00',
E                          '2000-02-20 00:00:00-08:00', '2000-02-21 00:00:00-08:00',
E                          '2000-02-22 00:00:00-08:00', '2000-02-23 00:00:00-08:00',
E                          '2000-02-24 00:00:00-08:00', '2000-02-25 00:00:00-08:00',
E                          '2000-02-26 00:00:00-08:00', '2000-02-27 00:00:00-08:00',
E                          '2000-02-28 00:00:00-08:00', '2000-02-29 00:00:00-08:00',
E                          '2000-03-01 00:00:00-08:00', '2000-03-02 00:00:00-08:00',
E                          '2000-03-03 00:00:00-08:00', '2000-03-04 00:00:00-08:00',
E                          '2000-03-05 00:00:00-08:00', '2000-03-06 00:00:00-08:00',
E                          '2000-03-07 00:00:00-08:00', '2000-03-08 00:00:00-08:00',
E                          '2000-03-09 00:00:00-08:00', '2000-03-10 00:00:00-08:00',
E                          '2000-03-11 00:00:00-08:00', '2000-03-12 01:00:00-07:00',
E                          '2000-03-13 01:00:00-07:00', '2000-03-14 01:00:00-07:00',
E                          '2000-03-15 01:00:00-07:00', '2000-03-16 01:00:00-07:00',
E                          '2000-03-17 01:00:00-07:00', '2000-03-18 01:00:00-07:00',
E                          '2000-03-19 01:00:00-07:00', '2000-03-20 01:00:00-07:00',
E                          '2000-03-21 01:00:00-07:00', '2000-03-22 01:00:00-07:00',
E                          '2000-03-23 01:00:00-07:00', '2000-03-24 01:00:00-07:00',
E                          '2000-03-25 01:00:00-07:00', '2000-03-26 01:00:00-07:00',
E                          '2000-03-27 01:00:00-07:00', '2000-03-28 01:00:00-07:00',
E                          '2000-03-29 01:00:00-07:00', '2000-03-30 01:00:00-07:00',
E                          '2000-03-31 01:00:00-07:00'],
E                         dtype='datetime64[ns, tzlocal()]', freq='D')
pandas\tests\indexes\datetimes\test_astype.py:268: ValueError
```

So the `expected` value will be different for  `tzlocal` compared to the other timezones tested.
> Okay so this looks like a tzlocal discrepancy:

OK next question: what timezone is the appveyor instance in?  what timezone is the local machine in?  If they aren't the same, then that explains why tzlocal would be different.
My local is `US/Pacific`, and it looks like AppVeyor is in a timezone where there's a daylight savings transition on March 11, 2000.

However, I think test in particular is mainly trying to test if `to_period` works for different timezone types (pytz and dateutil) and not necessarily concerned with dst transitions, so I ended up shortening the generated date_range to not hit that transition.
1) It looks like `DateOffset` nor any of the subclasses exists in `api.rst`. I'll make an issue to include them (including `CalendarOffset`)

2) RE dst_tz_fitxure: I think the existing `tz_aware_fixture` should include `tzlocal` and others, but I suspect it may break some existing tests, so I think that'll be a good follow up.
ping @jbrockmendel @chris-b1 
Thanks for the feedback @jbrockmendel @chris-b1 

All green minus an unrelated build failure seen in https://github.com/pandas-dev/pandas/pull/22499
@jreback all green.
@jreback Addressed your comments and all green. 
Hello @mroeschke! Thanks for updating the PR.

 - There are no PEP8 issues in the file [`pandas/core/arrays/datetimes.py`](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/core/arrays/datetimes.py) !

 - In the file [`pandas/core/indexes/datetimes.py`](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/core/indexes/datetimes.py), following                are the PEP8 issues :
> [Line 389:17](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/core/indexes/datetimes.py#L389): [W504](https://duckduckgo.com/?q=pep8%20W504) line break after binary operator
> [Line 390:17](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/core/indexes/datetimes.py#L390): [W504](https://duckduckgo.com/?q=pep8%20W504) line break after binary operator


 - There are no PEP8 issues in the file [`pandas/core/indexes/interval.py`](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/core/indexes/interval.py) !

 - There are no PEP8 issues in the file [`pandas/core/indexes/period.py`](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/core/indexes/period.py) !

 - There are no PEP8 issues in the file [`pandas/core/indexes/timedeltas.py`](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/core/indexes/timedeltas.py) !

 - There are no PEP8 issues in the file [`pandas/tests/indexes/datetimes/test_astype.py`](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/tests/indexes/datetimes/test_astype.py) !

 - There are no PEP8 issues in the file [`pandas/tests/indexes/datetimes/test_date_range.py`](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/tests/indexes/datetimes/test_date_range.py) !

 - There are no PEP8 issues in the file [`pandas/tests/indexes/datetimes/test_timezones.py`](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/tests/indexes/datetimes/test_timezones.py) !

 - There are no PEP8 issues in the file [`pandas/tests/indexes/timedeltas/test_timedelta_range.py`](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/tests/indexes/timedeltas/test_timedelta_range.py) !

 - There are no PEP8 issues in the file [`pandas/tests/series/test_timezones.py`](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/tests/series/test_timezones.py) !

 - There are no PEP8 issues in the file [`pandas/tests/test_resample.py`](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/tests/test_resample.py) !

 - There are no PEP8 issues in the file [`pandas/tests/tseries/offsets/test_offsets.py`](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/tests/tseries/offsets/test_offsets.py) !

 - There are no PEP8 issues in the file [`pandas/tests/tseries/offsets/test_ticks.py`](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/tests/tseries/offsets/test_ticks.py) !

 - There are no PEP8 issues in the file [`pandas/tseries/offsets.py`](https://github.com/mroeschke/pandas/blob/9ed681c9bba45ac2f939e85951127de569ae28c7/pandas/tseries/offsets.py) !



##### Comment last updated on September 05, 2018 at 16:52 Hours UTC
@jreback made the requested changes and all green
thanks! @mroeschke nice addition
Sorry that I did not comment earlier (also didn't read all review comments), but I think this is a much bigger break in behaviour than we think / is documented. 

This completely changes the default behaviour of `date_range` when using tz aware dates:

Now on master:
```
In [4]: pd.date_range("2016-10-30", periods=4, tz='Europe/Helsinki')
Out[4]: 
DatetimeIndex(['2016-10-30 00:00:00+03:00', '2016-10-30 23:00:00+02:00',
               '2016-10-31 23:00:00+02:00', '2016-11-01 23:00:00+02:00'],
              dtype='datetime64[ns, Europe/Helsinki]', freq='D')
```

Previously:

```
In [6]: pd.date_range("2016-10-30", periods=4, tz='Europe/Helsinki')
Out[6]: 
DatetimeIndex(['2016-10-30 00:00:00+03:00', '2016-10-31 00:00:00+02:00',
               '2016-11-01 00:00:00+02:00', '2016-11-02 00:00:00+02:00'],
              dtype='datetime64[ns, Europe/Helsinki]', freq='D')
```

This also changed the inferring of freq (now '24H' instead of 'D'), changes the behaviour of `resample('D')`, ...


(I would personally revert this PR until we have better discussed it)
I left a https://github.com/pandas-dev/pandas/issues/22274#issuecomment-419549602 with the rational.


In terms of documentation, there's a section in the whatsnew v0.24 that notes this API change:

https://github.com/pandas-dev/pandas/blob/70c9003c9e67c34fb31d4b6be8d8b970dfacbd38/doc/source/whatsnew/v0.24.0.txt#L288-L326

And new reference in `timeseries.rst` that discusses the difference between `Day` and `CalendarDay`

https://github.com/pandas-dev/pandas/blob/master/doc/source/timeseries.rst#day-vs-calendarday
Yeah, there were certainly nice new docs about the CalendarDay, but the illustration of the changed behaviour was only for, IMO, a minor use case of arithmetic with offset objects, while the more relevant user-facing changes as date_range and resample were not mentioned.
That's a fair concern; I can certainly give more attention to this impact on `date_range` and `resample` in the whatsnew and the CalendarDay section in `timeseries.rst`