BUG: Timestamp.replace chaining not compat with datetime.replace
State: closed

- [x] closes #15683
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

This is a clean version of https://github.com/pandas-dev/pandas/pull/16110 and the last thing I’m going to do with this issue.

Labels:
name: Bug
name: Timezones
description: Timezone data dtype

Comments:
@sscherfke : This look pretty good (test-wise) save for a minor linting issue, which you can see here:

https://travis-ci.org/pandas-dev/pandas/jobs/269100110#L1809
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=h1) Report
> Merging [#17356](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/473a7f3c186f6b0bfd9d3ce413fb627cf7a8f111?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/17356/graphs/tree.svg?width=650&src=pr&token=eZ4WkYLtcO&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #17356      +/-   ##
==========================================
- Coverage   91.03%   91.02%   -0.02%     
==========================================
  Files         162      162              
  Lines       49567    49567              
==========================================
- Hits        45125    45116       -9     
- Misses       4442     4451       +9
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.8% <ø> (ø)` | :arrow_up: |
| #single | `40.24% <ø> (-0.07%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.72% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=footer). Last update [473a7f3...02297a5](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=h1) Report
> Merging [#17356](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/473a7f3c186f6b0bfd9d3ce413fb627cf7a8f111?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/17356/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #17356      +/-   ##
==========================================
- Coverage   91.03%   91.02%   -0.02%     
==========================================
  Files         162      162              
  Lines       49567    49567              
==========================================
- Hits        45125    45116       -9     
- Misses       4442     4451       +9
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.8% <ø> (ø)` | :arrow_up: |
| #single | `40.24% <ø> (-0.07%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.72% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=footer). Last update [473a7f3...02297a5](https://codecov.io/gh/pandas-dev/pandas/pull/17356?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

My bad. Fixed it.
Can this PR *please* be merge? This function computes **wrong results** since more than half a year by now! 
From https://github.com/pandas-dev/pandas/pull/16110#discussion_r133924066

> `change the definition of tz_convert_single to cpdef int64_t`

Can you try that?
I gave this branch + changing `tz_convert_single` to a `cpdef int64_t` and ran through a small benchmark (with profiling enabled):

```python
from datetime import datetime
import pytz
from pandas import Timestamp
import cProfile
import pstats

dt = datetime(2016, 3, 27, 1)
tzinfo = pytz.timezone('CET').localize(dt, is_dst=False).tzinfo

result_dt = dt.replace(tzinfo=tzinfo)
ts = Timestamp(dt)

code = '''
for _ in range(10000):
    ts.replace(tzinfo=tzinfo)
'''

cProfile.runctx(code, globals(), locals(), "Profile.prof")

s = pstats.Stats("Profile.prof")
s.strip_dirs().sort_stats("time").print_stats()
```

Master:

```
Mon Sep  4 08:16:09 2017    Profile.prof

         150005 function calls in 0.104 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    10000    0.039    0.000    0.072    0.000 tslib.pyx:4210(tz_convert_single)
    10000    0.015    0.000    0.097    0.000 tslib.pyx:677(replace)
    20000    0.015    0.000    0.026    0.000 tslib.pyx:1708(_get_zone)
    20000    0.009    0.000    0.009    0.000 tslib.pyx:4280(_treat_tz_as_dateutil)
        1    0.008    0.008    0.105    0.105 <string>:2(<module>)
    10000    0.006    0.000    0.006    0.000 tslib.pyx:110(create_timestamp_from_ts)
    10000    0.004    0.000    0.005    0.000 tslib.pyx:4322(_get_dst_info)
    20000    0.003    0.000    0.003    0.000 tslib.pyx:1705(_is_utc)
    10000    0.001    0.000    0.001    0.000 tslib.pyx:236(_is_tzlocal)
    10000    0.001    0.000    0.001    0.000 tslib.pyx:4289(_tz_cache_key)
    10000    0.001    0.000    0.001    0.000 tslib.pyx:1043(__get__)
    10000    0.001    0.000    0.001    0.000 tslib.pyx:1770(_check_dts_bounds)
    10000    0.001    0.000    0.001    0.000 tslib.pyx:1044(__get__)
        1    0.000    0.000    0.105    0.105 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 tslib.pyx:4402(_unbox_utcoffsets)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.000    0.000    0.000    0.000 tslib.pyx:4276(_treat_tz_as_pytz)
```

PR:

```
Mon Sep  4 08:19:48 2017    Profile.prof

         210003 function calls in 0.149 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    30000    0.043    0.000    0.043    0.000 {method 'replace' of 'datetime.datetime' objects}
    10000    0.030    0.000    0.143    0.000 tslib.pyx:676(replace)
    10000    0.020    0.000    0.104    0.000 tslib.pyx:1439(convert_to_tsobject)
    10000    0.010    0.000    0.045    0.000 tzinfo.py:179(fromutc)
    10000    0.010    0.000    0.010    0.000 tslib.pyx:3966(_delta_to_nanoseconds)
        1    0.007    0.007    0.150    0.150 <string>:2(<module>)
    10000    0.007    0.000    0.067    0.000 tzinfo.py:189(normalize)
    10000    0.006    0.000    0.006    0.000 tslib.pyx:109(create_timestamp_from_ts)
    10000    0.004    0.000    0.004    0.000 {built-in method _bisect.bisect_right}
    10000    0.002    0.000    0.002    0.000 {built-in method builtins.max}
    10000    0.002    0.000    0.002    0.000 tslib.pyx:1745(maybe_get_tz)
    10000    0.001    0.000    0.001    0.000 tslib.pyx:1704(_is_utc)
    20000    0.001    0.000    0.001    0.000 tslib.pyx:1769(_check_dts_bounds)
    10000    0.001    0.000    0.001    0.000 datetime.pxd:144(_pydatetime_to_dts)
    10000    0.001    0.000    0.001    0.000 tslib.pyx:1432(_get_utcoffset)
    10000    0.001    0.000    0.001    0.000 tslib.pyx:1429(__get__)
    10000    0.001    0.000    0.001    0.000 tslib.pyx:1042(__get__)
    10000    0.001    0.000    0.001    0.000 tslib.pyx:1330(is_timestamp)
    10000    0.001    0.000    0.001    0.000 tslib.pyx:1043(__get__)
        1    0.000    0.000    0.150    0.150 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
```

I'm out of time now, but some quick thoughts:

- `datetime.datetime.replace` is now the most expensive function call
- We hit it 3 times per `.replace`, can we reduce that?
- We don't call `tz_convert_single` now (should re-run this with a non-naive one, but that's good)

replaced by #17507