PERF: fixed performance degration becasue doesn't slice values array in to_native_types of DateTimeBlock
State: closed

- [x] closes #25708 
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry
Slicing in [line 2181](https://github.com/pandas-dev/pandas/blob/707c7201a744c48c835f719421e47480beee2184/pandas/core/internals/blocks.py#L2181) only slice of i8values that make a whole of values array is processed in _is_dates_only function. This change was from pull #24543
```python
        values = self.values
        i8values = self.values.view('i8')

        if slicer is not None:
            i8values = i8values[..., slicer]

        from pandas.io.formats.format import _get_format_datetime64_from_values
        fmt = _get_format_datetime64_from_values(values, date_format)
```

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25765?src=pr&el=h1) Report
> Merging [#25765](https://codecov.io/gh/pandas-dev/pandas/pull/25765?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/707c7201a744c48c835f719421e47480beee2184?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25765/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25765?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25765      +/-   ##
==========================================
+ Coverage   91.25%   91.25%   +<.01%     
==========================================
  Files         172      172              
  Lines       52977    52978       +1     
==========================================
+ Hits        48342    48344       +2     
+ Misses       4635     4634       -1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.82% <100%> (ø)` | :arrow_up: |
| #single | `41.74% <0%> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25765?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/internals/blocks.py](https://codecov.io/gh/pandas-dev/pandas/pull/25765/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzL2Jsb2Nrcy5weQ==) | `94.08% <100%> (ø)` | :arrow_up: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/25765/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `89.08% <0%> (+0.09%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25765?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25765?src=pr&el=footer). Last update [707c720...54502c0](https://codecov.io/gh/pandas-dev/pandas/pull/25765?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25765?src=pr&el=h1) Report
> Merging [#25765](https://codecov.io/gh/pandas-dev/pandas/pull/25765?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/c18c8be779240be49872860ddbfec9bc5ef0a548?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25765/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25765?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25765      +/-   ##
==========================================
- Coverage   91.99%   91.98%   -0.01%     
==========================================
  Files         175      175              
  Lines       52382    52383       +1     
==========================================
- Hits        48188    48184       -4     
- Misses       4194     4199       +5
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.54% <100%> (ø)` | :arrow_up: |
| #single | `40.74% <0%> (-0.14%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25765?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/internals/blocks.py](https://codecov.io/gh/pandas-dev/pandas/pull/25765/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzL2Jsb2Nrcy5weQ==) | `94.08% <100%> (ø)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/25765/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `75% <0%> (-12.5%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/25765/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.9% <0%> (-0.12%)` | :arrow_down: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/25765/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `90.61% <0%> (-0.11%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25765?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25765?src=pr&el=footer). Last update [c18c8be...d135af2](https://codecov.io/gh/pandas-dev/pandas/pull/25765?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

or if this is a perf fix, then show the updated asv's (and/or add one)
Hello @hksonngan! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-04-19 00:10:51 UTC
@jreback I add asv's test, test with my macair 2012.
```
· Running 2 total benchmarks (2 commits * 1 environments * 1 benchmarks)
[  0.00%] · For pandas commit db6993cd <master^2> (round 1/2):
[  0.00%] ·· Building for conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
[  0.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 25.00%] ··· Running (io.csv.ToCSVDatetimeBig.time_frame--).
[ 25.00%] · For pandas commit 4472dc9d <25708-fix-reregression-slicing> (round 1/2):
[ 25.00%] ·· Building for conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt...
[ 25.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 50.00%] ··· Running (io.csv.ToCSVDatetimeBig.time_frame--).
[ 50.00%] · For pandas commit 4472dc9d <25708-fix-reregression-slicing> (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 75.00%] ··· io.csv.ToCSVDatetimeBig.time_frame                                    912±50ms
[ 75.00%] · For pandas commit db6993cd <master^2> (round 2/2):
[ 75.00%] ·· Building for conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt...
[ 75.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[100.00%] ··· io.csv.ToCSVDatetimeBig.time_frame                                   1.28±0.2s
```
@jreback ping
@hksonngan I think you ran asv dev, you need to run asv continuous here it should the ratio of the change.
also pls add a whatsnew note. in performance improvements.
@jreback I think for small under 10.000obs is not gain to notice
This is with oservation > 1.000.000obs, we can see change so much. This is test with 1mil obs, I don't know why asv doesn't report ratio.
```
asv continuous -s -f 1.1 upstream/master 25708-fix-reregression-slicing -b ^io.csv.ToCSVDatetimeBig
· Creating environments
· Discovering benchmarks.
·· Uninstalling from conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
·· Installing 0df6b571 <25708-fix-reregression-slicing> into conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
· Running 2 total benchmarks (2 commits * 1 environments * 1 benchmarks)
[  0.00%] · For pandas commit db6993cd <master^2> (round 1/2):
[  0.00%] ·· Building for conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt......
[  0.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 25.00%] ··· Running (io.csv.ToCSVDatetimeBig.time_frame--).
[ 25.00%] · For pandas commit 0df6b571 <25708-fix-reregression-slicing> (round 1/2):
[ 25.00%] ·· Building for conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt.....
[ 25.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 50.00%] ··· Running (io.csv.ToCSVDatetimeBig.time_frame--).
[ 50.00%] · For pandas commit 0df6b571 <25708-fix-reregression-slicing> (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 75.00%] ··· io.csv.ToCSVDatetimeBig.time_frame                       16.6±0.2s
[ 75.00%] · For pandas commit db6993cd <master^2> (round 2/2):
[ 75.00%] ·· Building for conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt......
[ 75.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[100.00%] ··· io.csv.ToCSVDatetimeBig.time_frame                         22.4±1s

BENCHMARKS NOT SIGNIFICANTLY CHANGED.
```
can you merge master and update
@jreback Here is CProfile upstream/master and branch fix.
* Upstream/Master
![CProfile_master](https://user-images.githubusercontent.com/780912/55937678-6ff23880-5bee-11e9-8566-af30e0e540dc.png)
* Fix in branch
![CProfile_Patch](https://user-images.githubusercontent.com/780912/55937679-6ff23880-5bee-11e9-8e00-4a950cdb9208.png)

lgtm. ping on green.
@jreback ping.

'''
E:\dev\pandas\asv_bench>asv continuous -f 1.1 upstream/master HEAD -b ^io.csv.ToCSVDatetimeBig
· Creating environments
· Discovering benchmarks
·· Uninstalling from virtualenv-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytest-scipy-sqlalchemy-tables-xlrd-xlsxwriter-xlwt
·· Building 04afb4dd <25708-fix-reregression-slicing> for virtualenv-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytest-scipy-sqlalchemy-tables-xlrd-xlsxwriter-xlwt
·· Installing 04afb4dd <25708-fix-reregression-slicing> into virtualenv-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytest-scipy-sqlalchemy-tables-xlrd-xlsxwriter-xlwt
· Running 2 total benchmarks (2 commits * 1 environments * 1 benchmarks)
[  0.00%] · For pandas commit 7b3bf2dc <master-upstream> (round 1/2):
[  0.00%] ·· Building for virtualenv-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytest-scipy-sqlalchemy-tables-xlrd-xlsxwriter-xlwt
[  0.00%] ·· Benchmarking virtualenv-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytest-scipy-sqlalchemy-tables-xlrd-xlsxwriter-xlwt
[ 25.00%] ··· Running (io.csv.ToCSVDatetimeBig.time_frame--).
[ 25.00%] · For pandas commit 04afb4dd <25708-fix-reregression-slicing> (round 1/2):
[ 25.00%] ·· Building for virtualenv-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytest-scipy-sqlalchemy-tables-xlrd-xlsxwriter-xlwt
[ 25.00%] ·· Benchmarking virtualenv-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytest-scipy-sqlalchemy-tables-xlrd-xlsxwriter-xlwt
[ 50.00%] ··· Running (io.csv.ToCSVDatetimeBig.time_frame--).
[ 50.00%] · For pandas commit 04afb4dd <25708-fix-reregression-slicing> (round 2/2):
[ 50.00%] ·· Benchmarking virtualenv-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytest-scipy-sqlalchemy-tables-xlrd-xlsxwriter-xlwt
[ 75.00%] ··· io.csv.ToCSVDatetimeBig.time_frame                                                                                                                                ok
[ 75.00%] ··· ========== ============
                 obs
              ---------- ------------
                 1000      7.81±3ms
               1000000    4.73±0.01s
               10000000   49.1±0.4s
              ========== ============

[ 75.00%] · For pandas commit 7b3bf2dc <master-upstream> (round 2/2):
[ 75.00%] ·· Building for virtualenv-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytest-scipy-sqlalchemy-tables-xlrd-xlsxwriter-xlwt
[ 75.00%] ·· Benchmarking virtualenv-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytest-scipy-sqlalchemy-tables-xlrd-xlsxwriter-xlwt
[100.00%] ··· io.csv.ToCSVDatetimeBig.time_frame                                                                                                                                ok
[100.00%] ··· ========== =============
                 obs
              ---------- -------------
                 1000     0±8000000ns
               1000000     5.68±0.06s
               10000000    2.16±0.01m
              ========== =============

       before           after         ratio
     [7b3bf2dc]       [04afb4dd]
     <master-upstream>       <25708-fix-reregression-slicing>
-      2.16±0.01m        49.1±0.4s     0.38  io.csv.ToCSVDatetimeBig.time_frame(10000000)

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
'''
thanks @hksonngan 
Can someone check the perf for this benchmark: http://pandas.pydata.org/speed/pandas/index.html#series_methods.IsInForObjects.time_isin_long_series_long_values_floats?commits=f53bb061939fb54d6c826c4db408c458d3d2a1db

Not sure if real or a spurious regression. 
@TomAugspurger this is real, because of setting this params = [1000, 10000, 100000].

I don’t think so. The regression is for a different benchmark, no?