PERF: improve performance of NDFrame.describe
State: closed

A one-line change that enables to calculate the percentiles in `describe` more efficiently. The point is that calculating percentiles in one pass is faster than separately.

`describe` (with default `percentiles` argument) becomes 25-30% faster than before for numerical Series and DataFrames.

### Setup 
```
import timeit

setup = '''
import numpy as np
import pandas as pd
np.random.seed(123)
s = pd.Series(np.random.randint(0, 100, 1000000))
'''
```

### Benchmark
```
min(timeit.Timer('s.describe()', setup=setup).repeat(100, 1))
```

### Results
On master:
```
0.06349272100487724
```

With this change:
```
0.04745814300258644
```

Results are similar for DataFrames.

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Performance
description: Memory or execution speed performance

Comments:
Typically for performance-related changes we look for an ASV to measure and track over time. Can you add one to `asv_bench/benchmarks/frame_methods.py` and post the results of the benchmark here?
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21274?src=pr&el=h1) Report
> Merging [#21274](https://codecov.io/gh/pandas-dev/pandas/pull/21274?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/cbec58eacd8e9cd94b7f42351b8de4559c250909?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/21274/graphs/tree.svg?width=650&height=150&token=eZ4WkYLtcO&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/21274?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #21274      +/-   ##
==========================================
+ Coverage   91.85%   91.85%   +<.01%     
==========================================
  Files         153      153              
  Lines       49546    49549       +3     
==========================================
+ Hits        45509    45512       +3     
  Misses       4037     4037
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.25% <ø> (ø)` | :arrow_up: |
| #single | `41.87% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/21274?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/21274/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `96.12% <ø> (ø)` | :arrow_up: |
| [pandas/io/formats/csvs.py](https://codecov.io/gh/pandas-dev/pandas/pull/21274/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvY3N2cy5weQ==) | `98.14% <0%> (+0.01%)` | :arrow_up: |
| [pandas/core/indexes/interval.py](https://codecov.io/gh/pandas-dev/pandas/pull/21274/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9pbnRlcnZhbC5weQ==) | `93.16% <0%> (+0.02%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21274?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21274?src=pr&el=footer). Last update [cbec58e...6dda68e](https://codecov.io/gh/pandas-dev/pandas/pull/21274?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Sure. Thanks for the suggestion. Here are my ASV benchmarks. These also show the improvement.

### Setup

```
class Describe(object):

    goal_time = 0.2

    def setup(self):
        np.random.seed(123)
        self.df = DataFrame({
            'a': np.random.randint(0, 100, int(1e6)),
            'b': np.random.randint(0, 100, int(1e6)),
            'c': np.random.randint(0, 100, int(1e6)),
        })

    def time_series_describe(self):
        self.df['a'].describe()

    def time_dataframe_describe(self):
        self.df.describe()
```
### Results
|before|after|ratio|    |
|---|---|---|---|
|689±10ms|495±6ms|0.72|frame_methods.Describe.time_dataframe_describe|
|234±9ms|166±6ms|0.71|frame_methods.Describe.time_series_describe|


OK thanks. Can you update your PR to include the benchmark and a whatsnew note for 0.24?
Hello @DataOmbudsman! Thanks for updating the PR.





Cheers ! There are no PEP8 issues in this Pull Request. :beers: 

##### Comment last updated on June 05, 2018 at 12:40 Hours UTC
@DataOmbudsman Thanks!