ENH: show percentiles in timestamp describe (#30164)
State: closed

closes https://github.com/pandas-dev/pandas/issues/30164

When creating a pandas Series of timestamps and calling `describe` on it, it will not show the percentiles of the data, even if these are specified as arguments (`s.describe(percentiles = [0.25, 0.5, 0.75])`).

This PR solves the issue by introducing a new describe logic for timestamps that would:
* Add the percentiles in the same way as for numeric.
* Rename `first` and `last` to `min` and `max` in order to match numeric types.
* Add the mean of the series.
* ~~Show datetime columns alongside numerics by default in the describe method for DataFrames.~~ (just realized there were other issues in which this was deemed to be the desirable behavior)

After this, it more or less matches the functionality of R's `summary` on `POSIXct` and `Date` types.

Why I think this is a good idea: I oftentimes find myself trying to inspect tables of mixed types, and want to get a quick overview of which kind of ranges and variations are there in each column, including timestamps. Currently this is very inconvenient in pandas, and they are not displayed alongside numeric columns by default and don't match the names if passing `include = "all"`, which is what I want to see when I call `DataFrame.describe`.

Examples:
```python
import numpy as np, pandas as pd
dt_list = [
    "2019-01-01 00:01:00",
    "2019-01-01 00:02:00",
    "2019-01-01 00:02:10",
    "2019-01-01 00:10:01",
    "2019-01-01 00:11:00"
]
s = pd.to_datetime(pd.Series(dt_list))
s.describe()
```
```python
pd.DataFrame({
    "col1" : s,
    "col2" : np.arange(s.shape[0])
}).describe(include = "all")
```

Update: corrected the coding style to comply with `black` and pass the automatic test.

Labels:
name: Enhancement
name: Datetime
description: Datetime data dtype
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
Hello @david-cortes! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-01-20 21:54:50 UTC
Added the issue number in the code, and also modified it to handle cases with empty inputs. Modified the tests on `describe` in `Series` and `DataFrame` too.

EDIT: updated yet again for code style checks.

EDIT2: updated again for alphabetical order of imports.
Hi @david-cortes - seems like one of the Azure jobs is failing (but the build isn't there anymore so we can't check it), and that there are now some conflicts with _master_. Is this something you're still working on?
@MarcoGorelli : No, not working on anything else here, the PR is finished. Solved the merge conflicts now (they were caused by some tests moving to separate files).

From what I get of the automatic checks that were failing, they were caused by something unrelated to this PR.
Yes, after the latest changes the internal datetime methods now work just fine with empty inputs and missing values. Changed the code to use those instead, and added the changes to the 1.1.0 what’s new entries. Will now open another one for moving the `describe` methods to a new file.

EDIT: corrected for linting now.
@jreback : Did you mean to say to move them to `pandas/core/algorithms.py`? Would seem out of place in `io`.
> @jreback : Did you mean to say to move them to `pandas/core/algorithms.py`? Would seem out of place in `io`.

yes though i’d like to create

pandas/core/algos/describe.py

and split up algorithms (so this can. e a step there)
Moved the changes in the what’s new and added tests for `Period` and `Timedelta`. The describe for `Timedelta` is already handled as a numeric dtype, and the describe for `Period` as object dtype, so those are not affected by this PR.
thanks @david-cortes 

if you woudn't mind parameterize / separating out the other dtypes tests would be great (could be combined with moving the code out from pandas/core/algorithms.py