use npy_datetimestruct and NPY_DATETIMEUNIT instead of pandas versions
State: closed

pandas implements `pandas_datetimestruct` and `PANDAS_DATETIMEUNIT`, AFAICT because at the time they were implemented, pandas was supporting versions of numpy that did not yet have `npy_datetimestruct` and `NPY_DATETIMEUNIT`.  The implementations are identical.

The upshot of using the numpy versions is dependency simplification.  There are a handful of functions/modules in `tslibs` (also `tslib`, `_libs.period`) that depend on `tslibs.np_datetime` and as a result need to have `tseries_depends` and `np_datetime_sources` in their extension specification.  In many cases, this is including much more than is necessary (and while I haven't timed it, I assume this impacts the build speed).

By using the numpy versions, we can import just datetimestruct and DATETIMEUNIT in the relevant places, avoiding dependencies on src/datetime/, lightening the build, and simplifying setup.py.

Note this PR doesn't actually _do_ any of those simplifications, is just the preliminary search+replace.

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:

Comments:
-1 on this. i would rather leave the numpy things isolated in a single place. you can certainly typedef things to use the same structure, though as this didn't appear to actually change anything in setup.py I am not sure this changes anything.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18403?src=pr&el=h1) Report
> Merging [#18403](https://codecov.io/gh/pandas-dev/pandas/pull/18403?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/509e03c6b113e78935ae46707572e4bf7f7a1df5?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18403/graphs/tree.svg?src=pr&token=eZ4WkYLtcO&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/18403?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18403      +/-   ##
==========================================
- Coverage   91.36%   91.34%   -0.02%     
==========================================
  Files         164      164              
  Lines       49730    49730              
==========================================
- Hits        45435    45426       -9     
- Misses       4295     4304       +9
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.14% <ø> (ø)` | :arrow_up: |
| #single | `39.62% <ø> (-0.07%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18403?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18403?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18403?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.8% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18403?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18403?src=pr&el=footer). Last update [509e03c...301cb4f](https://codecov.io/gh/pandas-dev/pandas/pull/18403?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

> though as this didn't appear to actually change anything in setup.py I am not sure this changes anything

Correct.  As mentioned, this PR is just the search+replace, does not implement the mentioned upsides.

> i would rather leave the numpy things isolated in a single place.

That's a reasonable preference.  The two big motivations I have in mind are:

- In trying to get `cythonize` working, the src directory is a PITA.  Having fewer things depend on it will make life better.

- I'm pretty sure that 'source'/'include'/'pxdfiles' keys affect build time.  I'll try to do some profiling and check this.  If that turns out to be wrong, I'll drop this.

> you can certainly typedef things to use the same structure

I poked at this but without much luck.  Is there a way to do this such that I could import npy_datetimestruct in a cython file and use it interchangeably with pandas_datetimestruct?  Or heck, import it `as pandas_datetimestruct`?  That would be just as effective as the changes in this PR, but with a much smaller footprint.
Got some profiling info on the build.  Let me know if there's a more appropriate place for this since its part of a larger issue.

Ran `python setup.py clean` followed by `python setup.py build_ext --inplace` N times, timed the latter.

A) Under master (as of yesterday morning):
N = 59
mean = 307.963 seconds
stdev  = 2.155 seconds

B) Branch in which I removed all the unnecessarily-specified files from `ext_data` in setup.py:
N = 59
mean = 305.954 seconds
stdev = 2.699 seconds

C) Branch in which I _added_ a bunch of unnecessary junk to `ext_data` in setup.py:
N = 10
mean = 323.757 seconds
stdev = 2.103 seconds

2 seconds cut between A and B isn't that big a deal (note `t=4.47`), but there wasn't much cruft to cut in the first place.  Trial C intended as a proxy for what might be available if dependencies were trimmed more thoroughly.  5% isn't world-breaking, but nothing to scoff at.
I am not enthused by this PR. other things to worry about, if in the future you want to come back to this, let's discuss in an issue.