Separate out strptime.pyx from tslib
State: closed

This is the 2nd of an N part series of PRs to split `tslib` into independent modules.

At the moment there is a big chunk of code at the bottom of `tslib` that looks like it was pasted in from somewhere else.  The header for that section of the file reads `# Don't even ask`.  So I won't.

The new `tslibs.strptime` only used in one place: `array_strptime` is called in `tools.datetimes`.  Other than that, nothing needs to be exposed, and nothing else in `tslib` relies on it.

The one function from `tslib` that `strptime` _does_ need is `_check_dts_bounds`.  This (mostly) moves that up to datetime.pxd, which both `tslib` and `strptime` already import anyway.

This is _mostly_ a copy/paste of the existing functions+classes.  I cleaned up a couple of places where variables used camelCase.

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Internals
description: Related to non-user accessible pandas implementation
name: Clean

Comments:
Please don't make changes and move things in the same commit - it completely hides the changes when we're reviewing - thanks!
what is the rationale for moving this in the first place?
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=h1) Report
> Merging [#17342](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/473a7f3c186f6b0bfd9d3ce413fb627cf7a8f111?src=pr&el=desc) will **decrease** coverage by `0.04%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/17342/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #17342      +/-   ##
==========================================
- Coverage   91.03%   90.99%   -0.05%     
==========================================
  Files         162      162              
  Lines       49567    49567              
==========================================
- Hits        45125    45103      -22     
- Misses       4442     4464      +22
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.77% <ø> (-0.03%)` | :arrow_down: |
| #single | `40.24% <ø> (-0.07%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `63.23% <0%> (-1.82%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.72% <0%> (-0.1%)` | :arrow_down: |
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `95.23% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=footer). Last update [473a7f3...262da03](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=h1) Report
> Merging [#17342](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/d43aba82e218cd8187769a07c487709aa86de693?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/17342/graphs/tree.svg?width=650&height=150&src=pr&token=eZ4WkYLtcO)](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #17342      +/-   ##
==========================================
- Coverage   91.26%   91.24%   -0.02%     
==========================================
  Files         163      163              
  Lines       49776    49807      +31     
==========================================
+ Hits        45426    45447      +21     
- Misses       4350     4360      +10
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.04% <100%> (ø)` | :arrow_up: |
| #single | `40.3% <50%> (-0.04%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/\_libs/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree#diff-cGFuZGFzL19saWJzL19faW5pdF9fLnB5) | `100% <ø> (ø)` | :arrow_up: |
| [pandas/core/tools/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvdG9vbHMvZGF0ZXRpbWVzLnB5) | `85.24% <100%> (+0.04%)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/accessor.py](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYWNjZXNzb3IucHk=) | `93.65% <0%> (-6.35%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.73% <0%> (-0.15%)` | :arrow_down: |
| [pandas/core/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc2VyaWVzLnB5) | `94.93% <0%> (ø)` | :arrow_up: |
| [pandas/core/indexes/category.py](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9jYXRlZ29yeS5weQ==) | `97.74% <0%> (ø)` | :arrow_up: |
| [pandas/core/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY2F0ZWdvcmljYWwucHk=) | `95.28% <0%> (ø)` | :arrow_up: |
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.48% <0%> (+0.02%)` | :arrow_up: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `92.05% <0%> (+0.05%)` | :arrow_up: |
| ... and [2 more](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=footer). Last update [d43aba8...5478aa2](https://codecov.io/gh/pandas-dev/pandas/pull/17342?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

> what is the rationale for moving this in the first place?

In order of "most specific to this PR" to "most general to this sequence of PRs":

1) It is not immediately obvious that `strptime` is independent of the rest of `tslib`.  Separating it out makes both pieces easier to maintain.  DAGs FTW.

2) `tslib` is huge and gets imported in a whole bunch of places, often times just for `NaT`.  Making that import more light-weight is a win.

3) Over at dateutil we're talking about the next big push, and I'm advocating for making things more pandas-friendly.  e.g. see `tslib.dateutil_parse` docstring `""" lifted from dateutil to get resolution"""`.  Having dateutil return this resolution information would make this workaround unnecessary, improving performance and code complexity.
To that end, I'm trying to centralize the date parsing code that happens in a handful of places around pandas (`core.tools.datetimes`, `_libs.tslib`, `_libs/src/inference.pyx`) and separate it out into module(s) with minimal intra-pandas dependencies.

4) A big part of the goal for this series of PRs is to disentangle the dependencies between `tslib`, `period`, and `tseries`, as `tseries` is a big performance bottleneck (in particular `tseries.offsets.DateOffset`, see #17181 ).

5) See this comment in `_libs.__init__`: 

```
# TODO
# period is directly dependent on tslib and imports python
# modules, so exposing Period as an alias is currently not possible
```

One of the big goals of this sequence of PRs is handling this.


Just pushed a commit to remove cython "non-standard" cython decorators and revert a camelCase fix.  i.e. with the exception of a couple of flake8 whitespace changes, this should be a cut/paste of the existing functions.
this will need a rebase after #17422 
Hello @jbrockmendel! Thanks for updating the PR.





 - In the file [`setup.py`](https://github.com/pandas-dev/pandas/blob/5478aa265a18cf55e778d20f5d3d6621233f7943/setup.py), following are the PEP8 issues :

> [Line 487:80](https://github.com/pandas-dev/pandas/blob/5478aa265a18cf55e778d20f5d3d6621233f7943/setup.py#L487): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (84 > 79 characters)
> [Line 488:80](https://github.com/pandas-dev/pandas/blob/5478aa265a18cf55e778d20f5d3d6621233f7943/setup.py#L488): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (94 > 79 characters)



##### Comment last updated on September 25, 2017 at 00:13 Hours UTC
There is some other code related to parsing strings in tslib as well. Does it make sense to move those as well?
> There is some other code related to parsing strings in tslib as well. Does it make sense to move those as well?

Ah, I see there is already https://github.com/pandas-dev/pandas/pull/17363 :-)
needs a rebase
> checknull_with_nat, this seems like its repeating in lots of places

Ultimately both `checknull_with_nat` and `_nat_strings` belong in the not-yet-introduced `tslibs.nattype`.  First choice would be to live with this bit of redundancy until that is ready.  But putting them in util.pxd (or datetime.pxd) wouldn't be the end of the world.
thanks!