Make khash its own extension
State: closed

The path towards cythonize continues.

Labels:
name: Clean

Comments:
this is a header only, I don't think this is necessary. I was referring to skiplist becoming its own module, khash is fine as is.
> I don't think this is necessary. I was referring to skiplist becoming its own module, khash is fine as is.

As discussed, it is necessary for cythonize.  In attempts to date, cythonize chokes on khash, skiplist, datetime, and util.
BTW skiplist push is coming before long, but I'm testing locally before pushing.
well, then you should edit everything to remove the direct imports from khash.h and put the declarations in .pxd.
> well, then you should edit everything to remove the direct imports from khash.h and put the declarations in .pxd.

OK.  I'm not clear on what you're asking for.  The only things that have "direct imports from khash.h" (based on a quick grep) are "src/klib/khash_python.h" and "src/parser/tokenizer.h".  The only place that has directly imports from khash_python.h is khash.pxd.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=h1) Report
> Merging [#18472](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/aaee541b538559f8887881ab23d2734dddd920d3?src=pr&el=desc) will **decrease** coverage by `0.03%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18472/graphs/tree.svg?height=150&width=650&token=eZ4WkYLtcO&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18472      +/-   ##
==========================================
- Coverage   91.36%   91.32%   -0.04%     
==========================================
  Files         163      163              
  Lines       49700    49717      +17     
==========================================
- Hits        45407    45404       -3     
- Misses       4293     4313      +20
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.12% <ø> (-0.03%)` | :arrow_down: |
| #single | `40.51% <ø> (+0.79%)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/io/clipboard/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree#diff-cGFuZGFzL2lvL2NsaXBib2FyZC9fX2luaXRfXy5weQ==) | `56.86% <0%> (-19.33%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.8% <0%> (-0.1%)` | :arrow_down: |
| [pandas/io/clipboard/clipboards.py](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree#diff-cGFuZGFzL2lvL2NsaXBib2FyZC9jbGlwYm9hcmRzLnB5) | `32.18% <0%> (+8.13%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=footer). Last update [aaee541...6741d4c](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=h1) Report
> Merging [#18472](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/aaee541b538559f8887881ab23d2734dddd920d3?src=pr&el=desc) will **decrease** coverage by `0.02%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18472/graphs/tree.svg?src=pr&width=650&token=eZ4WkYLtcO&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18472      +/-   ##
==========================================
- Coverage   91.36%   91.33%   -0.03%     
==========================================
  Files         163      164       +1     
  Lines       49700    49819     +119     
==========================================
+ Hits        45407    45503      +96     
- Misses       4293     4316      +23
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.13% <ø> (-0.01%)` | :arrow_down: |
| #single | `40.8% <ø> (+1.07%)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/io/clipboard/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree#diff-cGFuZGFzL2lvL2NsaXBib2FyZC9fX2luaXRfXy5weQ==) | `56.86% <0%> (-19.33%)` | :arrow_down: |
| [pandas/core/dtypes/missing.py](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL21pc3NpbmcucHk=) | `90.24% <0%> (-0.5%)` | :arrow_down: |
| [pandas/core/panel.py](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcGFuZWwucHk=) | `96.85% <0%> (-0.29%)` | :arrow_down: |
| [pandas/core/indexes/datetimelike.py](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZWxpa2UucHk=) | `96.91% <0%> (-0.2%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.81% <0%> (-0.09%)` | :arrow_down: |
| [pandas/core/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc2VyaWVzLnB5) | `94.76% <0%> (-0.08%)` | :arrow_down: |
| [pandas/core/dtypes/cast.py](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Nhc3QucHk=) | `88.52% <0%> (-0.08%)` | :arrow_down: |
| [pandas/tseries/offsets.py](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvb2Zmc2V0cy5weQ==) | `96.94% <0%> (-0.07%)` | :arrow_down: |
| [pandas/core/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYmFzZS5weQ==) | `96.54% <0%> (-0.03%)` | :arrow_down: |
| ... and [13 more](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=footer). Last update [aaee541...b584985](https://codecov.io/gh/pandas-dev/pandas/pull/18472?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

One down, thee to go. Let’s knock this out this week.
i don't like the dummy file here. 
does this help with cythonize?
> does this help with cythonize?

Good question.  With the pyx file it was a "yes".  I'll have to double-check without.
Yes, this helps with cythonize.  i.e. the setup that I got working with cythonize also works without the dummy .pyx file.

FYI, the steps to get cythonize working (not necessarily the only way, just the way that I found):
- Move khash, skiplist, datetime, util up out of src to fix `ImportError`s.  (update tslibs cimports to relative cimports)
- Fix `issubclass` problem generated in inference.pyx caused by mismatch between cython's numpy pxd files and src/numpy.pxd.(https://github.com/numpy/numpy/issues/10063#issuecomment-346741067)
- Add `cythonize` in setup.py