CLN: Refactor cython to use memory views
State: closed

Refactored cython files (.pxi.in, .pyx):

1. Made cdef usage consistent (using a cdef block instead of multiple cdef modifier when possible)
2. Replaced type hints with cython type declaration in function signatures
3. Replaced ndarray with memory view in function signatures
4. Added the const modifier to memory views in function signatures

@jreback I took your suggestion in a previous [PR](https://github.com/pandas-dev/pandas/pull/24795#issuecomment-454815511) and implemented on a broader scope.


Labels:
name: Internals
description: Related to non-user accessible pandas implementation
name: Clean

Comments:
@jbrockmendel 

> 2\. Replaced type hints with cython type declaration in function signatures

Might have this backwards but thought we were moving towards doing this, not away. See https://github.com/pandas-dev/pandas/pull/22283/
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24932?src=pr&el=h1) Report
> Merging [#24932](https://codecov.io/gh/pandas-dev/pandas/pull/24932?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/c8aae3540e4d22d2581e66843740fba10e9ea0b1?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24932/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24932?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #24932   +/-   ##
=======================================
  Coverage   92.38%   92.38%           
=======================================
  Files         166      166           
  Lines       52388    52388           
=======================================
  Hits        48397    48397           
  Misses       3991     3991
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.8% <ø> (ø)` | :arrow_up: |
| #single | `42.89% <ø> (ø)` | :arrow_up: |


------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24932?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24932?src=pr&el=footer). Last update [c8aae35...55e5dbc](https://codecov.io/gh/pandas-dev/pandas/pull/24932?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24932?src=pr&el=h1) Report
> Merging [#24932](https://codecov.io/gh/pandas-dev/pandas/pull/24932?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/2626215ca5dd168ea311f117b988693cef437885?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24932/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24932?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #24932   +/-   ##
=======================================
  Coverage   92.37%   92.37%           
=======================================
  Files         166      166           
  Lines       52388    52388           
=======================================
  Hits        48396    48396           
  Misses       3992     3992
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.8% <ø> (ø)` | :arrow_up: |
| #single | `42.9% <ø> (ø)` | :arrow_up: |


------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24932?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24932?src=pr&el=footer). Last update [2626215...4b5c8d9](https://codecov.io/gh/pandas-dev/pandas/pull/24932?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Yah, where viable we've been trying to use type hints.  The idea is to get as close to valid python as possible.

That aside, this looks really good (still reading)
Is type hints the preferred way for def functions only, or for cdef and cpdef as well?

AFAIK for cdef and cpdef, type hints can only be used for the parameters but not for the return type.
> AFAIK for cdef and cpdef, type hints can only be used for the parameters but not for the return type.

Yah this is a bit of a hassle.  I opened an issue over at cython a while back.

For cdef and cpdef functions, generally only cython-style annotations work.  For `def` functions go for the pure-python variant.

There are probably still some functions that are `cpdef` but never called from within cython.  These can be changed to just `def`
thanks @noamher