ENH: refactor cython extension, no need to duplicate everything.
ENH: fast path applies to the Cython extension too.