Remove numpy and cython deps during setup.py install. pyximport should do the trick