Make sure not to import numba before building it