Remove numpy header file dependency at cost of slight slowdown