It would be nice if we could avoid this, but `ufunc.at` isn't available until numpy 1.8, so this will have to do.