Instead of shuffling it is possible to keep the idx for later and make y = y[idx].
(Sorry to repeat, but you might not have seen it)