This requires memory for the old matrix and the new matrix at once for a short amount of time.