Invalidate matrix and factorized inverse if the model changes.