The normalization in the A matrix would affect the residue of the linear constraint. So we should normalize the constraint first