Disable fit gradients when unnecessary in RMSD cvc