return residuals from fit