Residual sum of squares - wikipedia Residual sum of squares This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. ( April 2013 ) In statistics, the residual sum of squares ( RSS ), also known as the sum of squared residuals ( SSR ) or the sum of squared errors of prediction ( SSE ), is the sum of the squares of residuals ( deviations predicted from actual empirical values of data ). It is a measure of the discrepancy between the data and an estimation model. A small RSS indicates a tight fit of the model to the data. It is used as an optimality criterion in parameter selection and model selection. In general, total sum of squares = explained sum of squares + residual sum of squares. For a proof of this in the multivariate ordinary least squares ( OLS ) case, see partitioning in the general OLS model. Contents 1 One explanatory variable 2 Matrix expression for the OLS residual sum of squares 3 Relation with Pearson 's product - moment correlation 4 See also 5 References One explanatory variable In a model with a single explanatory variable, RSS is given by : R S S = ∑ i = 1 n ( y i − f ( x i ) ) 2 ( displaystyle RSS = sum _ ( i = 1 ) ^ ( n ) ( y_ ( i ) - f ( x_ ( i ) ) ) ^ ( 2 ) ) where y is the i value of the variable to be predicted, x is the i value of the explanatory variable, and f ( x i ) ( displaystyle f ( x_ ( i ) ) ) is the predicted value of y ( also termed y i ^ ( displaystyle ( hat ( y_ ( i ) ) ) ) ). In a standard linear simple regression model, y i = a + b x i + ε i ( displaystyle y_ ( i ) = a + bx_ ( i ) + \varepsilon _ ( i ) , ), where a and b are coefficients, y and x are the regressand and the regressor, respectively, and ε is the error term. The sum of squares of residuals is the sum of squares of estimates of ε ; that is R S S = ∑ i = 1 n ( ε i ) 2 = ∑ i = 1 n ( y i − ( α + β x i ) ) 2 ( displaystyle RSS = sum _ ( i = 1 ) ^ ( n ) ( \varepsilon _ ( i ) ) ^ ( 2 ) = sum _ ( i = 1 ) ^ ( n ) ( y_ ( i ) - ( alpha + \beta x_ ( i ) ) ) ^ ( 2 ) ) where α ( displaystyle alpha ) is the estimated value of the constant term a ( displaystyle a ) and β ( displaystyle \beta ) is the estimated value of the slope coefficient b. Matrix expression for the OLS residual sum of squares The general regression model with ' n ' observations and ' k ' explanators, the first of which is a constant unit vector whose coefficient is the regression intercept, is y = X β + e ( displaystyle y = X \beta + e ) where y is an n × 1 vector of dependent variable observations, each column of the n × k matrix X is a vector of observations on one of the k explanators, β ( displaystyle \beta ) is a k × 1 vector of true coefficients, and e is an n × 1 vector of the true underlying errors. The ordinary least squares estimator for β ( displaystyle \beta ) is X β ^ = y ⟺ ( displaystyle X ( hat ( \beta ) ) = y iff ) X T X β ^ = X T y ⟺ ( displaystyle X ^ ( T ) X ( hat ( \beta ) ) = X ^ ( T ) y iff ) β ^ = ( X T X ) − 1 X T y. ( displaystyle ( hat ( \beta ) ) = ( X ^ ( T ) X ) ^ ( - 1 ) X ^ ( T ) y. ) The residual vector e ^ ( displaystyle ( hat ( e ) ) ) = y − X β ^ = y − X ( X T X ) − 1 X T y ( displaystyle y-X ( hat ( \beta ) ) = y-X ( X ^ ( T ) X ) ^ ( - 1 ) X ^ ( T ) y ), so the residual sum of squares is : R S S = e ^ T e ^ = ‖ e ‖ 2 ( displaystyle RSS = ( hat ( e ) ) ^ ( T ) ( hat ( e ) ) = e ^ ( 2 ) ), ( equivalent to the square - root of the norm of residuals ) ; in full : R S S = y T y − y T X ( X T X ) − 1 X T y = y T ( I − X ( X T X ) − 1 X T ) y = y T ( I − H ) y ( displaystyle RSS = y ^ ( T ) y-y ^ ( T ) X ( X ^ ( T ) X ) ^ ( - 1 ) X ^ ( T ) y = y ^ ( T ) ( I-X ( X ^ ( T ) X ) ^ ( - 1 ) X ^ ( T ) ) y = y ^ ( T ) ( I-H ) y ), where H is the hat matrix, or the projection matrix in linear regression. Relation with Pearson 's product - moment correlation The least - squares regression line is given by y = a x + b ( displaystyle y = ax + b ), where b = y  ̄ − a x  ̄ ( displaystyle b = ( \bar ( y ) ) - a ( \bar ( x ) ) ) and a = S x y S x x ( displaystyle a = ( \frac ( S_ ( xy ) ) ( S_ ( xx ) ) ) ), where S x y = ∑ i = 1 n ( x  ̄ − x i ) ( y  ̄ − y i ) ( displaystyle S_ ( xy ) = sum _ ( i = 1 ) ^ ( n ) ( ( \bar ( x ) ) - x_ ( i ) ) ( ( \bar ( y ) ) - y_ ( i ) ) ) and S x x = ∑ i = 1 n ( x  ̄ − x i ) 2. ( displaystyle S_ ( xx ) = sum _ ( i = 1 ) ^ ( n ) ( ( \bar ( x ) ) - x_ ( i ) ) ^ ( 2 ). ) Therefore, R S S = ∑ i = 1 n ( y i − f ( x i ) ) 2 = ∑ i = 1 n ( y i − ( a x i + b ) ) 2 = ∑ i = 1 n ( y i − a x i − y  ̄ + a x  ̄ ) 2 ( displaystyle RSS = sum _ ( i = 1 ) ^ ( n ) ( y_ ( i ) - f ( x_ ( i ) ) ) ^ ( 2 ) = sum _ ( i = 1 ) ^ ( n ) ( y_ ( i ) - ( ax_ ( i ) + b ) ) ^ ( 2 ) = sum _ ( i = 1 ) ^ ( n ) ( y_ ( i ) - ax_ ( i ) - ( \bar ( y ) ) + a ( \bar ( x ) ) ) ^ ( 2 ) ) = ∑ i = 1 n ( a ( x  ̄ − x i ) − ( y  ̄ − y i ) ) 2 = a 2 S x x − 2 a S x y + S y y = S y y − a S x y = S y y ( 1 − S x y 2 S x x S y y ) ( displaystyle = sum _ ( i = 1 ) ^ ( n ) ( a ( ( \bar ( x ) ) - x_ ( i ) ) - ( ( \bar ( y ) ) - y_ ( i ) ) ) ^ ( 2 ) = a ^ ( 2 ) S_ ( xx ) - 2aS_ ( xy ) + S_ ( yy ) = S_ ( yy ) - aS_ ( xy ) = S_ ( yy ) ( 1 - ( \frac ( S_ ( xy ) ^ ( 2 ) ) ( S_ ( xx ) S_ ( yy ) ) ) ) ) where S y y = ∑ i = 1 n ( y  ̄ − y i ) 2. ( displaystyle S_ ( yy ) = sum _ ( i = 1 ) ^ ( n ) ( ( \bar ( y ) ) - y_ ( i ) ) ^ ( 2 ). ) The Pearson product - moment correlation is given by r = S x y S x x S y y ; ( displaystyle r = ( \frac ( S_ ( xy ) ) ( sqrt ( S_ ( xx ) S_ ( yy ) ) ) ) ; ) therefore, R S S = S y y ( 1 − r 2 ). ( displaystyle RSS = S_ ( yy ) ( 1 - r ^ ( 2 ) ). ) See also Sum of squares ( statistics ) Squared deviations Errors and residuals in statistics Lack - of - fit sum of squares Degrees of freedom ( statistics ) # Sum of squares and degrees of freedom Chi - squared distribution # Applications References Draper, N.R. ; Smith, H. ( 1998 ). Applied Regression Analysis ( 3rd ed. ). John Wiley. ISBN 0 - 471 - 17082 - 8. Retrieved from `` https://en.wikipedia.org/w/index.php?title=Residual_sum_of_squares&oldid=836036149 '' Categories : Least squares Errors and residuals Hidden categories : Articles needing additional references from April 2013 All articles needing additional references Talk Contents About Wikipedia Deutsch Italiano 日本 語 Edit links This page was last edited on 12 April 2018, at 08 : 53 ( UTC ). Text is available under the Creative Commons Attribution - ShareAlike License ; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia ® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization. About Wikipedia

Question: how do you find the sum of squared residuals
Answer: total sum of squares = explained sum of squares + residual sum of squares