GaussMarkov theorem

# Overview
In statistics, the GaussMarkov theorem, named after Carl Friedrich Gauss and Andrey Markov, states that in a linear model in which the errors have expectation zero and are uncorrelated and have equal variances, a best linear unbiased estimator (BLUE) of the coefficients is given by the least-squares estimator. The errors are not assumed to be normally distributed, nor are they assumed to be independent (but only uncorrelated  a weaker condition), nor are they assumed to be identically distributed (but only having zero mean and equal variances).

# Statement
Suppose we have
for i = 1, . . ., n, where j are non-random but unobservable parameters, Xij are non-random and observable (called the "explanatory variables"), i are  random , and so Yi are random. The random variables i are called the "errors" (not to be confused with "residuals"; see errors and residuals in statistics). Note that to include a constant in the model above, one can choose to include the XiK = 1.
The GaussMarkov assumptions state that
- {\rm E}\left(\varepsilon_i\right)=0,
- {\rm Var}\left(\varepsilon_i\right)=\sigma^2
(i.e., all errors have the same variance; that is "homoscedasticity"), and
- {\rm Cov}\left(\varepsilon_i,\varepsilon_j\right)=0
for i\not=j; that is "uncorrelatedness."
A linear estimator of j is a linear combination
in which the coefficients cij are not allowed to depend on the earlier coefficients , since those are not observable, but are allowed to depend on X, since this data is observable, and whose expected value remains j even if the values of X change.  (The dependence of the coefficients on X is typically nonlinear; the estimator is linear in Y and hence in  which is random; that is why this is "linear" regression.)  
The estimator is unbiased iff
Now, let \sum_{j=1}^K\lambda_j\beta_j be some linear combination of the coefficients. Then the mean squared error of the corresponding estimation is defined as
i.e., it is the expectation of the square of the difference between the estimator and the parameter to be estimated.  (The mean squared error of an estimator coincides with the estimator's variance if the estimator is unbiased; for biased estimators the mean squared error is the sum of the variance and the square of the bias.) A best linear unbiased estimator of  is the one with the smallest mean squared error for every linear combination .  This is equivalent to the condition that
is a positive semi-definite matrix for every other linear unbiased estimator \tilde\beta.
The ordinary least squares estimator (OLS) is the function
-f Y and X that minimizes the sum of squares of residuals
(It is easy to confuse the concept of error introduced early in this article, with this concept of residual.  For an account of the differences and the relationship between them, see errors and residuals in statistics).
The theorem now states that the OLS estimator is a BLUE. The main idea of the proof is that the least-squares estimator isuncorrelated with every linear unbiased estimator of zero, i.e., with every linear combination a_1Y_1+\cdots+a_nY_n
whose coefficients do not depend upon the unobservable  but whose expected value is always zero.

# Generalized least squares estimator
The GLS or Aitken estimator extends the Gauss-Markov Theorem to the case where the error vector has a non-scalar covariance matrix - the Aitken estimator is also a BLUE.