Cochran's theorem
In statistics, Cochran's theorem, devised by William G. Cochran, is a theorem used to justify results relating to the probability distributions of statistics that are used in the analysis of variance. Statement. Suppose "U", ..., "U" are i.i.d. standard normally distributed random variables, and formula_1are positive semidefinite matrices with formula_2. Further suppose that formula_3, where "r" is the rank of formula_4. If we write so that the "Q" are quadratic forms, then Cochran's theorem states that the "Q" are independent, and each "Q" has a chi-squared distribution with "r" degrees of freedom. Less formally, it  is the number of linear combinations included in the sum of squares defining "Q", provided that these linear combinations are linearly independent. Proof. We first show that the matrices "B" can be simultaneously diagonalized and that their non-zero eigenvalues are all equal to +1. We then use the vector basis that diagonalize them to simplify their characteristic function and show their independence and distribution. Each of the matrices "B" has rank "r" and thus "r" non-zero eigenvalues. For each "i", the sum formula_6 has at most rank formula_7. Since formula_8, it follows that "C" has exactly rank "N  − r". Therefore "B" and "C" can be simultaneously diagonalized. This can be shown by first diagonalizing "B". In this basis, it is of the form: Thus the lower formula_10 rows are zero. Since formula_11, it follows that these rows in "C" in this basis contain a right block which is a formula_12 unit matrix, with zeros in the rest of these rows. But since "C" has rank "N" − "r", it must be zero elsewhere. Thus it is diagonal in this basis as well. It follows that all the non-zero eigenvalues of both "B" and "C" are +1 . Moreover, the above analysis can be repeated in the diagonal basis for formula_13. In this basis formula_14 is the identity of an formula_15 vector space, so it follows that both "B" and formula_16 are simultaneously diagonalizable in this vector space (and hence also together with "B"). By iteration it follows that all "B"-s are simultaneously diagonalizable. Thus there exists an orthogonal matrix formula_17 such that for all formula_18, formula_19 is diagonal, where any entry formula_20 with indices formula_21, formula_22, is equal to 1, while any entry with other indices is equal to 0. Let formula_23 denote some specific linear combination  of all formula_24 after transformation by formula_17. Note that formula_26 due to the length preservation of the orthogonal matrix S, that the Jacobian of a linear transformation is the matrix associated with the linear transformation itself, and that the determinant of an orthogonal matrix has modulus 1. The characteristic function of "Q" is: This is the Fourier transform of the chi-squared distribution with "r" degrees of freedom. Therefore this is the distribution of "Q". Moreover, the characteristic function of the joint distribution of all the "Q"s is: From this it follows that all the "Q"s are independent. Examples. Sample  mean and sample variance. If "X", ..., "X" are independent normally distributed random variables with mean "μ" and standard deviation "σ" then is standard normal for each "i". Note that the total "Q" is equal to sum of squared "U"s as shown here: which stems from the original assumption that formula_31. So instead we will calculate this quantity and later separate it into "Q"'s. It is possible to write (here formula_33 is the sample mean). To see this identity, multiply throughout by formula_34 and note that and expand to give The third term is zero because it is equal to  a constant times and the second term has just "n" identical terms added together. Thus and hence Now formula_40 with formula_41 the matrix of ones which has rank 1. In turn formula_42 given that formula_43. This expression can be also obtained by expanding formula_44 in matrix notation. It can be shown that the rank of formula_45 is formula_46 as the addition of all its rows is equal to zero. Thus the conditions for Cochran's theorem are met. Cochran's theorem then states that "Q" and "Q" are independent, with chi-squared distributions with "n" − 1 and 1 degree  of freedom respectively. This shows that the sample mean and sample variance are independent. This can also be shown by Basu's theorem, and in fact this property "characterizes" the normal distribution – for no other distribution are the sample mean and sample variance independent. Distributions. The result for the distributions is written symbolically as Both these random variables are proportional to the true but unknown variance "σ". Thus their ratio does not depend on "σ" and, because they are statistically independent. The distribution of their ratio is given by where "F" is the F-distribution with 1 and "n" −  1 degrees of freedom (see also Student's t-distribution). The final step here is effectively the definition of a random variable having the F-distribution. Estimation of variance. To estimate the variance "σ", one estimator that is sometimes used is the maximum likelihood estimator of the variance of a normal distribution Cochran's theorem shows that and the properties of the chi-squared distribution show that Alternative formulation. The following version is often seen when considering linear regression. Suppose that formula_53 is a standard multivariate normal random vector (here formula_54 denotes the "n"-by-"n" identity matrix), and if formula_55 are  all "n"-by-"n" symmetric matrices with formula_56. Then, on defining formula_57, any one of the following conditions implies the other two: