By definition a 2 variable with  degrees of freedom is the sum of squares of  independent, standard normal variables N(0,1). Remember also that the unbiased estimate of 2 is the sample variance defined by From which we obtain = (n-1) s2. Since this is the sum of squares of n-1 normal variables {u2, …, un} which are all independent with means 0 and variance 2, what we have demonstrated is that, for a random sample of size n from a normal population, , independently of FINALLY let’s return to the logLikelihood for the random normal sample {y1, y2, …, yn} discussed earlier. The last form we looked at on page 4 was Rather than look at the logLikelihood of this set of variables, we look instead at the logLikelihood of the transformed set of variables {u1, u2, …, un} for which the Jacobian was seen to be 1 (and remember u1 = n and ): Given that u1 is normally distributed with mean n  and variance 2, we now can separate out the two terms. Thus, the logLikelihood of the transformed set of variables {u1, u2, …, un} The first is the likelihood for u1 from which can be maximized to provide the ML/REML estimate of . The second is the likelihood for the set of variables {u2, u3, …, un} which are all independent of u1, and this provides the REML estimate of 2.
Next, E(u) = P E(y) = P X, that is Recall that rows 3 to n of P are orthogonal to rows 1 and 2 of P, and notice that the two columns of the design matrix X are proportional to rows 1 and 2 of P. Hence the means of {u3, …, un} must all be 0 by orthogonality. Next, looking the just the first two rows of these matrices, and given that , after matrix multiplication we obtain Finally, we make the transformation u = Py and develop the logLikelihood of {u1, u2, …, un}. The Jacobian of the transformation is 1 (since the transformation is orthogonal and the absolute value of det(P)=1). Since the inverse of P is PT, we can replace y = P-1u in the logLikelihood of {y1, y2, …, yn} by PTu instead: so Next PTP = I so this can be added inside the two brackets without changing their values. Taking the common PT from both brackets, preserving the correct order of multiplication and noting that (PT)T = P gives:
The Wald test of fixed effects using REML So, we now wish to test that a linear function of the fixed effects is some fixed value. Specifically, we test for a matrix of order r×p and a vector of length r. Then following immediately from the result for the distribution of we can say that is the Wald statistic. Note that the REML estimates of the variance parameters are used in this expression. The scaled Wald statistic therefore is F=W/r and this has an asymptotic 2 distribution with r degrees of freedom. However, for the reasons just pointed out, the P values will be of the true P values, so if the P value of the scaled Wald statistic is calculated using this asymptotic distribution then care needs to be taken with the interpretation in many cases. In 1997 Kenward and Roger (in Small Sample Inference for Fixed Effects from Restricted Maximum Likelihood, Biometrics, 53, 983–997) developed a method of improving the P values by scaling by a further factor: F* =  F. They developed the equations necessary to evaluate  as well as the denominator df (the numerator df = r). They showed by simulation that the new P values were much more reliable. In fact, two important properties of this approach can be stated: For an orthogonal design such as in ANOVA with no missing values, the P values of the scaled Wald statistic are exact, that is, they reproduce the ANOVA F P values. When r = 1 (that is, testing the equality of two treatment the P values of the scaled Wald statistic are the same as the Satterthwaite P values from an unpaired t test with unequal treatment variances.
Examples of correlated error structures GenStat allows the Random Model to be defined with a reasonable selection of correlations structures. The * indicates models that StATS has used relatively frequently. Model Commonly used for: Identity* independent, normally distributed errors in a regression or ANOVA with constant variance uniform* essentially the correlated structure for a multi-strata design (RCB, split-plot etc) diagonal* for any design with changing variance AR* autoregressive (AR1 or AR2) serially correlated errors in time series/repeated measures; spatial models in field trials equivalent to AR1 but allows unequally spaced time points; spatial models in field trials with unequally spaced coordinates unstructured* time series/repeated measures data where no assumption is made about the correlations over time; MANOVA data antedependence* time series/repeated measures data allowing changing variance, plus: order = 1 reproduces sample correlations for neighbouring time points; order = 2 reproduces sample correlations for first and second neighbouring time points; involves fewer parameters than unstructured ARMA a mixture of autoregressive and moving average serially correlated errors in time series/repeated measures boundedlinear correlations decrease linearly in proportion to ratio of distance apart spherical correlations decay spherically with distance, more common in soil science banded correlation equally close points have the same correlation, the order determines how many are non-zero FA & FAequal Correlation structure is in terms of a factor analysis model using fewer parameters than unstructured; more common in plant breeding Fixed correlation matrix specified by user MA moving average serially correlated errors in time series/repeated measures circular serially correlated errors in time series/repeated measures in the correlation changes with distance in a way that depends on the sin-1 function; linearvariance correlations decay linear with distance, more common in soil science What follows is a selection of examples that use some of the correlation models above. The first half of this manual described the Identity structure. The examples are mostly illustrated in an earlier manual available on the Resources page on . Example 1 – uniform structure: randomised block models
where represents a diagonal matrix with b (matrix) elements on the leading diagonal, each equal to , a t×t matrix of all 1s (which is also ). To see whether the assumption random blocks affects the estimation of the fixed effects - recall that - we need to look at the matrix H for the RCB design: The inverse of exists and will clearly be a block diagonal matrix with diagonal matrices each being the inverse of . There is a standard formula for such a matrix. Let be a nonsingular matrix and let both and be column vectors. So, here we have , , . is a diagonal matrix composed of b such matrices. Next we look at the individual matrices and for random blocks. Now with our parameterization of the design matrix X has columns each of which contains b cells containing 1 and the remaining cells contain 0. Also, an entry of 1 is unique in any row, so must equal . Next, for each block in the diagonal matrix above, simply adds the numbers in the columns of the design matrix X in the block under consideration. However, in that (and every) block,
every entry is 0 except for a single entry of 1. Because of the nature of the design matrix X, must equal . Since each of the t rows in contains b cells equal to 1 and the rest 0, we must have and using we obtain This is a matrix with diagonal elements equal to and off-diagonal elements . The last term to evaluate is partly resolved we know : it consists of t rows, each having b cells equal to and t(b-1) cells all equal to . The positions of these cells are dictated by the design matrix X, however when is post-multiplied by y, the ith row results in the ith treatment mean as well as the grand mean . We need to introduce the vector of data means which we’ll denote as . Then Combining the two terms and replacing terms like by leads to