1EDF 6481
Quantitative Research Methods in Education
You will need pages 202 to 216 of the Target Material and a t table and an F table.
1. From page 204, the required formula for t ist=

Y j − Y j′
⎡11⎤
MS S / A ⎢ + ⎥⎣⎢ n j n j′ ⎦⎥We need MS S / A which is the pooled variance for the five levels of the factor. Thus

( n1 − 1) S12 + " + ( n5 − 1) S52( n1 − 1) + " + ( n5 − 1)
( 20 − 1) 3 + ( 20 − 1) 4 + ( 20 − 1) 5 + ( 20 − 1) 4 + ( 20 − 1) 4=( 20 − 1) + ( 20 − 1) + ( 20 − 1) + ( 20 − 1) + ( 20 − 1)

MS S / A ==4As an example calculation, we make the comparison of the means for F1 and F2:

t=14.6 − 14.91⎤⎡1
4⎢ + ⎥⎣ 20 20 ⎦= −.47

To determine significance while controlling the familywise error rate, we could
use the Welsch procedure or the Shaffer-Holm procedure. First I report results of theShaffer-Holm procedure.
In the Shaffer-Holm procedure, we first test the omnibus null hypothesis. The
problem provides the F statistic: F ( 4,95) = 11.62 . The critical value isFα , J −1, N − J = F.05, 4,95 = 2.467 , so the omnibus null hypothesis is rejected. Next the Shaffer-

Holm table is completed. (The table is presented on the next page.) The ts are ranked
from most extreme to least extreme. The values of C are taken from Table 1 on page 209

2of the Target materials. The critical values were computed by using the following
program:data;
input alphafw df c;
prob =1-alphafw/(2*c);cval = tinv(prob,df);
cards;.05 95 6.05 95 4.05 95 3.05 95 2.05 95 1
proc print;
run;
quit;
Obs

alphafw

dfcprobcval

12
34
5

0.05
0.05
0.05
0.05
0.05

95
95
95
95
95

643
2
10.99583
0.99375
0.99167
0.98750
0.97500

2.694632.54615
2.437112.27748
1.98525

Step

T

C

±tα fw 2, C , N − J

Comparison

12345678910

5.06
4.90
4.59
4.42
3.32
3.16
1.74
1.26
-0.47
0.16

666664432
1± 2.69463± 2.69463± 2.69463± 2.69463± 2.69463± 2.54615± 2.54615± 2.43711± 2.27748± 1.98525

F2 vs. I2*F2 vs. I1F1 vs. I2
F1 vs. I1
N vs. I2**
N vs. I1F2 vs. N
F1 vs. N
F1 vs. F2I1 vs. I2*I2 was mislabeled as I1 in the problem. The mean for I2 was 11.7.
** N was mislabeled as C. the mean for N was 13.8
We find that the following pairs result in significant differences: F2 vs. I2, F2 vs. I1, F1vs. I2, F1 vs. I1, N vs. I2, and N vs. I1. Thus the means under facilitative (F1 and F2) and3
the neutral (N) conditions were significantly different from the means under interferingconditions.
To use the Welsch procedure the following table is constructed. The criticalvalues were approximated using Table E.1 (page 210) and degrees of freedom 60 and120. The critical values in this table are called q. All ts are listed as negative as would be
the case if int=

Y j − Y j′
⎡11⎤
MS S / A ⎢ + ⎥⎣⎢ n j n j′ ⎦⎥the larger mean were subtracted from the smaller mean.

11.7

11.8

11.7
--0.15
11.8
-13.8
14.6
14.9
* Significant by testing** Significant by implication

13.8

14.6

14.9

-3.32**-3.16*--

4.59**-4.43**-1.26--

5.06**
4.90**
1.74
-0.47
--

±q 2
± 2.82± 2.63± 2.62± 2.34The conclusions are the same as from application of the Shaffer-Holm procedure.
To determine significance while controlling the per comparison error rate we use
the same t statistics but change the critical value to ±tα pc 2, N − J = ±t.05 2,95 = ±1.98525 . Theomnibus test is not used. In this example even though we have different critical values
when we control the familywise error rate and the per comparison error rate, the same tstatistics are significant and our conclusions are the same.

4
2. From page 204, the required formula for t ist=

Y j − Y j′
⎡11⎤
MS S / A ⎢ + ⎥⎢⎣ n j n j′ ⎥⎦We need MS S / A which is the pooled variance for the five levels of the factor. Thusn1 − 1) S12 + ( n2 − 1) S 22 + ( n3 − 1) S32(MS S / A =( n1 − 1) + ( n2 − 1) + ( n3 − 1)
(10 − 1) 72 + (10 − 1) 62 + (10 − 1) 76=(10 − 1) + (10 − 1) + (10 − 1)
= 70

As an example calculation we take the comparison of the means for Before and After:t=

20 − 141⎤⎡170 ⎢ + ⎥⎣ 10 10 ⎦= 2.26

To determine significance while controlling the familywise error rate, we could
use the Welsch test or the Shaffer-Holm test. In the Shaffer-Holm procedure, we first test
the omnibus null hypothesis. The problem provides the F statistic: F ( 2, 27 ) = 2.047 . The
critical value is Fα , J −1, N − J = F.05, 2,27 = 3.35 and the omnibus null hypothesis is not rejected.
Thus all comparisons are declared non-significant.
In the Welsch procedure we construct the table on the following page. The
critical values were approximated using Table E.1 (page 210) and degrees of freedom 24and 30.

51313-1420* Significant by testing** Significant by implication

14200.38---2.65**-2.26*--

q2± 2.67± 2.06±We can conclude there are significant differences between Before and Control (20 vs. 13)
and Before and After (20 vs. 14).
To determine significance while controlling the per comparison error rate we use
the same t statistics but change the critical value to ±tα pc 2, N − J = ±t.05 2,27 = ±2.052 . Theomnibus test is not used. We find the same results as were found by using the Welschtest.
3. From page 204, the required formula for t ist=C.
SC2nBecause the data are available, all calculations can be done by using SAS. However, theformula tells the critical value formula the t statistics will be n − 1 . Here is the program I
used. It includes code for obtaining descriptive statistics and the F test
data;
input x1 1-3 x2 5-7 x3 9-11 x4 13-15;c12=x1-x2;
c13=x1-x3;
c14=x1-x4;
c23=x2-x3;
c24=x2-x4;
c34=x3-x4;
datalines;
1.8
2.2
3.2
2.4
2.4
1.5
1.9
2.7
1.9
1.7
2.5
3.5
2.7
2.6
2.4
3.1

6
4.7
4.8
4.4
4.8
3.6
3.1
4.2
5.4
4.4
4.2
4.1
4.9
5.8
6.1
6.4
6.6proc means;var x1 x2 x3 x4;
proc glm;
model x1 x2 x3 x4=/nouni;
repeated tmt 4/nom;proc means n mean var t prt;
var c12 c13 c14 c23 c24 c34;
run;
quit;
The MEANS Procedure
Variable N
Mean
Std Dev
Minimum
Maximumƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ
x1
8
3.4125000
1.4535056
1.8000000
5.8000000
x283.2750000
1.6228282
1.5000000
6.1000000
x38
3.6375000
1.4530141
1.9000000
6.4000000
x484.1750000
1.4752724
2.4000000
6.6000000
ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒThe GLM Procedure
Number of observations

8The GLM Procedure
Repeated Measures Analysis of VarianceRepeated Measures Level Information
Dependent Variable

x1x2x3x4

Level of tmt

1234

The GLM Procedure
Repeated Measures Analysis of Variance
Univariate Tests of Hypotheses for Within Subject Effects
SourceDF

Type III SS

Mean Square

F Valuetmt
Error(tmt)321

3.76250000
4.14750000

1.25416667
0.19750000

6.35

SourcePr > F

Adj Pr > F
G - G
H - F

tmt
Error(tmt)

0.00310.0093

Greenhouse-Geisser Epsilon
Huynh-Feldt Epsilon

0.00310.7086
1.0263

7The MEANS Procedure
VariableN
MeanVariancet ValuePr > |t|ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ
c1280.1375000
0.1798214
0.920.3896
c138-0.2250000
0.4507143-0.95
0.3747c148-0.7625000
0.3798214-3.50
0.0100c238-0.3625000
0.3226786-1.80
0.1141c248-0.9000000
0.6457143-3.170.0158
c348-0.5375000
0.3912500-2.430.0454
ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ

If we use the Shaffer-Holm procedure to control the familywise error rate we startwith the omnibus test. The ANOVA resulted inF (ε ( P − 1) , ε ( P − 1) ( n − 1) ) = F (1 ( 4 − 1) ,1 ( 4 − 1) ( 8 − 1) ) = F ( 3, 21) = 6.35, p = .003 ,so the omnibus null hypothesis is rejected. The Shaffer-Holm Table isStept

C±tα fw 2, C ,n −1

Comparison

123
456-3.50-3.17-2.43
-1.80
-0.95
0.92

33
3
3
2
1

3.13
3.13
3.13
3.13
2.84
2.36

1 vs. 42 vs. 4
3 vs. 42 vs. 3
1 vs. 3
1 vs. 2

We find that the mean for group 4 (4.18) is significantly different from the means forgroup 1 (3.41), group 2 (3.28), and group 3 (3.64), but there are no significant differencesamong the latter means.
To control the per comparison error rate the critical value is±tα pc 2, n −1 = ±t.05 2,8−1 = 2.36 and we find the same results.
4. From page 204, the required formula for t ist=

CSC2n.

8Because the data are available, all calculations can be done by using SAS. Again, theformula tells the critical value formula the t statistics will be n − 1 . Here is the program I
used. It includes code for obtaining descriptive statistics and the F testdata;
input x1 1-2 x2 4-5 x3 7-8 x4 10;
datalines;
9 6 7 5
11 8 6 66 8 7 5
13 10 10 9
12 8 9 6
proc means;var x1 x2 x3 x4;
proc glm;
model x1 x2 x3 x4=/nouni;
repeated tmt 4/nom;proc means n mean var t prt;
var linear quad cubic;
run;
quit;

Here are the results. Note that means suggest that scores on the dependent variable
declines as time passes.
The SAS System

2

The MEANS Procedure
Variable NMean
Std Dev
Minimum
Maximumƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒx1
5
10.2000000
2.7748874
6.0000000
13.0000000
x2
58.0000000
1.4142136
6.0000000
10.0000000
x3
57.8000000
1.6431677
6.0000000
10.0000000
x456.2000000
1.6431677
5.0000000
9.0000000
ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒThe SAS System

3The GLM Procedure
Number of Observations Read
Number of Observations Used55

9The SAS System

4The GLM Procedure
Repeated Measures Analysis of VarianceRepeated Measures Level Information
Dependent Variable

x1x2x3x4

Level of tmt

1234

The SAS System

5

The GLM Procedure
Repeated Measures Analysis of Variance
Univariate Tests of Hypotheses for Within Subject Effects
SourceDF

Type III SS

Mean SquareF ValuetmtError(tmt)

312

40.55000000
17.70000000

13.51666667
1.47500000

9.16

SourcePr > F

Adj Pr > F
G - G
H - F

tmt
Error(tmt)

0.00200.0145

Greenhouse-Geisser Epsilon
Huynh-Feldt Epsilon

0.00340.54760.8789The SAS System

6

The MEANS Procedure
VariableN
Mean
Variancet Value
Pr > |t|ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ
linear
5-12.2000000
28.7000000-5.09
0.0070quad
50.6000000
7.3000000
0.50
0.6455cubic
5-3.4000000
23.3000000-1.58
0.1904ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ

We can use the Shaffer-Holm or Bonferroni-Holm procedure depending onwhether we want to incorporate the omnibus hypothesis test or not. In the Shaffer-Holmprocedure, we first test the omnibus null hypothesis. The F statistic is

F (.8783(3),.8789(12) ) = 9.16, p < .05 , so the omnibus null hypothesis is rejected. Next
the Shaffer-Holm table is completed. (The table is presented on the next page.) The ts are10ranked from most extreme to least extreme. The values of C are the ranks in reverse withthe first C reduced by one. The critical values were computed by using the following
program:data;input alphafw df c;
prob =1-alphafw/(2*c);cval = tinv(prob,df);
cards;.05 4 2.05 4 1
proc print;
run;
quit;

Stept

C±tα fw 2, C ,n −1

123-5.09-1.58
0.50

221± 3.49541± 3.49541± 2.77645

Only the linear trend is significant.
In the Bonferroini-Holm procedure, we do not incorporate the test of the omnibus
null hypothesis. The Bonferroni-Holm table is like the Shaffer-Holm table, except thatthe first C is not reduced by one. The critical value program isdata;
input alphafw df c;
prob =1-alphafw/(2*c);cval = tinv(prob,df);
cards;.05 4 3.05 4 2.05 4 1
proc print;
run;
quit;

Stept

C±tα fw 2, C ,n −1

1
2
3-5.09
-1.58
0.50

32
1

± 3.96079± 3.49541± 2.77645

The results again indicate a significant linear trend.