1
EDF 6481
Quantitative Research Methods in Education2012 Spring Term
Instructor:

James Algina

Office:119D Norman Hall

Phone:

273-4305

Email:

algina@ufl.edu

Course Web Page: http://plaza.ufl.edu/algina/index.html
To help me keep track of email messages, please include EDF 6481 and yourlast name in the Subject line of any email message you send to me.

If you have never send me an email prior to today, please send me an email
message today so I can put your email address in my address book andconstruct a distribution list for the class.

Office Hours: Tuesday and Thursday 1:00 – 2:30 and by appointment. (To set an appointment,
it is best to send me an email indicating several times when you are free.)Course Description
An introduction to the basic designs used in educational research and more generally inbehavioral science research. This course focuses on normal-theory procedures for analyzing
data collected in these designs and, for the simpler designs, on alternatives for use when theassumptions of normal-theory procedures are violated.
Prerequisite
The prerequisite for this course is a statistics course including the following topics:
measures of central tendency and variability, correlation, hypothesis testing (null and alternate
hypothesis, Type I error rate, and power), and t-tests on means.
Texts
Myers, J. L., Well, A. D., & Lorch, R. F. (2010). Research design and statistical analysis(3rd. Ed.). New York: Routledge
Most of the material I cover in class is also covered in Myers, Well, & Lorch. In addition it isan excellent reference for those who will be doing research throughout their lives.

2

Reserve Text
Huck, S. W. (2008). Reading statistics and research (5th. Ed.). Boston: Allyn & Bbacon
Huck provides examples of many of the analyses presented in the class, but does not cover thematerial from class.
Other Materials
There is a packet of handouts at Target Copy Center (1412 West University Avenue).
Purchase a copy of the packet and bring it to class every day. You also need to bring the t table
and F table I distribute to you.
SAS
Information about licensing SAS is available on the course website. See SAS Student
Use Program. Students can also access SAS in 115 Norman Hall. This lab is usually open
between 9:00 and 5:00 but the hours can be irregular on Fridays. The computers with SAS
installed on them are also used by students who are not enrolled in EDF 6481.
G*Power 3.1.3
Download and Installation Instructions
Enter the following in your browser:
http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3
Click Download and register
Find and click on download
Download G*Power... 3.1.3 for Windows XP, Vista, or 7 (32 and 64 bit) (10 MB) [download]
Save the zip file and then extract the contents
There will be a folder with the title GPower_3.1.3. Open GPower_3.1.3 and you will find afolder with the name Gpower. Open Gpower and click on setup.exe. Follow the steps to install
Gpower 3.1.3.
Once Gpower 3.1.3. is installed you can access it from your program list.

3
Computer Manuals
In this course the Statistical Analysis System (SAS) is used for computerized data
analysis. You are not required to buy any computer manuals. However, some students may findthe following introduction to SAS useful.
Delwiche, L. D., & Slaughter, S. J. (2008). The little SAS book, a primer (4th. Ed.).
Cary, NC: SAS Institute, Inc.
A more advanced text that includes material on proc glm is:
Littell, R. C., Stroup, W. W., & Freund,R. J. (2004) SAS for Linear Models, (4th. Ed.).
Cary, NC: SAS Institute, Inc.
References
The following are titles of statistics texts that are more comprehensive than our text:
Kirk, R. E. (1995). Experimental Design: Procedures for the Behavioral Sciences (3rd.
Ed.). Belmont, CA: Brooks/Cole.
Maxwell, S. E., & Delaney, H. D. (2004). Designing Experiments and Analyzing Data
(2nd. Ed.). Erlbaum.
Winer, B. J., Brown, D. R., & Michaels, K. M. (1991). Statistical Principles inExperimental Design. New York: McGraw-Hill.
The following treat the design of experiments in more detail than is possible in class:Shadish, W. R., Cook, T. D., & Campbell, D. T. (2001). Experimental and QuasiExperimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin Co.
Cox, D. R. (1958). Planning of Experiments. New York: Wiley.
Cox, D. R., & Reid, N. R (2006) The Theory of the Design of Experiments (Chapman &Hall/CRC Monographs on Statistics & Applied Probability. Boca Raton: Chapman & Hall/CRC
Course Requirements
Examinations. I plan to have three examinations. Dates are shown on the list of course
topics and are approximate. Descriptions of the content of the exams follow the list of course
topics.
Exercises. There are two sets of data analysis exercises that you must submit andcomplete.

4 The first set is due on the day of the first exam, which I predict to be February 14, 2012 The second set includes exercises that cover the material in Topic E (Multiple Comparisons
for One Factor Designs) and exercises that cover the material in Topics F, G, and H.
Those that cover Topic E are due on the day of the second exam, which I predict to be
March 22, 2012. Those that cover Topics F, G, and H are due on April 24, 2012, thelast day of class.
I urge you to begin working on exercises in advance of the due date. For exercises other than
the final exercise, each late exercise will be assessed two points per class it is late. Each latefinal exercise assessed two points on the following days: Thursday of reading days, Monday
of finals week, Wednesday of finals week, and Friday of finals week. Exercises will not be
accepted after Friday of finals weeks and will receive a zero. If you have trouble completingan exercise, you should see me. You should not consult with other students or faculty--see theAcademic Honesty section of this syllabus. I will answer questions about exercises, though I
may ask you to pose your questions in general terms, rather than as a question about a specificexercise.
There are three power problems that you must submit and complete. These threeproblems receive one grade. The power problems are due two weeks after Topic D iscompleted. I predict that the power problems will be due on February 23, 2012.

GradesEach examination will be scored on a percentage correct scale (0-100%). Each exercise
will be scored on a 14-point scale (0 to 13). The relationship between scores and grades is shownin the following table:
Exams
Score
 90
86 2  903

83 1  86 233
80  83 13

 76 2  803

 73 1  76 2
33

 70  73 13

 66 2  70
3

 63 1  66 2
3
3














Exercises
Grade Score Grade
A
11-13
A
A10
AB+

9

B+

B

8

B

B-

7

B-

C+

6

C+

C

5

C

C-

4

C-

D+

3

D+

5
 60  63 1
3

56 2  60
3

 56 2
3




D

2

D

D-

1

D-

E

0

E

 Averaging scores on the individual exercises into a single score will compute an exercisescore.
 Averaging scores on the three examinations will compute an examination score.
 Final grades will be based on1
Score  Examination  55  3  Exercise  3Final grades will be awarded based on the following table

Score
 90
86 2  903

83 1  86 233
80  83 13

 76 2  803

 73 1  76 2
33

 70  73 13

 66 2  70
3

 63 1  66 2
3
3

 60  63 1
3

56 2  60
3

 56 2
3












Grade
A
A-



B+
B
B-



C+
C
C-



D+
D
DE

Failure to submit exercises can have a severe negative impact on your grade.

6
Studying for the Course
The following are suggestions about studying for the course. Following these should
help most students, but in particular will help students who have difficulty in quantitative
courses.
1. Read the text before a topic is covered in class. This will provide backgroundinformation to help you comprehend the lectures.
2. Study your notes after the class.
3. Try the exercises that will be discussed in class before the dates on which they are to
be discussed.
4. Ask questions about the material you do not understand. The course moves quickly
and neglected misunderstandings may never be resolved and may make future material difficult
to understand. Ask questions in the class and consult with the instructor outside of class.
Academic Honesty

Exercises to be submitted for a grade. Students are expected to do their own work on
exercises that are submitted for a grade. You should not solicit help with your exercises fromother people (including students and faculty), nor should you provide help to otherstudents. Soliciting help includes discussing with other people (including students andfaculty) what the appropriate approach to an exercise might be. If you need to discuss an
exercise you should talk with me. Be advised however that if your question is about the correct
analysis for an exercise I will usually require you to phrase your question about analysis in
general and not about the specific exercise. If you know the correct analysis and have a question
about a computer program or formula to implement the analysis, I will usually answer thequestion directly.
Examinations. You are expected not to cheat on examinations.
Penalty for academic dishonesty. In the event a violation of standards of academic
honesty occurs, I will contact the Office of Student Conduct and Conflict Resolution todetermine if it is the first violation. If not I will turn the matter over to the Student Conduct andConflict Resolution. For a first offense the exercise or the exam will receive a grade of zero.
Students with Disabilities

Students with disabilities who need reasonable modifications to complete assignments
successfully and otherwise satisfy course criteria are encouraged to meet with the instructor as
early in the course as possible to identify and plan specific accommodations. Students must
supply a letter from the UF Office for Students with Disabilities to assist in planning
modifications.

7
Course Topics
A. t Tests Comparing Two Means. (1/0 to 1/17)
MWL: Chapters 1 – 5 are background for the following two subtopics. Chapter 6 (Sections
6.1 – 6.5.3) is foundational.
1. Between-subjects design--independent samples t.
MWL: Chapter 6 (Sections 6.4 and 6.7).
H: Chapter 5, 6, 7, 8, 10.
2. Within-subjects design--dependent samples t.
MWL: Chapter 6 (Section 6.5 and 6.7).
H: Chapter 5, 6, 7, 8, 10.
B. Analysis of Variance for Between-Subjects Designs with One or Two Factors. (1/17 to 1/24)
1. One factor.
MWL: Chapter 8 (Sections 8.1 – 8.6) and Appendix 8.1.
H: Chapter 11.
2. Two factors.
MWL: Chapter (Sections 9.1 – 9.3, 9.6, and 9.10).
H: Chapter 13.
C. Analysis of Variance for Within-Subjects Designs with One or Two Factors and for Mixed(Split-Plot) Designs (1/24 to 1/31)
1. One within-subjects factor.
MWL: Chapter 14 (Sections 14.1 –14.7.1 and 14.8).
H: Chapter 14.
2. Two within-subjects factors.
MWL: Chapter 15 (Sections 15.1 – 15.2.2 and 15.9.2).
H: Chapter 14.
3. One between-subjects and one within-subjects factor.
MWL: Chapter 15 (Sections 15.3 – 15.3.3 and 15.9.2).
H: Chapter 13.
The first exam will be two weeks after Topic C is completed. The expected date for the exam is
2/14/12 Exercises covering Topics A to C are due on the day of the first exam.
D. Selecting Sample Size for One and Two Factor Designs. (2/2- 2/09)
MWL: Chapter 4 (Section 4.5), Chapter 5 (Section 5.6), and Chapter 13 (Sections 13.1 –13.3.4 and 13.5) are background for the following subtopics.
1. One-factor designs with two levels.
MWL: Chapter 6 (Sections 6.8 – 6.9).
2. One between-subjects factor.
MWL: Chapter 8 (Section 8.7).
3. One within-subjects factor.
MWL: Chapter 14 (Section 14.9).
4. Two between-subjects factors.
5. Two within-subjects factors.

8
MWL: Chapter 15 (Section 15.10.1).
6. One between-subjects and one within-subjects factor.
MWL: Chapter 15 (Section 15.10.2).
E. Multiple Comparisons for One-Factor Designs. (2/16 to 3/1)
1. One-factor, between-subjects design.
MWL: Chapter 10 (Sections 10.1 – 10.7.3 and 10.8 – 10.9.1) and Chapter 11 (Sections 11.1 –11.4)
H: Chapter 8 (pp. 242-243) and 12.
2. One-factor, within-subjects designs.
MWL: Chapter 14 (Section 14.10).
The second exam will be on 3/22/12. Exercises covering Topic E are due on the day of thesecond exam.
F. Multiple Comparisons for Two Factor Designs. (3/13-3/20)
1. Two-factor, between-subjects designs.
MWL: Chapter 10 (Sections 10.10 – 10.10.4).
2. Two-factor, within-subjects designs.
MWL: Chapter 15 (Sections 15.8 – 15.8.3).
3. One-between-subjects and one-within-subjects factor.
MWL: Chapter 15 (Sections 15.8.7)
G. Factorial Designs with Three Factors. (3/27-4/03)
MWL: Chapter 9 (Section 9.4) and Chapter 15 (Section 15.4.4).
H. Analysis of Covariance for One-factor Between-Subjects Designs. (4/03-4/10)
MWL: Chapter 25 (Sections 25.1-25.4.6)
H: Chapter 15.
I. Assumptions. (4/10-4/17)
1. The dependent samples t test.
2. The independent sample t test.
MWL: Chapter 6 (Sections 6.6 – 6.6.2).
H: Chapter 10 (pp 248 – 262).
3. Analysis of variance for designs with one between-subjects factor.
MWL: Chapter 8 (Section 8.8).
H: Chapter 11 (pp272 – 276).
Assuming we meet the preceding schedule, we will have a week at the end of the course during
which as many of the following topics as time permits will be covered.
J. Analysis of Unbalanced Completely Crossed Between-Subjects Designs
MWL: Chapter 9 (Section 9.8).
K. Designs with Random or Nested Factors.

9
MWL: Chapter 15 (Sections 15.4)
L. Nonparametric Procedures for use in Place of t and F Tests.
M. Tests on Variances.
N. Tests on Correlations.
Third Examination: Thursday May 1, 2008 7:45 PM to 10:00 PM. Topic F is not included in thisexam. The exam time is the scheduled examination time for courses scheduled Tuesday andThursday, period 4.

10Examination Content
First examination. The following are descriptions of the kinds of items that will appear
on the first examination.
1. Definitions of terminology.
2. Stating appropriate null and alternate hypotheses with regard to two means. Justify thealternative hypothesis. Make correct conclusions (directional vs. nondirectional) in light of thealternative hypothesis.
3. Finding critical values. Make decisions about rejecting the null hypothesis.
Calculating p values for designs in which two means are compared.
4. Given a description of a research study:i. List the factors in the study.
ii. For each factor to which the terminology between-subjects and within-subjects factor
applies, state whether it is a between-subjects or a within-subjects factor.
iii. List the sources of variance; for each source list the degrees of freedom.
iv. List the mean square ratios for each effect that should be tested.
5. Given population cell means, determine which effects are present.
6. Select appropriate p value from among unadjusted, Huynh-Feldt adjusted andGreenhouse-Geisser adjusted p values. Justify the choice.
7. Interpret printouts for analysis of data collected in the designs studied in this section.
8. Compute Cohen’s effect size from means and the pooled standard deviation andinterpret the effect size.
9. Predict the effect of changes in sample size and changes in design on the standard
error of a mean difference and the width of a confidence interval in studies comparing the means
for two groups.

11Second examination. The following are descriptions of the tasks that will appear on thesecond examination.
1. Definitions of terminology: per comparison error rate, familywise error rate, planned
comparison, post hoc comparison, pairwise contrast, complex contrast, orthogonal contrast, trend
analysis, control of the per comparison error rate, weak control of the familywise error rate,strong control of the familwise error rate.
2. Compare and contrast (a) completely randomized design, (b) randomized block
design and (c) subjects as own control (repeated measures) design.
3. In a 2  2 counter balanced design, determine whether or not the main effect of thewithin-subjects factor should be interpreted and explain the reason for your decision.
4. Sample size calculations for one-factor designs with two levels.
5. State what a mean configuration is, what ES max is, and why ES max and meanconfigurations are used in power analysis.
6. State what central and non-central distributions are and explain their roles in power
analysis.
7. Explain why particular multiple comparisons are used.
a. Scheffe' versus Roy-Bose.
b. Scheffe' and Roy-Bose versus Bonferroni-Holm. Shaffer-Holm, and Welsch.
c. Bonferroni-Holm, Shaffer-Holm, and Welsch versus using procedures that bothdo not use an omnibus test and use t pc 2, df .
d. For pairwise comparisons: Shaffer-Holm and Welsch versus Bonferroni-Holm.
e. For orthogonal comparisons: Shaffer-Holm versus Bonferroni-Holm.
8. Explain the consequences for Type I error rates and power of choosing among thefollowing pairs of proceduresa. Post hoc versus planned comparison procedures.
b. Procedures that strongly control the familywise error rate versus procedures that
control the per comparison error rate12c. Incorporating the omnibus test in procedures that need not incorporate it.
d. Procedures for a subset of pairwise comparisons versus procedures for all
possible pairwise comparisons.
e. Procedures for a subset of orthogonal comparisons versus procedures for acomplete set of orthogonal comparisons.
f. Welsch versus Tukey.
g. Shaffer-Holm versus Bonferroni.
9. Given the statistics necessary to conduct multiple comparisons for a one-factor
design, implement the multiple comparison procedure for pairwise and for complex
comparisons. You need to know how (a) calculate t statistics for pairwise and for complex
comparisons implement, determine degrees of freedom, and (c0 implement Scheffe', Roy-Bose,
Bonferroni-Holm, Shaffer-Holm, Welsch, and Fisher’s LSD procedures.

Third examination. The following are descriptions of the tasks that will appear on thethird examination.
1. Given a description of a one, two or three-factor design.
a. List the factors
b. For those factors to which the terminology applies, state whether they are between- or
within-subjects factors.
c. State whether the factors are crossed or nested.
d. Identify any covariates.
2. Given a three-factor design described in terms of the within- and between-subjects
structure, list the sources of variance, degrees of freedom, and mean square ratios.
3. Given descriptive statistics and the results of a three-factor ANOVA, construct tablesof means appropriate for the pattern of significant results and write interpretations of the results.
4. Interpret printouts for the analysis of data collected in a three-factor design.
5. Describe the following designs: (a) completely randomized design, (b) completely
randomized design with a covariate, (c) repeated measures design, (d) randomized block design,13
and (e) generalized randomized block design. Discuss the advantages and disadvantages of these
designs. (H).
6. For a design in which subjects are randomly assigned to treatments and the same
variable is measured before the treatments are administered and after the treatments areadministered: (a) state the alternative methods of analysis (b) state and explain the relationshipbetween the effects tested in these methods and the hypotheses of interest in the study, and (c)select the best method of analysis and justify your choice. (H).
7. Present and explain the steps in an analysis of Covariance. Interpret printouts fordata analyzed by using the analysis of covariance. (H).
8. State assumptions for the independent-samples t-test, one-factor between-subjects
ANOVA. State the impact of violations of assumptions. (I).
9. Select appropriate SS in unbalanced designs and explain your selection. (J).
10. Describe the uses of expected mean squares. (K).
11. Given expected mean squares for a design, indicate the mean square ratios thatshould be used to test the effects in the design. (K).
12. Explain why a factor would be declared as random rather than fixed. (K).
13. Discuss designs in which subjects are nested within groups and groups are nested
within treatments. State the impact of using the wrong error term when groups within treatments
is included as a source of variance and of ignoring groups within treatments as a source ofvariance. Compare the nested design to one in which groups and treatments are crossed in termsof hypotheses that can be tested and practical issues. Present the problems that arise when eachtreatment has just one group nested in it and when the design is unbalanced. (K).
14. Discuss nonparametric procedures in terms of motivation for their development andadvantages and disadvantages relative to parametric procedures. (L).
15. Compare and contrast the F test and Brown-Forsythe test on variance for a twogroup between-subjects design in terms of data analysis method and Type I error rates. (M).