Selecting a Valid Sample Size 

for Longitudinal and 

Multilevel Studies in Oral 

Behavioral Health 

Henrietta L. Logan, Ph.D.1, Aarti Munjal, Ph.D.2, Sarah M. 

Kreidler, DPT, M.S.2, Deborah H. Glueck, Ph.D.2

1 Department of Community Dentistry and Behavioral Science, University of Florida College of 

Dentistry

2 Department of Biostatistics and Bioinformatics, Colorado School of Public Health, University of 

Colorado Denver

1 
Co-Authors

 

2 

Anna E. Barón, PhD

Brandy M. Ringham, PhD

Uttara Sakhadeo, MS

Department of Biostatistics & Informatics

University of Colorado Denver

Yi Guo, MSPH, PhD

Keith E. Muller, PhD

Department of Health Outcomes & Policy

University of Florida
Contributors

3 

Zacchary Coker-Dukowitz, MFA

Mildred Maldonado-Molina, PhD, MS

Department of Health Outcomes & Policy

University of Florida
Conflict of Interest

We have no conflicts of interest to declare.

4 
Acknowledgments

The project was supported in large part by the National

Institute of Dental and Craniofacial Research under

award NIDCR 1 R01 DE020832-01A1. 

Startup funds were provided by the National Cancer Institute

under an American Recovery and Re-investment Act supplement

(3K07CA088811-06S) for NCI grant K07CA088811. 

The content is solely the responsibility of the authors, and does

not necessarily represent the official views of the National Cancer

Institute, the National Institute of Dental and Craniofacial

Research nor the National Institutes of Health. 

5 
Poll Question #1

Which of the following best describes your

role at the VA? 

A. Clinician

B. Data scientist / Statistician

C. Student / Intern / Fellow 

D. Other researcher

E. Support staff

6 
Learning Objectives

• Learn a conceptual framework for 

conducting a power analysis.

• Understand how to interact with our free, 

web-based power and sample size 

software.

• Write a sample size analysis.

7 
Agenda

8 

How Do We Choose Sample Size and Power 

for Complex Oral Health Designs?

Dr. Henrietta Logan

10 minutes

Choosing a Hypothesis, Outcomes, and 

Predictors with Our Free, Web-based 

Software 

Dr. Aarti Munjal

10 minutes

Choosing Means, Variances, and Correlations 

with Our Free, Web-based Software

Dr. Sarah M. Kreidler

10 minutes
Agenda

9 

Wrapping it Up: Writing the Grant

Dr. Deborah H. Glueck

10 minutes

Discussion: Question and Answer 10 minutes
10

How Do We Choose Sample Size and 

Power for Complex Oral Health Designs?

Dr. Henrietta Logan

University of Florida
Ethics of Sample Size 

Calculations

• If the sample size is too small, the study 

may be inconclusive and waste resources

• If the sample size is too large, then the 

study may expose too many participants to 

possible harms due to research

11
Perceived Control

Low 

1 2 

3 4 

High Low 

High 

Previous Study on Sensory 

Focus to Alleviate Pain

• Participants categorized 

into four coping styles

• Randomized to one of 

two treatment arms: 

sensory focus

standard of care

• Measured experienced 

pain after root canal

12

(Logan, Baron, Kohout, 1995)

Desired Control
Memory of Pain Trial

Study Design

13
Memory of Pain Trial

Research Question

14
Memory of Pain Trial

Study Population

• Recruit participants who have a high 

desire/low felt coping style

• 30 patients / week

• 40% consent rate for previous studies

15
How do we calculate 

an accurate sample size? 

16
Inputs for Power Analysis

• Type I error rate:

• Desired power:

• Loss to follow-up:

17
Inputs for Power Analysis

• Type I error rate:

• Desired power:

• Loss to follow-up:

18

0.01
Inputs for Power Analysis

• Type I error rate:

• Desired power:

• Loss to follow-up:

19

0.01

0.90 
Inputs for Power Analysis

• Type I error rate:

• Desired power:

• Loss to follow-up:

20

0.01

0.90 

25% 
Agenda

21

How Do We Choose Sample Size and Power 

for Complex Oral Health Designs?

Dr. Henrietta Logan

10 minutes

Choosing a Hypothesis, Outcomes, and 

Predictors with Our Free, Web-based 

Software 

Dr. Aarti Munjal

10 minutes

Choosing Means, Variances, and Correlations 

with Our Free, Web-based Software

Sarah M. Kreidler

10 minutes
22

Choosing a Hypothesis, Outcomes, 

and Predictors with Our Free, 

Web-based Software

Dr. Aarti Munjal

University of Colorado Denver
GLIMMPSE is a user-friendly online tool for 

calculating power and sample size for 

multilevel and longitudinal studies.

http://glimmpse.samplesizeshop.org/ 

GLIMMPSE

23
Salient Software Features

• Free 

• Requires no programming expertise

• Allows saving study designs for later use

• Also available on smartphones 

• Coming soon on iPad

24
Create a Study Design

25
Create a Study Design

Select Guided Mode

26
GLIMMPSE Solving For

Select Guided Mode

27
GLIMMPSE Solving For

Select Guided Mode

Checkmark = complete

Pencil = incomplete

28
GLIMMPSE Solving For

Select Guided Mode

Checkmark = complete

Pencil = incomplete

29
GLIMMPSE Desired Power

30
GLIMMPSE Desired Power

Enter desired power values 

here and click “Add”

31
Enter desired power values 

here and click “Add”

GLIMMPSE Desired Power

32
GLIMMPSE Type I Error Rate

33
GLIMMPSE Type I Error Rate

Enter Type I error rate 

values here and click “Add”

34
Enter Type I error rate 

values here and click “Add”

GLIMMPSE Type I Error Rate

35
GLIMMPSE Predictors

36
GLIMMPSE Predictors

Enter predictors here and 

click “Add”

37
Enter predictor categories 

here and click “Add”

Enter predictors here and 

click “Add”

GLIMMPSE Predictors

38
GLIMMPSE Outcome

39
GLIMMPSE Outcome

Enter outcomes here and 

click “Add”

40
Enter outcomes here and 

click “Add”

GLIMMPSE Outcome

41
42

GLIMMPSE Repeated Measures
43

GLIMMPSE Repeated Measures
44

GLIMMPSE Repeated Measures
Hypothesis

time by treatment interaction

45
GLIMMPSE Hypothesis

46
GLIMMPSE Hypothesis

47
GLIMMPSE Hypothesis

48
GLIMMPSE Hypothesis

49
50

Next step: 

Choosing Means, Variances, and 

Correlations
Agenda

51

How Do We Choose Sample Size and Power 

for Complex Oral Health Designs?

Dr. Henrietta Logan

10 minutes

Choosing a Hypothesis, Outcomes, and 

Predictors with Our Free, Web-based 

Software 

Dr. Aarti Munjal

10 minutes

Choosing Means, Variances, and Correlations 

with Our Free, Web-based Software

Sarah M. Kreidler

10 minutes
52

Choosing Means, Variances, 

and Correlations with Our Free, 

Web-based Software

Sarah M. Kreidler

University of Colorado Denver
• Pilot study

• Similar published research

• Unpublished internal studies

• Clinical experience

53

Where Can I Find Means, 

Variances, and Correlations?
Hypothesis

time by treatment interaction

54
Means

55

Intervention Baseline 6 Months 12 Months

Sensory Focus

(SF) 3.6 2.8 0.9

 

Standard of Care

(SOC) 4.5 4.3 3.0

 

Intervention Difference

(SF - SOC) 

 

Net Difference Over Time

(12 Months - Baseline) 

0.9 1.5 -2.1

 -1.2
Means

56

Intervention Baseline 6 Months 12 Months

Sensory Focus

(SF) 3.6 2.8 0.9

 

Standard of Care

(SOC) 4.5 4.3 3.0

 

Intervention Difference

(SF - SOC) 

 

Net Difference Over Time

(12 Months - Baseline) 

-0.9 1.5 -2.1

-1.2
Means

57

Intervention Baseline 6 Months 12 Months

Sensory Focus

(SF) 3.6 2.8 0.9

 

Standard of Care

(SOC) 4.5 4.3 3.0

 

Intervention Difference

(SF - SOC) 

 

Net Difference Over Time

(12 Months - Baseline) 

-0.9 -1.5 -2.1

-1.2
Means

58

Intervention Baseline 6 Months 12 Months

Sensory Focus

(SF) 3.6 2.8 0.9

 

Standard of Care

(SOC) 4.5 4.3 3.0

 

Intervention Difference

(SF - SOC) 

 

Net Difference Over Time

(12 Months - Baseline) 

-0.9 -1.5 -2.1

-1.2
Means

59

Intervention Baseline 6 Months 12 Months

Sensory Focus

(SF) 3.6 2.8 0.9

 

Standard of Care

(SOC) 4.5 4.3 3.0

 

Intervention Difference

(SF - SOC) 

 

Net Difference Over Time

(12 Months - Baseline) 

-0.9 -1.5 -2.1

-1.2
Variances and Correlations

60

• Consider the sources of variability and

correlation in the study design

▫ Repeated measures within a given participant

will be correlated

▫ Outcome measurements will vary between

participants
Variances and Correlations

61

Correlation Between Outcomes Over Time

Gedney, Logan, and Baron (2003) identified predictors of

the amount of experienced pain recalled over time...One of

the findings was that memory of pain intensity at 1 week

and 18 months had a correlation of 0.4. ...assume that the

correlation between measures 18 months apart will be 

similar to the correlation between measures 12 months

apart. Likewise, the correlation between measures 6 

months apart will be only slightly greater than the

correlation between measures 18 months apart. 
Variances and Correlations

62

Standard Deviation of the Outcome

Logan, Baron, and Kohout (1995) examined whether

sensory focus therapy during a root canal procedure could

reduce a patient’s experienced pain. The investigators 

assessed experienced pain on a 5 point scale both 

immediately and at one week following the procedure. The

standard deviation of the measurements was 0.98. 
63

GLIMMPSE Means

Specifying a Mean Difference
64

GLIMMPSE Means

Specifying a Mean Difference

Choose a timepoint
65

Enter the expected net mean 

difference

GLIMMPSE Means

Specifying a Mean Difference

Choose a timepoint
66

GLIMMPSE Variability

Entering Standard Deviation of the Outcome
Enter the standard deviation 

of the outcome variable

67

GLIMMPSE Variability

Specifying Correlations
68

GLIMMPSE Variability

Specifying Correlations

Enter correlations between 

repeated measures
69

GLIMMPSE Hypothesis Test
70

GLIMMPSE Hypothesis Test
71

GLIMMPSE Calculate Button
72

GLIMMPSE Results
73

Total sample size to achieve at 

least 90% power

GLIMMPSE Results
Total sample size to achieve at 

least 90% power

74

Scale the standard deviation to 1⁄2 

and 2 times to see how it affects 

sample size

GLIMMPSE Results
Total sample size to achieve at 

least 90% power

Scale the standard deviation to 1⁄2 

and 2 times to see how it affects 

sample size

75

GLIMMPSE Results
76

Funding the Planned Study
Agenda

77

Wrapping it Up: Writing the Grant

Deborah H. Glueck

10 minutes

Discussion: Question and Answer 10 minutes
78

Wrapping it Up: Writing the Grant

Dr. Deborah Glueck

University of Colorado Denver
• Aligning power analysis with data analysis

• Justifying the power analysis

• Accounting for uncertainty

• Handling missing data

• Demonstrating enrollment feasibility

• Planning for multiple aims

79

Outline 

Writing the Grant
We plan a repeated measures ANOVA using

the Hotelling-Lawley Trace to test for a time

by treatment interaction. 

80

Sample Size Calculation Summary
We plan a repeated measures ANOVA using

the Hotelling-Lawley Trace to test for a time

by treatment interaction. 

81

Sample Size Calculation Summary
• Type I error rate

• α = 0.01

• Hypothesis test

• Wrong: power = treatment

data analysis = time x treatment

• Right: power = time x treatment

data analysis = time x treatment

82

Aligning Power Analysis with 

Data Analysis
Based on previous studies, we predict

memory of pain measures will have a 

standard deviation of 0.98 and the

correlation between baseline and 6 months

will be 0.5. Based on clinical experience, we

believe the correlation will decrease slowly

over time, for a correlation of 0.4 between

pain recall measures at baseline and 12

months. 

83

Sample Size Calculation Summary
Based on previous studies, we predict

memory of pain measures will have a 

standard deviation of 0.98 and the

correlation between baseline and 6 months

will be 0.5. Based on clinical experience, we

believe the correlation will decrease slowly

over time, for a correlation of 0.4 between

pain recall measures at baseline and 12

months. 

84

Sample Size Calculation Summary
• Give all the values needed to recreate the 

power analysis

• Provide appropriate citation

85

Justifying the Power Analysis
For a desired power of 0.90 and a Type I 

error rate of 0.01, we estimated that we

would need 44 participants to detect a mean

difference of 1.2. 

86

Sample Size Calculation Summary
For a desired power of 0.90 and a Type I

error rate of 0.01, we estimated that we

would need 44 participants to detect a mean

difference of 1.2. 

87

Sample Size Calculation Summary
Power 

88

Accounting for Uncertainty

Mean Difference
Power 

Mean Difference

89

0.90

Accounting for Uncertainty
Power 

Mean Difference

0.90

90

Accounting for Uncertainty
We plan a repeated measures ANOVA using the HotellingLawley Trace to test for a time by treatment interaction. 

Based on previous studies, we predict measures of pain

recall will have a standard deviation of 0.98. The

correlation in pain recall between baseline and 6 months

will be 0.5. Based on clinical experience, we predict that

the correlation will decrease slowly over time. Thus, we

anticipate a correlation of 0.4 between pain recall measures 

at baseline and 12 months. For a desired power of 0.90 and

a Type I error rate of 0.01, we need to enroll 44 participants

to detect a mean difference of 1.2. 

91

Sample Size Calculation Summary 

Draft

 
•

92

Handling Missing Data
•

93

Handling Missing Data
Over 12 months, we expect 25% loss to

follow up. We will inflate the sample size by

25% to account for the attrition, for a total

enrollment goal of 56 participants, or 28 

participants per treatment arm. 

94

Sample Size Calculation Summary
Over 12 months, we expect 25% loss to

follow up. We will inflate the sample size by

25% to account for the attrition, for a total

enrollment goal of 56 participants, or 28 

participants per treatment arm. 

95

Sample Size Calculation Summary
• Is the target population sufficiently large?

• Can recruitment be completed in the 

proposed time period?

96

Demonstrating Enrollment 

Feasibility
• 30 patients per week with a high desire / 

low felt coping style

• 40% consent rate

97

Sample size needed

56

Sample size available

Planned Sample Size vs. 

Available Sample Size
• 30 patients per week with a high desire / 

low felt coping style

• 40% consent rate

98

Sample size needed

56

Sample size available

36

Planned Sample Size vs. 

Available Sample Size

3 week enrollment period
• 30 patients per week with a high desire / 

low felt coping style

• 40% consent rate

99

Sample size needed

56

Sample size available

60

Planned Sample Size vs. 

Available Sample Size

5 week enrollment period
The clinic treats 30 patients per week with

the high desire/low felt coping style. Based

on recruitment experience for previous

studies, we expect a 40% consent rate. At an

effective enrollment of 12 participants per 

week, we will reach the enrollment goal of

56 participants in 5 weeks time. 

100

Sample Size Calculation Summary
The clinic treats 30 patients per week with

the high desire/low felt coping style. Based

on recruitment experience for previous

studies, we expect a 40% consent rate. At an

effective enrollment of 12 participants per 

week, we will reach the enrollment goal of

56 participants in 5 weeks time. 

101

Sample Size Calculation Summary
• Aims typically represent different 

hypotheses

• Maximum of the sample sizes calculated 

for each aim

102

Planning for Multiple Aims
Poll Question #2

What topics would you like to see in the

future?

A. Power and sample size for missing data

B. Power and sample size for mixed models

C. Power and sample size for binary or Poisson

outcomes

D. Choosing cluster sizes for multilevel studies

103
104

Questions?

For more information visit

http://www.SampleSizeShop.org

Or contact Dr. Deb Glueck at

deborah.glueck@ucdenver.edu
Adams, G., Gulliford, M. C., Ukoumunne, O. C., Eldridge, S., Chinn, S., & 

Campbell, M. J. (2004). Patterns of intra-cluster correlation from primary 

care research to inform study design and analysis. Journal of clinical 

epidemiology, 57(8), 785-794.

Catellier, D. J., & Muller, K. E. (2000). Tests for gaussian repeated measures 

with missing data in small samples. Statistics in Medicine, 19(8), 1101-

1114.

Demidenko, E. (2004). Mixed Models: Theory and Applications (1st ed.). 

Wiley-Interscience.

Glueck, D. H., & Muller, K. E. (2003). Adjusting power for a baseline covariate 

in linear models. Statistics in Medicine, 22, 2535-2551.

105

References
Gedney , J.J., Logan, H.L., Baron, R.S. (2003). Predictors of short-term and 

long-term memory of sensory and affective dimensions of pain. Journal of 

Pain, 4(2), 47–55.

Gedney, J.J., Logan H.L. (2004). Memory for stress-associated acute pain. 

Journal of Pain, 5(2), 83–91.

Gurka, M. J., Edwards, L. J., & Muller, K. E. (2011). Avoiding bias in mixed 

model inference for fixed effects. Statistics in Medicine, 30(22), 2696-

2707. doi:10.1002/sim.4293

Kerry, S. M., & Bland, J. M. (1998). The intracluster correlation coefficient in 

cluster randomisation. BMJ (Clinical research ed.), 316(7142), 1455.

106

References
Kreidler, S.M., Muller, K.E., Grunwald, G.K., Ringham, B.M., Coker-Dukowitz, 

Z.T., Sakhadeo, U.R., Barón, A.E., Glueck, D.H. (accepted). GLIMMPSE: 

Online Power Computation for Linear Models With and Without a 

Baseline Covariate. Journal of Statistical Software.

Laird, N. M., & Ware, J. H. (1982). Random-effects models for longitudinal 

data. Biometrics, 38(4), 963-974.

Law, A., Logan, H., & Baron, R. S. (1994). Desire for control, felt control, and 

stress inoculation training during dental treatment. Journal of Personality 

and Social Psychology, 67(5), 926-936.

Logan, H.L., Baron, R.S., Keeley, K., Law, A., Stein, S. (1991). Desired control 

and felt control as mediators of stress in a dental setting. Health 

Psychology, 10(5), 352–359.

107

References
Logan, H.L., Baron, R.S., Kohout, F. (1995). Sensory focus as therapeutic 

treatments for acute pain. Psychosomatic Medicine, 57(5), 475–484.

Muller, K. E, & Barton, C. N. (1989). Approximate Power for RepeatedMeasures ANOVA Lacking Sphericity. Journal of the American Statistical 

Association, 84(406), 549-555.

Muller, K. E, Edwards, L. J., Simpson, S. L., & Taylor, D. J. (2007). Statistical 

Tests with Accurate Size and Power for Balanced Linear Mixed Models. 

Statistics in Medicine, 26(19), 3639-3660.

Muller, K. E, Lavange, L. M., Ramey, S. L., & Ramey, C. T. (1992). Power 

Calculations for General Linear Multivariate Models Including Repeated 

Measures Applications. Journal of the American Statistical Association, 

87(420), 1209-1226.

108

References
Muller, K. E, & Peterson, B. L. (1984). Practical Methods for Computing Power 

in Testing the Multivariate General Linear Hypothesis. Computational 

Statistics and Data Analysis, 2, 143-158.

Muller, K.E., & Stewart, P. W. (2006). Linear Model Theory: Univariate, 

Multivariate, and Mixed Models. Hoboken, NJ: Wiley.

Taylor, D. J., & Muller, K. E. (1995). Computing Confidence Bounds for Power 

and Sample Size of the General Linear Univariate Model. The American 

Statistician, 49(1), 43-47. doi:10.2307/2684810

109

References