Funded by the VA HSR&D (Project# SDR 10-012)

Spotlight on Women Cyberseminar Series
Hierarchical Research De

Hierarchical Research De

Design Strategies and Statistical Appro

Martin Lee, PhD 

Center for the Study ofHealthcare Provider Behavior
Example: Influences on Delivery of Heal

Environment

Organization

& Practice Provider P

MM

Healthcare Delivery

Health Outcomes
Backgr

• Healthcare delivery interventions: Process o

– clinical guideline or clinical pathway 

implementation

– collaborative care models

– clinical care reorganization– managed care practice adoption (incentive

provider education, etc.)

– require relatively complex research design

sampling methods
Types of Interven

• Act upon sometimes diverse provider groups 

individuals

• Act upon the health care environment• Frequently cross general medicine and specia

Frequently cross general medicine and specia

group lines of authority

• Require involvement of clinical and administ

Require involvement of clinical and administ

leadership
Goals of Today’s Discu

• Review key research designs

– eg, repeated measures, group randomizat

eg, repeated measures, group randomizat

• Review suitable sampling techniques

– eg hierarchical sampling of patients with

eg, hierarchical sampling of patients with

providers within clinics

– address issues of power and sample size address issues of power and sample size

• Discuss statistical methods and software pro

suited to these desi

g p ns/sam

ples
Process of Care Interven

Why Comp

• Interventions implemented at provider level o

(eg, clinic, hospital, hospital group, etc.)

• O t d t l l l hi hi 

Outcome measured at lower level = hierarchi

• Randomization of interventions at levels othe

outcomeWhy?

–

“contamination” effect

– difficulty in implementing multiple 

interventions in one facility

– preferred technique in given facility

• Small number of groups per intervention
Exam

• Rapid Early Action for Coronary Treatment (REAC

(1995)

– goal to reduce time in seeking medical attention post goal to reduce time in seeking medical attention post-MI– intervention: includes mass-media campaign– randomize by city

• N ii d i Nutrition education

– goal to increase awareness of reduced fat/salt diet– intervention: label menus in restaurant– randomize by restaurant

• QUITS Study

– goal t i it t t l to increase quit rates among veterans

– intervention: EBQI guideline implementation– randomize by medical center
Design I

• Group randomization (similar to cluster o

hierarchical random samp g) y ling): may not be able to 

data as simple random samples

– sampling unit ≠ analytical unit– unit of randomization ≠ unit of analysis

– implications for sample size/power and a

– “problem” clustering of data may not res

from design 

• do we “test” for it? (conditional analysis)
Design I

• Matching/stratification

– only a few (usually <10-20) group to be random– pair-mat hi ? diffi lt ith l f 

tching?: difficult with only a few group

advantage is gain in power:

σ2diff = 2σ2within σ (1 - ρm) (ρm = correlation bet

diff σ within ( ρm) (ρm co e o be 

matching factor & o

with group-randomized, ρ > 0.3 or number of groups > 1usef l (M ti 1993 Di h 1995) ful (Martin, 1993; Diehr, 1995)

– stratification?: hard to detect stratum-by-outcom

interaction

– probably a good idea anyway because of small n

groups; match if close agreement is possible
Design I

• Alternatives to matching/stratificati

– post-hoc stratification (but stratification d

hoc stratification (but stratification d

in advance)

– reg j ression adjustment for covariates 

(ANCOVA)
Design I

• Repeated measures

– improve precision of within group data improve precision of within group data– t = 2 (“pre-test/post test” or panel design)– t > 2 (usual repeated measures design) t > 2 (usual repeated measures design)– analytical issues
Sample Size Considera

• Use “inflation” factor based on design effec

clustering( ) Donner & Klar, 2000)

• Fixed number of clusters/groups
Sample Size Considera

• Easy method to employ

– Inflation factor = 1 + (m - 1) ρ to tradit

sample 

( )

where m = average cluster size

ρ = intracluster correlation coeff

sample formula

ρ

= variation between cluste

variation between variation between + variatio

variatio

• Note significance of:

– m = 1 or m = 1 or ρ = 0– small ρ, but large m
Sample Size Considera

How do we obtain these values

m = average number of units to be samp

each l t (if thi b i id 

h cluster (if this number varies wid

may need more exact approach using “

ρ = difficult to estimate; use survey info
INTRACLASS CORRELAT

EXAMPLES FROM PUBLISHED & UNPUBLISH

Published intraclass correlations:

General practice General practice • Fahey & Peters, BMJ 1996; 313:93-6 (prop pts controlled htn) • McKinley et al, BMJ 1997;314:190-3

SF36 scores

Physical functioning Role physical Role emotional Social functioning Social functioning Mental health Energy and vitality Pain General health perception 

Satisfaction scores

Communication Attitude of doctor Attitude of doctor Continuity of care Delay of visit Overall satisfaction 
Unpublished Data:

General practice IC• North of England Study of Standards & Performance in Gen Pract (SPGP)

Medical history recorded? 0.1Child examined? <0.Investigations recorded? 0.0

Previous diagnosis recorded? Previous diagnosis recorded? <0.Diagnosis recorded? <0.Previous drug management recorded? <0.Drug management recorded? <0.Non-drug management recorded? 0.0Ad i d d? Advice recorded? 0 1

.

Referral decision recorded? <0.Follow-up decision recorded? 0.0Reasons for management recorded? 0.1

•Aberdeen Gramp j( ) ian Referral Initiatives Project (GRIP)

Appropriateness of referral 0.0Number of annual referrals 0.2

•Aberdeen Urological Guideline Evaluation Project (URGE)SF36 scores

Physical functioning Physical functioning 0 0

.

Role physical 0.0Role emotional 0.0Social functioning 0.0Mental health 0.0Energy and vitality 0.0Pain 0.0General health perception 0.0
Methodological Approaches for An

Example formula:• Comparison of two means ( Comparison of two means (no matchin

no matchin

n = 

Si (k) th k /

(z α/2 + z β)2 (2σ2) [1 + (m - 1)ρ]

(μ1 - μ2)2

• Since n = m(k), then k = n/m

C i ft ( hi )

(# of groups)

• Comparison of two means (matching)

nmatch = n (1 - ρM) (conservative: set 
Methodological Approaches to An

• Efficiency of increasing number of clu

versus increasing number per cluster versus increasing number per cluster

var (y ) 2 2 1+( 1) 1 - y2) = 2 σ2 1 + (m - 1) ρ

km

– if k --> large, var --> 0– if m --> large, var --> 2 σ2 ρ

k
Methodological Approaches to An

• Fixed number of clusters, kmax:

m = (1- ρ) / k max - ρ

k m=1

where k m 1= = “usual” samp( 

le size (no c

-- notice restriction on k max:

k max > ρ k m=1
Methodological Approaches to An

I. USE SUMMARY STATISTIC, e.g., mean, FOR A

UNITS IN A GIVEN CLUSTER AND THEN UTTRADITIONAL STATISTICAL METHODOLOTHESE SUMMARY STATISTICS

Example: INTERVENTION #1 INTERVENTION #2

x clinic11 x clinic 21

• •• •• •

x cli i 1 n c n

1 x cli i 2 n c n

2

Compare the means using t-test, say THIS MODEL ALLOWS FOR 

CLUSTER-LEVEL COVARIATE ADJUSTMENT

BUT

NO AVAILABILITY OF SUBJECT LEVEL ADJUSTMENTi.e., information wasted
Methodological Approaches to An

• If two time points:

– use post use post-pre as measurement for analysis pre as measurement for analysis– better analyze post-data, adjusted for pre

with ANOVA with ANOVA

– better still, adjust individual data for cova

by regression model, then ANCOVA by regression model, then ANCOVA

– even better, adjust by regression model, t

repeated measures ANOVA

– best?
Methodological Approaches to An

II INFLATE STANDARD ERROR OF USUAL TESII. INFLATE STANDARD ERROR OF USUAL TES

STATISTIC BY AN AMOUNT EQUAL TO

1 + (m - 1) ρ (use 1 + (m - 1)ρ for χ2 and F

_

_ 1 (m 1) ρ (use 1 + (m 1)ρ for χ and F 

for 2-sample z or t-tests

x1 - x2

t = 1 1 1 + (m - 1) ρ

n n

_

S __ + __ w

n1 n2

S2b

Where ρ is estimated as: S2 S2

Obtain f

Where ANOVA 

ρ is estimated as: S b + S w

Problem: Tends to be conservative; examine with & with
EXAMPLE ON THE USE OF THE “CL

CORRECTED” T-TEST (Kis

• exami i diff i i b h 

ining differences in average income between homeowner

• 40 neighborhoods (clusters)• total sample of 400/group

• m = 40• estimate of intracluster correlation=.201

So 1 + (m - 1) r = 2 97

_

_

________________

So, 1 + (m 1) r 2.97

Here x homeowner = $40,000 x renter = $35,000

S w = $16,000

S t 40 000 35 000

_

___

So t =40,000 - 35,000

16,000 1 + 1 (2.97)

400 400

___ ___

______________________________

= 1.49 (P=.14)(UNINFLATED: T = 4.42, p < < .001!!)
Methodological Approaches to An

III. USE MIXED MODEL ANALYSIS OF 

VARIANCE (quantitative data)

• Unit of analysis becomes repeat factor and cl

unit becomes a “nested” factor within “interv

group

• Sometimes thought of as a random coefficien

regressi dl on model

• Relationship between outcome variable and c

may ( dd ) f l t t l t (and does) vary from cluster to cluster

• Can use with repeated measures/covariates
Methodological Approaches to An

IV. HUBER CORRECTION TO STANDAR

ERRORS

• Bootstrap/Jackknife approach (nonparam• Used in connection with: Used in connection with:

– standard regression– logistic regression– Cox regression– Others

• Implemented in STATA
Methodological Approaches to An

V. OTHERS (similar to repeated measures analysi• Panel data analysis• MANOVA• Growth curve models (like random coeff• Bayesian/Empirical Bayesian
Sof

• STATA (define cluster variable in regression proced• SUDAAN (generalized estimation equations)• WESREG (SAS procedure callable in v. 6.08 and h• BMDP 3V/5V (repeated measures)• SAS PROC MIXED (not PROC GLM; works onl

data are balanced and covariates are “well-behaved”)

• Others 

– GENMOD– HLM (2 or 3 HLM (2 or 3-level hierarchies) level hierarchies)– ML3 (2 or 3-level hierarchies)– VARCL (up to 9-level hierarchies)
Example: Kerr et al 

DOES DISSATISFACTION WITH ACCESS TO SPEDOES DISSATISFACTION WITH ACCESS TO SPE

AFFECT THE DESIRE TO LEAVE A MANAGED CAR

• Surveyed >120 physician groups throughou

California 

( p p) from 

particular health 

plan

)

• Obtained 17,196 patients from within these • Examine

d

– Satisfaction scales (quantitative)– Desire to change health plan (binary)

• Huber regression using average number of 

enrollees per group
Exam

STUDY OF THE RATE OF ADHEREN

TREATMENT REGIMENS IN VETERANS WI

• Randomize clinics at regional level 

– San Diego Greater Los Angeles Tucson 

San Diego, Greater Los Angeles, Tucson 

Palo Alto

• Sample size calculation for t Sample size calculation for t-test

comparison inflated by a factor of a1.5 to account for almost certain 

clustering effect

• Allowance for matching/stratificatio
Stepped Wedge D

• Type of crossover design• Different clusters crossover (1 directio

Different clusters crossover (1 directio

different time points

• Fi t ti i t b li First time point: baseline

• Then, different clusters initiate the 

intervention at different time points (oby randomization mechanism)
Stepped Wedge D

Cluster Time 112

11

Cluster Time

1

Time 2

Time 3

Tim4

1 0 1 1 1 

234

100

1234

0000

1000

1100

1110

P ll l Cluster Time 1 Time 2

4 0 0 0 0 

Parallel group

X-over

Cluster Time 1 Time 2123

110

001 Stepped Wedge 34 00 11
Stepped Wedge Design: Advan

• Parallel/x-over designs: intervention implem

in half of all clusters simultaneously y : ma b

logistically impossible; stepped wedge allowlimited rollout

• Only one direction for treatment interventio

removed); however, complicates analysis 

(t t t ff t t b ti t d f (treatment effect cannot be estimated from w

cluster comparisons)
Example: Golden et al, NEJM, 

• Partner notification of patients with STDs• Standard: public health authorities notify Standard: public health authorities notify• Intervention: patients given drugs/drug vouc

to g p ive to partners

• Implemented in one county in WA and then 

randomly p rolled out in multiple counties ove
Statistical M

• Yijk = response corresponding to individual 

time j from cluster I

• μ ij = μ + αi + βj + Xijθ, where αi is the rando

effect for cluster i, βj is the fixed effect for t

interval j, Xij is an indictor variable for treat

and θ is the treatment effect

• Yijk = μ ij + eij• Usual assumptions on the random effects
Statistical Ana

• Linear mixed models• Generalized linear mixed models ( 

no norma

• Generalized estimating equations (allows fo

misspecification of the variance-covariance 

stt) ruc ure)

• Research ongoing to compare these approac
Why this talk for

VA women’s health researc

Top priority based on national needassessment:

Study design, sampling strategies (into achieve adequate #s of women veteransto achieve adequate #s of women veteransStatistical analysis (and power calculSupport for multi Support for multi

-site research site research

Dr. Lee pre-eminent biostatistician

And core faculty of VA Women’s HeaResearch Consortium
VA Women’s Health Resea

Application of Session Con

Gender sensitivity 

implementation trial Specialty clinics

(Vogt/Yee)

Use EBQI to Use EBQI to 

adapt, deploy Women’s Health 

CBOC

and evaluateimplementation

Clinic

effectiveness Emerg

CBOC Roo

VA Women’s Health PBRN Implementation Evaluation Project
Example of VA

Multi-Level Implementatio

VISN

F ilit ac

y

Clinic

Provide

r

Patient 
VA Women’s Health Resea

Application of Session Con

• I t f ti Impac

t o

f practice 

structure on quality of 

care for women veterans

Area

care for women veterans

(Yano, Bean-Mayberry, 

Washington)

Facility

• Examined associations of 

care models and features

Clinic

care models and features

on patient care

–

A

djusted for facilit

y and area 

Patien

j y

characteristics

– Adjusted for patient 

cl ti us

tering 
Thank you! Thank you!