# Presentation: 546020

## Statistical Analysis Overview ISession 2 

**Peg Burchinal**
**Frank Porter Graham **
**Child Development Institute, **
**University of North Carolina-Chapel Hill**

## Overview:  Statistical analysis overview I-b

**Nesting and intraclass correlation**
**Hierarchical Linear Models**
**2 level models**
**3 level models**

## Nesting

- Nesting implies violation of the linear model assumptions of independence of observations
- Ignoring this dependency in the data results in inflated test statistics when observations are positively correlated
- CAN DRAW INCORRECT CONCLUSIONS

## Nesting and Design

- Educational data often collected in schools, classrooms, or special treatment groups
- Lack of independence among individuals -> reduction in variability
- Pre-existing similarities (i.e., students within the cluster are more similar than a students who would be randomly selected)
- Shared instructional environment (i.e., variability in instruction greater across classroom than within classroom)
- Educational treatments often assigned to schools or classrooms
- Advantage: To avoid contamination, make study more acceptable (often simple random assignment not possible)
- Disadvantage: Analysis must take dependencies or relatedness of responses within clusters into account

## Intraclass Correlation (ICC)

- For models with clustering of individuals
- “cluster effect”: proportion of variance in the outcomes that is between clusters (compares within-cluster variance to between-cluster variance)
- Example – clustering of children in classroom.  ICC describes proportion of variance associated with differences between classrooms

## Intraclass Correlation

- Intraclass correlation (ICC) – measure of relatedness or dependence of clustered data
- Proportion of variance that is between clusters
- ICC or  = b / (b + w)
- ICC = 0 } no correlation among individuals within a cluster
-       = 1 } all responses within the clusters are identical

## Nesting, Design, and ICC

- Taking ICC into account results in less power for given sample size
- less independent information
- Design effect = mk / (1 +  (m-1))
- m= number of individuals per cluster
- K=number of clusters
-   =ICC
- Effective sample size is number of clusters (k) when ICC=1 and is number of individuals (mk) when ICC=0

## ICC and Hierchical Linear Models

- Hierarchical linear models (HLM) implicitly take nesting into account
- Clustering of data is explicitly specified by model
- ICC is considered when estimating standard errors, test statistics, and p-values

## 2 level HLM

- One level of nesting
- Longitudinal: Repeated measures of individual over time
- Typically - Random intercepts and slopes to describe individual patterns of change over time
- Clusters: Nesting of individuals within classes, families, therapy groups, etc.
- Typically - Random intercept to describe cluster effect

## 2 level HLM Random-intercepts models

- Corresponds to One-way ANOVA with random effects (mixed model ANOVA)
- Example: Classrooms randomly assigned to treatment or control conditions
- All study children within classroom in same condition
- Post treatment outcome per child (can use pre-treatment as covariate to increase power)
- Level 1 = children in classroom
-    Level 2 = classroom
-    ICC reflects extent the degree of similarity among students within the classroom.

## 2 Level HLMRandom Intercept Model 

- Level 1 – individual students within the classroom
- Unconditional Model: Yij  = B0j  + rij
- Conditional Model: Yij  = B0j  + B1 Xij  + rij
- Yij= outcome for ith student in jth class
- B0j= intercept (e.g., mean) for  jth class
- B1= coefficient for individual-level covariate, Xij
- rij= random error term for ith student in jth class,
-    E ( rij) = 0, var (rij) = 

## 2 Level HLMRandom Intercept Model 

- Level 2 – Classrooms
- Unconditional model: B0j  = 00  + u 0j
- Conditional model: B0j = 00 + 01 Wj1 + 02 Wj2 + u 0j
- B0j j= intercept (e.g., mean) for  jth class
-  00 = grand mean in population
-  01  = treatment effect  for Wj, dummy variable indicating treatment status
- -.5 if control; .5 if treatment
-  02  coefficient for  Wj2, class level covariate
- u 0j = random effect associated with j-th classroom
-    E (uij) = 0, var (uij) = 

## 2 Level HLMRandom Intercept Model 

- Combined  (unconditional)
- Yij  =  00  +  u 0j  + rij
- Yij  = B0j  + rij
- B0j  = 00  + u 0j
- Combined  (conditional)
- Yij  =  00 + 01 Wj + 02 Wj2 + B1 Xij +  u 0j  + rij
- Yij  = B0j + B1 Xij + rij
- B0j  = 00 + 01 Wj + 02 Wj2 + u 0j
- Var (Yij ) = Var ( u 0j  + rij ) = (
- ICC = = (

## Example2 level HLM Random Intercepts

- Purdue Curriculum Study (Powell & Diamond)
- Onsite or Remote coaching
- 27 Head Start classes randomly assigned to onsite coaching and 25 to remote coaching
- Post-test scores on writing
- Onsite:    n=196, M=6.70, SD=1.54
-     Remote: n=171, M=7.05, SD=1.64

## Example2 level HLM Random Intercepts

- Level 1:    Writingij  = B0j + B1 Writing-preij + rij
- B1 =.56, se=.05, p<.001
- E ( rij) = 0, var (rij) =  1.67
- Level 2:   B0j  = 00 + 01 Onsitej +  u 0j
-                 00 (intercept- remote group adjusted mean)
- 			 = 3.74, se =.31
-            01(Onsite-Remote difference) = -.37, se=.17, p=.03
- 		 E (uij) = 0, var (uij) = 
- ICC = (
- 

## 2 Level HLM - Longitudinal (random-slopes and –intercepts models)

- Corresponds NOT to One-way ANOVA with random effects
- Example: Longitudinal assessment of children’s literacy skills during Pre-K years
- Level 1 = individual growth curve
-    Level 2 = group growth curve

## Level 1- Longitudinal HLM

- Level 1 – individual growth curve
- Unconditional Model: Yij  = B0j  + B1j  Ageij + rij
- Conditional Model: Yij  = B0j + B1j  Ageij + B2 Xij  + rij
- Yij= outcome for ith student on the  jth occasion
- Ageij = age at assessment for ith student on the  jth occasion
- B0j= intercept for ith student
-  B1j= slope for Age for ith student
- B2= coefficient for tiem-varying covariate, Xij\
- rij= random error term for ith student on the  jth occasion
-    E ( rij) = 0, var (rij) = 

## Level 2 – Longitudinal HLM

- Level 2 – predicting individual trajectories
- Unconditional model: B0j  = 00  + u 0j
-           			 B1j  = 10  + u 1j
- Conditional model: B0j = 00 + 01 Wj1 + 02 Wj2 + u 0j
- 			             B1j = 10 + 11 Wj1 + 12 Wj2 + u 1j
- B0j= intercept for ith student
-       B1j= slope for Age for ith student
-  00 = intercept in population
- 10 = slope in population
-  01  = treatment effect  on intercept for Wj, student -level covariate
- 11  = treatment effect  on slope for Wj, student -level covariate

## Level 2 – Longitudinal HLM

- Level 2 – predicting individual trajectories
- Unconditional model:	B0j  = 00  + u 0j
-           			B1j  = 10  + u 1j
- Conditional model: 	B0j = 00 + 01 Wj1 + u 0j
- 			             	B1j = 10 + 11 Wj1  + u 1j
- u 0j = random effect for individual intercept
-    u 0j = random effect for individual slope
- E (u0j) = 0, var (u0j) = 
-    E (u1j) = 0, var (u1j) = 
- cov u 0j, u 1j)  = 
-    var u 0j, u 1j)=
- 			
- level 1 and 2 error terms independent
-           cov  (rij, T) = 0

## Example – Longitudinal HLM

- Purdue Curriculum Study (Powell & Diamond)
- Level 1 – estimating individual growth curves for children in one treatment condition (Remote)
- Level 2 – estimating population growth curves for Remote condition

| Blending | Pre | Post | Follow-up |
| --- | --- | --- | --- |
| N
M (sd) | 187
9.48 (5.34) | 171
13.75 (4.57) | 63
15.14 (4.60) |

## Example

- Level 1: blendingij =  B0j  + B1j  Ageij + rij
-  estimated
- Level 2: B0j = 00 + 01 Wj1 + u 0j
- 		         	B1j = 10 +  u 1j
- Estimated results
- Intercept  00 = 11.86 (se=.48), 00 = 10.03**
- season      01 = 2.43* (se=.70)
- Slope        10  = 1.51* (se=.60), 11 = 4.24** 					    	 10 = -1.45**

## 3 level HLM 

- 2 levels of nesting
- Examples
- Longitudinal assessments of children in randomly assigned classrooms
- Level 1 – child level data
- Level 2 – child’s growth curve
- Level 3 – classroom level data
- Two levels of nesting such as children nested in classrooms that are nested in schools
- Level 1 – child level data
- Level 2 – classroom level data
- Level 3 – school level data

## 3 level Model-Random Intercepts

- Children nested in classrooms, classrooms nested in schools
- Level 1 child-level model  Y*ijk** = ****ojk** + e**ijk*
- Yijk is achievement of child I in class J in school K
-  ***ojk *is mean score of class j in school k
-  e*ojk *is random “child effect”
- Classroom level model  ***ojk**  = ****00k** + r**0jk*
-  ***00k* is mean score for school k
-  *r**0jk  *is random “class effect”
- School level model ***00k** = ****000** + u**00k*
-  ***000* is grand mean score
-  *u**00k   *is random “school effect”

## 3 level Model-Random Intercepts

- Children nested in classrooms, classrooms nested in schools
- Level 1 child-level model  Y*ijk** = ****ojk** + e**ijk*
- e*ojk *is random “child effect”,
- E (eijk) = 0 , var(eijk) = 
- Within classroom level model  ***ojk**  = ****00k** + r**0jk*
*r**0jk  *is random “class effect”,
- E (r0jk ) = 0 , var(r0jk ) = 
- Assume variance among classes within school is the same
*Between classroom  (school) ****00k** = ****000 + ****01 **trt + u**00k*
- E (*u**00k* ) = 0 , var(*u**00k* ) = 

## Partitioning variance

- Proportion of variance within classroom
- 
- Proportion of variance among classrooms within schools
- 
- Proportion of variance among schools
- 

## 3 Level HLM – level 2 longitudinal and level 3 random intercepts

- Typically – treatment randomly assigned at classroom level, children followed longitudinally (e.g., Purdue Curriculum Study)
- (within child) Level 1: Yijk  = 0j k + 1j k Ageijk + rijk
- E (eijk) = 0 , var(eijk) = 
- (between child ) Level 2:
- 0jk  = 00k  + r 0jk;      1j k = 10k + r 1jk
- E (r0jk ) = 0 , var(r0jk ) = E (r1jk ) = 0 , var(r1jk ) = 
- (between classes) Level 3:
***00k** = ****00** + u**00k;      ****10k** = ****10** + u**10k*
- E (*u**00k* ) = 0 , var(*u**00k* ) = E (*u**10k* ) = 0 , var(*u**10k* ) = 

## Example Purdue Curriculum Study

- Level 1 – individual growth curve
- Level 2 – classroom growth curve
- Level 3 – treatment differences in classroom growth curves

| Writing | Pre | Post | Follow-up |
| --- | --- | --- | --- |
| Onsite 
M (se) | N=199
5.98 (1.49) | N=196
6.70 (1.54) | N=79
6.92 (1.74) |
| Remote
M (se) | N=187
6.01 (1.55) | N=171
7.04 (1.64) | N=63
7.48 (1.62) |

## Purdue Curriculum Study

## Threats

- Homogeneity of variance – at each level
- Nonnormal data with heavy tails
- Bad data
- Differences in variability among groups
- Normality assumption
- Examine residuals
- Robust standard error (large n)
- Inferences with small samples

## 3 Level HLMLongitudinal assessments of individual in clustered settings