Using R and Shiny to Analyze

National Assessment Data

NCES Lead: Emmanuel Sikali

Team Leads: Paul Bailey & Ting Zhang

Development Team: Michael Lee, Huade Huo, Trang Nguyen, Thomas

Fink, & Sun-Joo Lee

Quality Control Team: Michael Cohen, Jiayi Li, Yuqi Liao, & Qingshu Xie

September 2019
Agenda

1. Context

2. Statement of problem

3. Demo R Package - EdSurvey

4. Demo Shiny App - interactive NCES Results Display (iNRD)

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 2 / 30
Context - NCES

and NAEP

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org
Context - NCES and NAEP

The National Center for Education Statistics (NCES) is the primary federal

entity for collecting and analyzing data related to education in the U.S.

and other nations.

The National Assessment of Educational Progress (NAEP) is the largest

nationally representative and continuing assessment of what America's

students know and can do in various subject areas.

Analyzing NAEP data for students achievement involves:

Using plausible values to account for uncertainty in the estimates of

student test scores; and

Employing special methods (e.g., jackknife) that account for the

NAEP complex sampling design.

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 4 / 30
Statement of

Problem

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org
Statement of Problem

1. Cost: requires specialized statistical software

2. Complexity:

survey data size can be unwieldy on some machines

custom code required to perform analyses

3. Access Limitations: impediments to downloading and reading-in

data

4. Inefficiency: cognitive burden creating and reproducing analyses

5. Software Fatigue: data downloading, processing, manipulation and

analysis all done in different software

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 6 / 30
Statement of Problem

Analyzing NCES data is too costly, complex,

inaccessible, and burdensome for researchers.

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 7 / 30
EdSurvey

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org
What is EdSurvey?

An R statistical package that gives users functions to connect

seamlessly with NCES data to perform analyses that take into account

complex sample survey design and the use of plausible values.

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 9 / 30
Why EdSurvey?

1. One-stop shop for data downloading, processing, manipulation

and analysis of survey data.

2. Automated: Weights and complex sampling design calculations

are automated following standard NCES methodology.

3. Simple: e.g., a regression with 90 replicate weights requires only a

few lines of code.

4. Flexible: You can use functions that rely on EdSurvey methods or

get the data and use traditional R.

5. Minimizes memory footprint by only reading in required data.

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 10 / 30
One-stop Shop for NCES Survey Analysis

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 11 / 30
One-stop Shop for NCES Survey Analysis

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 12 / 30
One-stop Shop for NCES Survey Analysis

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 13 / 30
One-stop Shop for NCES Survey Analysis

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 14 / 30
Demo -

EdSurvey

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org
Installing the EdSurvey Package

# to install the package

install.packagesル"EdSurvey"レ

# to load the package

libraryルEdSurveyレ

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 16 / 30
Data

Processing

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org
Data Processing

First, read in the data

sdf <- readNAEPルsystem.fileル"extdata/data", "MëÜNTäPM.dat", package = "NAEP

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 18 / 30
Meet Your Data - print

printルレ

Print returns detailed data file information:

printルsdfレ

## edsurvey.data.frame for ä00ö NAEP ルMathematicsレ in USA

## 

## Dimensions: ÜâÜ0Ü rows and ë0ä columns.

## 

## There are Ü full sample weightルsレ in this edsurvey.data.frame

## 'origwt' with Üä JK replicate weights ルthe defaultレ.

## 

## There are Ü subject scaleルsレ or subscaleルsレ in this edsurvey.data.frame

## 'num_oper' subject scale or subscale with ö plausible values.

## 'measurement' subject scale or subscale with ö plausible values.

## 'geometry' subject scale or subscale with ö plausible values.

## 'data_anal_prob' subject scale or subscale with ö plausible values.

## 'algebra' subject scale or subscale with ö plausible values.

## 'composite' subject scale or subscale with ö plausible values ルthe defaultレ.

## 

## Omitted Levels: 'Multiple', 'NA', 'Omitted'

## 

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 19 / 30
Data Exploration and Cross Tabulation

Summary table of NAEP composite mathematics performance scale

scores ( composite ) of 8th grade students by two student factors:

dsex : gender

b0ÜâïöÜ : frequency of talk about studies at home

esÜ <- edsurveyTableルcomposite ~ dsex + b0ÜâïöÜ, data = sdfレ

Male Never or hardly ever 2350 2434.844 29.00978 0.6959418 270.8243 1.057078

Male Once every few weeks 1603 1638.745 19.52472 0.5020657 275.0807 1.305922

Male About once a week 1384 1423.312 16.95795 0.5057265 281.5612 1.409587

Male 2 or 3 times a week 1535 1563.393 18.62694 0.4811497 284.9066 1.546072

Male Every day 1291 1332.890 15.88062 0.5872731 277.2597 1.795784

Female Never or hardly ever 1487 1517.609 18.20203 0.5078805 266.7897 1.519020

dsex b017451 N WTD_N PCT SE(PCT) MEAN SE(MEAN)

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 20 / 30
Linear Regression

fits a linear model formula using sampling weights and variance

estimates:

dsex : gender

b0ÜâïöÜ : frequency of talk about studies at home

lmÜ <- lm.sdfルcomposite ~ dsex + b0ÜâïöÜ, 

 weightVar = 'origwt', data = sdfレ

summaryルlmÜレ

## 

## Formula: composite ~ dsex + b0ÜâïöÜ

## 

## jrrIMax: Ü

## Weight variable: 'origwt'

## Variance method: jackknife

## JK replicates: Üä

## full data n: ÜâÜ0Ü

## n used: ÜÜëëÜ

## 

## Coefficients:

## coef se t dof Prル>|t|レ 

## ルInterceptレ äâ0.ïÜÜÜä Ü.0äïïë äÜë.9ÜÜï9 öï.Üâ0 < 0.000000000000000ää ***

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 21 / 30
EdSurvey Calls Network Connection

Small Memory Footprint

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 22 / 30
Extensive Function Library

Correlations

Proportion meeting expectations

Regression

Logit/probit

Quantile regression

Percentiles

Mixed models

Gap trend analysis

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 23 / 30
Learning EdSurvey

There are many ways to learn about how to use EdSurvey:

Frequently offered workshops

Reading vignettes on the EdSurvey Website

(https://www.air.org/page/edsurvey-installation-and-use)

Accessing vignettes via the package (requires internet)

vignetteル"introduction", package="EdSurvey"レ

R help

helpルpackage = "EdSurvey"レ

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 24 / 30
iNRD

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org
iNRD

A Shiny exploratory tool that visualizes results from one of the

NCES data -National Assessment of Educational Progress (NAEP)-

across year

The iNRD is built using the R statistical software and it features an

interactive interface with dropdowns, checkboxes and slider bars

for ease of use

Users can explore NAEP results and create customized plots

including charts and maps that display NAEP national, state, and

TUDA performances across multiple years, and broken down

across a variety of student groups. Users can also download and

save their plots to the png, pdf, or svg format.

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 26 / 30
iNRD

Some current NCES data visualization tools are cumbersome

In NAEP Data Explorer and International Data Explorer, a user

has to follow certain procedure to achieve a goal. Any mistake in

a intermediate step will mess up the whole process, and the

user has to go back to the beginning and restart the analysis.

Does not support sophisticated data exploration

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 27 / 30
iNRD

Trend in fourth-grade NAEP mathematics average scores by Gender

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 28 / 30
iNRD

Virginia fourth-grade NAEP mathematics achievement levels by race

(black)

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 29 / 30
iNRD

Percent of Fourth-grade students at or above Proficient in Reading by

IEP status (Y)

 NAEP DATA TRAINING WORKSHOP | NAEP-Research.airprojects.org 30 / 30