Middle	Grades	Longitudinal	Study	of	2016–17	(MGLS:2017)

Recruitment for

2016	Item	Validation	and	Operational	Field	Tests

Request	for	OMB	Clearance

OMB#	1850-0911	v.3

Supporting	Statement	Part	A

National	Center	for	Education	Statistics

U.S.	Department	of	Education

Institute	of	Education	Sciences

Washington,	DC

February 2015
1

Table	of	Contents

Preface.............................................................................................................................................................................................................2

A.	Justification...............................................................................................................................................................................................2

A.1	Importance	of	Information.........................................................................................................................................................2

A.2	Purposes	and	Uses	of	Data..........................................................................................................................................................4

A.3	Improved	Information	Technology	(Reduction	of	Burden).........................................................................................8

A.4	Efforts	to	Identify	Duplication ..................................................................................................................................................8

A.5	Minimizing	Burden	for	Small	Entities....................................................................................................................................9

A.6	Frequency	of	Data	Collection.....................................................................................................................................................9

A.7	Special	Circumstances..................................................................................................................................................................9

A.8	Consultations	Outside	NCES ......................................................................................................................................................9

A.9	Payments	or	Gifts	to	Respondents ....................................................................................................................................... 11

A.10	Assurance	of	Confidentiality................................................................................................................................................ 15

A.11	Sensitive	Questions.................................................................................................................................................................. 15

A.12	Estimates	of	Burden ................................................................................................................................................................ 15

A.13	Total Annual	Cost	Burden..................................................................................................................................................... 16

A.14	Annualized	Cost	to	Federal	Government........................................................................................................................ 16

A.15	Program	Changes	or	Adjustments..................................................................................................................................... 16

A.16	Plans	for	Tabulation	and	Publication............................................................................................................................... 17

A.17	Display	OMB	Expiration	Date .............................................................................................................................................. 17

A.18	Exceptions	to	Certification	Statement.............................................................................................................................. 17
2

Preface

The	MGLS:2017	will	be	the	first	study	sponsored	by	the	National	Center	for	Education	Statistics	(NCES),	

within	the	Institute	of	Education	Sciences	(IES) of	the	U.S.	Department	of	Education,	to	follow	a	nationally

representative	sample	of	students	as	they	enter	and	move	through	the	middle	grades	(grades	6–8).	In	

preparation	for	the	main study,	the data	collection	instruments	and	procedures	must	be	field	tested.	This	

submission	describes	the	overarching	plan	for	the	recruitment	of	schools, school	districts,	and	parents	to	

participate	in	the	MGLS:2017	Item	Validation	Field	Test	and	the	MGLS:2017	Operational	Field	Test.	A	

separate	OMB	clearance	request	for	the	field	test	data	collection	will	be	submitted	that	more	specifically	

covers	the	field	test	data	collection	methodology	and	provides	more	technical	detail	on	sampling.

This	package	requests	clearance	to	recruit	schools	and	participants for	two	field	tests	for	the	Middle	Grades	

Longitudinal	Study	of	2016–17 (MGLS:2017):	(1) the	Item	Validation	Field	Test and	(2)	the	Operational	Field	

Test.	The	primary	purpose	of	the	Item	Validation	Field	Test is	to	determine	the	psychometric	properties	of	

items	and	the	predictive	potential	of	assessment	and	survey	items so	that valid,	reliable, and	useful	

assessment	and	survey instruments can	be	composed for	the	main	study.	The	primary	purposes of	the	

Operational	Field	Test are	to	obtain	information	on	recruiting,	particularly	for	the	targeted	disability	groups;

on	obtaining	a	tracking	sample	that	can	be	used	to	study	mobility patterns in	subsequent	years; and	on	

administrative	procedures.

Part	A	of	this	submission	presents	information on	the	basic	design	of	the	Item	Validation	Field	Test and	the	

Operational	Field	Test.	Part	B	presents	information	on	the	collection	of	information	employing	statistical	

methods.	In	addition	to	the	supporting	statements in	Parts A	and	B,	Appendices	A	through	T provide	field	

test	recruitment	and	student	roster	collection	materials,	consisting	of	letters	to	state	and	district	officials,	

school	principals,	and	parents,	as well	as	text	for	an MGLS:2017	brochure,	frequently	asked	questions, and

website.	Additionally,	as	some	schools	and/or	districts	may	require	a	submission	of	a	research	application	

that	requests	detailed	information	on	the	proposed	assessments	and	surveys,	Appendices	K through	R

provide	content	summaries of	the	proposed	assessments	and	surveys,	and	Appendix	U	provides	the	current	

drafts	of	the	survey	instruments.	The	final	versions	of	the	surveys	to be	used	in	the 2016 field	tests	will	be	

provided	later	in	2015	in	the	request	for	the	field	test	data	collection.

A change	request	may follow	this submission.	The	data	collection	contractor,	upon	anticipated	award	in	June	

2015	of	the	MGLS:2017	contract	that covers	the	2016	field	tests	through	the	main	study’s 7th-grade	

collection,	may	propose	revisions to	the	recruitment	effort	in	July	or	August	of	2015.

A. Justification

A.1 Importance	of	Information

The	MGLS:2017	will	be	the	first	study	sponsored	by	NCES to	follow	a	nationally representative	sample	of	

students	as	they	enter	and	move	through	the	middle	grades	(grades	6–8).	A	study	of	the	middle	grades	will	

complement NCES’s	plans	for	implementing	a	multi-cohort	sequence	for	its	longitudinal	studies	series.	This	

means	that	the Early	Childhood	Longitudinal	Study,	Kindergarten	Class	of	2010–11	(ECLS-K:2011),	the	

MGLS:2017,	and	the	High	School	Longitudinal	Study	of	2020	(HSLS:2020)	will	synchronize, and	within	a	

given	10-year	span, collect	the	full	range	of	data	on	students’	school	experiences	as	they	transition	from	

elementary	school	into	high	school. The	federal	government	is	uniquely	positioned	to	undertake	the	needed	

comprehensive	large-scale	longitudinal	study	of	a	nationally	representative	sample	of	middle-grade youth

that	includes	measures	of	known	critical	influences	on	adolescents’	academic	and	socioemotional	
3

trajectories.	NCES	is	authorized	to	conduct	the	MGLS:2017	under	the	Education	Sciences	Reform	Act	of	2002	

(20	U.S.	Code,	Section	9543).

Baseline	data	for	the	MGLS:2017	will	be	collected	from	a	nationally	representative	sample	of	6th-grade	

students	in	January	through	March	of	winter	2017,	with	annual	follow-ups	in	the	winters of	2018	and	of	

2019,	when	most	of	the	students	in	the	sample	will	be	in	grades	7	and	8,	respectively.	The	MGLS:2017 will	

provide	a	rich	descriptive	picture	of	the	academic	experiences	and	development	of	students	during	these	

critical	years	and	will	allow	researchers	to	examine	associations	between	contextual	factors	and	student	

outcomes.	There	is	a	wealth	of	research	highlighting	the	importance	of	mathematics	and	literacy	skills	for	

success	in	high	school	and	subsequent	associations with	later	education	and	career	opportunities. Thus, the	

study	will	focus	on	student	achievement	in	these	areas, along	with	measures	of	student	socioemotional	wellbeing	and	other	outcomes.	The	study	will	also	include	a sample	of	students	with	different	types	of	disabilities	

that	will	provide descriptive	information	on	their	outcomes,	educational	experiences,	and	special	education	

services.

The	MGLS:2017	will	rely	on	a set	of	longitudinal	and	complementary	instruments	to	collect	data	across	

several	types	of	respondents to provide	information	on	the	outcomes,	experiences,	and	perspectives	of	

students	across	grades	6,	7,	and	8;	their	families	and	home	lives;	their	teachers,	classrooms,	and	instruction;	

and	the	school	settings,	programs,	and	services	available	to	them. At	each	wave	of	data	collection in	the	main

study,	students’ mathematics	and	reading	skills,	socioemotional	development,	and	executive	function	will	be	

assessed.	Students will	also	complete	a	survey	that	asks	about	their	engagement	in	school,	out-of-school	

experiences,	peer	group	relationships,	and	identity	development.	Parents	will	be	asked	about	their	

background,	family	resources,	and	involvement	with	their	child’s	education	and	their	school.	Students’	

mathematics	teachers	will	complete	a	two-part	survey:	In	part	1,	they	will	be	asked	about	their	background	

and	classroom	instruction.	In	part	2,	they	will	be	asked	to	report	on	the	academic	behavior,	mathematics	

performance,	and	classroom	conduct	of	each	study	child	in	their classroom.	For	students	receiving	special	

education	services,	their	special	education	teacher	or	provider will	also	complete	a	survey	questionnaire	

similar	in	structure to	the	two-part	mathematics	teacher	instrument,	consisting	of	a	teacher-level	

questionnaire	and	student-level	questionnaire,	but	with	questions	specific	to	the	special	education	

experiences and	services	of	the	study	child.	School	administrators	will	be	asked	to	report	on	school	

programs	and services,	as	well	as	on	school	climate.

In	short,	the	MGLS:2017	aims	to	obtain	an	understanding	of	the	development	and	learning	that	occur during	

students’	middle-grade	years	(grades	6–8)	and	that	are	predictive	of	future	success,	along	with	the	

individual,	social,	and	contextual	factors	that	are	related	to	successful development.	A	key	goal	of	the	study	is	

to	provide	researchers	and	policymakers	with	the	information they	need to	better	understand	the	school	and	

nonschool	influences	associated	with	mathematics	and	reading	success,	socioemotional	health,	and	positive	

life	development	during	the	middle-grade	years and	beyond. To	support	the	development	of	the	study,	the	

MGLS:2017	is	conducting	two	field	tests:	(1)	Item	Validation	Field	Test and	(2)	an	Operational	Field	Test.

The	study’s	success	is	dependent	on	the	development	of	reliable,	valid	measures.	Therefore	the	importance	

of	the	Item	Validation	Field	Test is	to	collect	data	to	support	examination	of the	mathematics	assessment,	

reading	assessment,	executive	function	assessment,	student	survey,	parent	survey,	and	school	staff	surveys.	

The	Item	Validation	Field	Test	will	provide	the	data	needed	to	determine	the	psychometric	properties	of	

items	and	the	predictive	potential	of	assessment	and	survey	items so	that	valid,	reliable,	and	useful	

assessment	and	survey	instruments	can	be	composed	for	the	main	study. As	the	focus	of	the	Item	Validation	

Field	Test	is	the	analyses	of	the	psychometric	properties	of	the	survey	items	and	assessments,	the	Item	

Validation	Field	Test	requires	a	large,	diverse	field	test	sample,	though	not	a	nationally	representative	one.
4

Gaining	schools’	cooperation	in	voluntary	research	is	increasingly	challenging.	Therefore,	the	importance	of	

the	Operational	Field	Test is	to	gain	a	deeper	understanding	of	effective	recruitment	strategies	that	lead	to	

higher	response	rates,	which	in	turn	improve	the	quality	of	the	data.	This	will	include	a	responsive	design	

approach	for	non-responders.	Additionally,	the	Operational	Field	Test	will	tighten	the	understanding	of	

assessment	and	survey	timing,	providing	crucial	information	in	terms	of	overall	functionality	of	the	

assessments	and	surveys	and the	time	it	takes	for	respondents	to	complete	them. In	turn,	this	will	inform	

potential	modifications	to	be	made	to	the	main	study	data	collection	instruments. With	the	focus	of	the	

Operational	Field	Test	on	recruitment	strategies,	tactics	for	retention	of	the	sample	within	the	study,	and	the	

operational	administration	of	the	surveys	and	assessments,	the	Operational	Field	Test	requires	a	close	to	

nationally	representative	sample.	However,	because	the	analyses	of	the	survey	and	assessment	items	are	the	

purpose	of	the	Item	Validation	Field	Test,	the	Operational	Field	Test	does	not	require	a	sample	as	large	in	

number	of	student participants.

A.2 Purposes	and	Uses	of	Data

The	Item	Validation	Field	Test will take	place in	January	through March 2016.	Its	purpose	is to	evaluate	a	

battery	of	student	assessments	(i.e.,	mathematics	and	reading	skills and	executive	function)	and	survey	

instruments	(i.e.,	student	survey,	parent	survey,	and	school	staff	surveys)	for	use	in	the	MGLS:2017	main	

study.	The	Item	Validation	Field	Test	will	provide	the	much	needed	information	to	establish	the validity	and	

reliability	of	the	direct	assessments	and	surveys.	The	Item	Validation	Field	Test	will	collect	data	from	

children	enrolled	in	5th through	8th grade	in	January	of	2016.

The	Operational	Field	Test will	run	concurrently to	the	Item	Validation	Field	Test,	also	taking	place	in	

January	through	March	of	2016.	Unlike	the	cross-sectional,	one	time	administration	of	the	Item	Validation	

Field	Test,	the	Operational	Field	Test	is	longitudinal	in	nature.	The	purpose	of	the	Operational	Field	Test	is	to	

recruit	an	approximately	nationally	representative	sample	of	6th-graders;	field	a	simulated	version	of	the	

final	6th-grade	assessments	and	surveys;	and	then	track	these	6th-graders	across	the next	2	years,	whether	

they	stay	within	their	6th-grade	school	or	move	to	another	school.	The	sample	will	be	followed up	in the	

winter	of	2017	and	of	2018;	however,	as	stated,	the	Operational	Field	Test	is	not	currently	designed	to	field	

the	assessments	and	surveys	past	the	6th-grade	collection	in	2016.	In	2017	and	of	2018,	the	Operational	

Field	Test	will	be	solely a	tracing	and	tracking	effort.	The	Operational	Field	Test	will	be	used	to	better	

understand	the	recruitment	strategies	necessary	for	a	large-scale	nationally	representative	effort,	the	

response	rates	during	recruitment	and	associated	with	a	6th-grade data	collection,	and	the	effort	involved	in	

the	subsequent	tracking	of	the	sample	from	the	base	year	(when	the	children	will	be	in	the	6th grade)	to	the	

first	follow-up	(when	most	of	the	children	will	be	in	the	7th grade)	and	the	second	follow-up	(when	most	of	

the	children	will	be	in	the	8th grade).

The	Operational	Field	Test	is	distinct	from	the	Item	Validation	Field	Test,	whose	primary	purpose	is	to	obtain	

information	on	the	psychometric	properties	of	items	so	that	valid	and	reliable	test	instruments	can	be	

composed	for	the	main	study.

Field	Test	Components

Both	the	Item	Validation	Field	Test	and	the	Operational	Field	Test	involve	the	same	basic	components. The	

difference in	the	two	field	tests	is	in	their	purpose,	their	sample,	and	the	analyses	that	will	follow.	The	Item	

Validation	Field	Test is	a	purposive	sample	of	5th through	8th grade	students	selected	to	support	the	

collection	of	data	to	establish	and	analyze	the	validity	and	reliability	of	the	surveys	and	assessments.	The	

Operational	Field	Test is	close	to	a	nationally	representative	sample	selected	to	support	a	deeper	
5

understanding	of	the	main	study’s	operations	in	terms	of	recruitment	of	the	sample	and	operational	

considerations	in	fielding	the	assessments	and	surveys (see	Part	B,	section	B.1.1-B.2.1 for	sample	details).

Student	Assessments and	Student	Survey.	Students	will	participate	in	assessments	and	a	survey,	designed	

to take	approximately	90	minutes	per	student

 Mathematics	Assessment.	The	MGLS:2017	main	study	mathematics	assessment	will	be	a	30-minute,	

two-stage adaptive	assessment	that	students	will	take	on	a	tablet computer.	The	focus	will	be	on	

domains of	mathematics	that	are	most	likely to	be	the	central	focus	of	middle	school	learning	now and	

in	the	future:	the	Number	System,	Ratios	and	Proportional	Relationships,	Expressions	and	Equations,	

and	Functions.	To	ensure	that	we	are	sensitive	to the	variation	in	students’	mathematics	ability, the	

assessment	will	include	items	with	appropriately	varying	cognitive	demand.	The	MGLS:2017	

mathematics assessment	will	provide	valuable	information	about	the	development	of	middle-grade	

students’	knowledge of	mathematics	and	their	ability	to	use	that	knowledge	to	solve	problems,	moving	

toward	stronger	reasoning	and	understanding	of	more	advanced	mathematics.

 Reading	Assessment.	The	MGLS:2017	reading	assessment	will	use a	two-stage	adaptive	assessment	

design	consisting	of	a	brief	routing	block	(first	stage:	approximately	10	minutes)	followed	by	a	skillbased	block	(second	stage:	approximately	20	minutes),	for	a	total	of	30	minutes.	The	routing	block	will	

include	items	that	measure	foundational	components	of	reading	that	are	important	for	comprehension:	

Vocabulary,	Morphological	Awareness,	and	Sentence	Processing.	Performance	on	the	routing	block	will	

direct students	to	one	of	three	types of	skill-based	reading	blocks (basic	components,	basic	

comprehension,	or	scenario-based comprehension) within	the	second stage.

The	second-stage	basic	components skill	block	will	be	used	to	gather	more	information	on	the	

foundational	reading	component skills, including	those	measured	in	the	first	stage	as	well	as	word	

recognition	and	decoding	skills.	The	basic	components	block	will	also	capture	information	about	

students’	efficiency	at basic	reading	comprehension	and	ability	to	comprehend	short	passages.	The	

second-stage	basic	comprehension skill	block	is	designed	to	gather	information	about	students’	

efficiency	at basic reading	comprehension	and	their	ability	to	comprehend	short	passages.	This	skillbased	block	will	measure	comprehension	in	a	traditional	design	where	unrelated	passages	and	

corresponding	questions	are	presented.	The	second-stage	scenario-based	comprehension skill	block	is	

designed	to	gather	information	about	students’	ability	to	comprehend	informational	text	and	reason	

more	deeply	about	text	and	to	apply	what	they	learn	from	passages.	The	scenario-based	block will	

include	a	scenario	or	a	purpose	for	reading	(e.g.,	preparing	for	a	classroom	discussion	or	creating	a	

website	on	a	topic).

 Executive	Function	Measures.	Executive	function,	a	set	of	capacities	and	processes	originating	in	the	

prefrontal	cortex	of	the	brain,	permits	individuals	to	self-regulate,	engage	in	purposeful	and	goaldirected	behaviors,	and	conduct	themselves	in	a	socially	appropriate	manner.	Self-regulation	is	needed	

for	social	success,	academic	and	career	success,	and	good	health	outcomes.	Executive	function	includes	

capacities	such	as	shifting	(cognitive	and	attention	flexibility),	inhibitory	control,	and	working	memory.	

Four	different executive	function	measures	will	be	included	in	the	field	tests:	Stop	Signal	(inhibitory	

control),	3-Back with	verbal	stimulus	(working	memory),	2-Back	with	nonverbal	stimulus	(working	

memory),	and	the	Hearts	and	Flowers	task	(shifting	or cognitive	flexibility).	As	the	Item	Validation	Field	

Test’s	purpose	is	to	inform	the	reliability	and	validity	of	the	measures,	all	four	executive	function	

measures	will	be	included	in	the	Item	Validation	Field	Test;	whereas	current	design	is	that	two	

measures	will	be	selected	for	inclusion	in	the	Operational	Field	Test.
6

 Student	Survey.	The	purpose	of	the	student	survey is	to	collect	information	on	students’	attitudes	and	

behaviors;	out-of-school	time	use;	and	family,	school,	and	classroom	environments.	The	student	survey

will	also	serve	as	a	source	for	information	about	socioemotional	outcomes	having	to	do	with	social	

relationships, support, and	school	engagement.

Parent	Survey. The	parent	survey will	take	30	minutes	to	complete	via	a	self-administered	web-based

questionnaire;	a	telephone	interview	follow-up	will	be	available	for	respondents	who	do	not	complete the	

questionnaire	via	the	web.	The	parent	survey will	focus	on	supplementing the	information	collected	from	

students	and	teachers	about	the	students’	educational	experiences	and	on	learning about	parents’	

expectations	for	their	children’s	academic	attainment	in	high	school	and	beyond.	It will	also	collect	

information	about	family	involvement	in	the children’s	education	and about	family	characteristics	that	are	

key	predictors	of	academic	achievement	and	other	student	outcomes.

Mathematics	Teacher	Survey/Teacher	Student	Report.	The	mathematics	teacher	survey	will	consist	of	

two	parts:	a	teacher survey and	a	series	of	teacher	student	reports	(TSRs).	Both the	mathematics	teacher	

survey and	the	TSR	will	be	web-based,	self-administered	surveys,	with	a	phone	interview	option	available.	

The	mathematics	teacher	survey	is	expected	to	take	approximately	20	minutes	to	complete,	and	the	TSR	will	

take	5	to	10	minutes	for	each	student that	is	rated.	The	mathematics	teacher	survey will	collect	data	on	

potential	classroom-level	correlates	of	students’	mathematics	achievement	as	well	as	school-level	services	

and	factors	such	as	special	programs,	school	climate,	and	instructional	leadership.

Teacher	responses	to	the TSR	will	capture	information	specific	to	the	sampled	student	and	his	or	her	

mathematics	class.	It	will	provide	information	on	the	classroom	attendance	and	performance	of	individual	

students,	which	will	augment	direct	student	assessments,	transcript	information,	and	student	and	parent	

reports.	The	TSR	will	also	serve	as	an	additional source	for	data	on	student	socioemotional	outcomes	related	

to	regulation,	school engagement,	and	externalizing	behaviors.	In	the	web	version	of	this	instrument,	

teachers	will	be	given	a	list	of	the	students	for	whom	they	should	complete	a	TSR	and	will	click	on	each	

student’s	name	to	launch	the	TSR	for	that	specific	student.	If	a	teacher opts	not	to	complete	the	web-based	

survey, a	follow-up	phone	interview	will	be	conducted.

Special	Education	Teacher	Survey/Teacher	Student	Report.	Like	the	mathematics	teacher	survey,	the	

special	education	teacher/service	provider	survey will	consist	of	two	parts. The	first	part consists	of	the	

teacher	questionnaire,	which	asks	questions	about	the	teacher’s	background	and	experiences	working	with	

students	with	disabilities.	The	second	part	contains the	TSR,	which	contains	specific	questions about	special	

education	services	and	other	contextual	variables	for	sampled	children	with	an	Individualized	Education	

Program	(IEP) as	well as	ratings	of	individual	academic	and	life	skills (the	special	educator	rating	scale,	

SPERS).

The	special	education teacher	survey will	be	web based and	self-administered, with	a	phone	interview

option available.	The	first	part of	the survey will take	approximately	10	minutes	to	complete,	and	the	second	

part	will	take	about	25 minutes	for	each	student. In	the	second	part	of	the	web	version	of	this	instrument,	

teachers	will	be	given	a	list	of	the	students	for	whom	they	should	complete	the	survey	and	will	click	on	each	

student’s	name	to	launch	this	part	for	that	specific	student.

School	Administrator	Survey.	The	school	administrator	survey will	be	web based	and	self-administered,	

with	a telephone	option	available,	and	will	take	the	administrator	(generally,	the	principal	or	principal’s	

designee)	approximately	20	minutes	to	complete.	The	school	administrator	survey	will	collect information	

about a	school’s	characteristics	and	staffing (specifically,	the	school’s	structure	and	climate,	including	safety,	
7

organization,	and	support).	It	will	also	collect	information	on the	student	population,	student	conduct,	

academic	culture,	course	offerings,	and	extended	learning	opportunities	(e.g.,	extracurricular	activities,	

summer	school,	or	supports	for	struggling	students).

Administration	of	Assessments	and	Survey	Components

Item	Validation	Field	Test.	In	the	Item	Validation Field	Test,	students’	parents,	math	teachers,	special	

education	providers	(as	applicable),	and	school	administrators	will	be	asked	to	complete	surveys	as	

described	above.	To	keep student participation	to	approximately	90	minutes and	gain	as	much	information	

on	as	many	assessment	and	survey	items	as	possible,	the	Item	Validation	Field	Test	will	employ	a	spiral	

design	in	which	not	all	students	will	receive	the	same	assessments	and	survey	items.	Table	1	below	presents	

a	summary	of	the	student	assessment	and	survey	booklet	spiral	design.

Table	1. Item	Validation	Field	Test	Student	Assessment	and	Spiral	Design	

Booklet	1 Booklet	2 Booklet	3 Booklet	4 Booklet	5 Booklet	6

Math	assessment Math	assessment Math	assessment Math	assessment Math	assessment Math	assessment

Demographic	items	

Executive	function	

task:

Hearts	&	Flowers

Executive	function	

task:

3-Back

Demographic	items	 Demographic	items	 Demographic	items	

Reading	

assessment

(two-stage)

No	reading	

assessment

No	reading	

Assessment

Reading	

assessment

(two-stage)

Reading	

assessment

(two-stage)

Reading	

assessment

(two-stage)

Executive	function	

task:

Stop	Signal	

Demographic	items	 Demographic	items

Executive	function	

task:

2-Back

No	executive	

function	task

No	executive	

function	task

Theories	of	

Intelligence	

(general)

Student	

Questionnaire

Student	

Questionnaire

No	student	

questionnaire	

No	student	

questionnaire

No	student	

questionnaire

Operational	Field	Test.	Similar	to	the	Item	Validation	Field	Test,	students’	parents,	math	teachers,	special	

education	providers	(as	applicable),	and	school	administrators	will	be	asked	to	complete	surveys	as	

described	above. The	Operational	Field	Test	will	not	employ	a	spiral	design	for	the	student	assessments	and	

student	survey	items.	The	purpose	of	the	Operational	Field	Test	is	to, as	closely	as	possible, mirror	the	main	

study	approach;	therefore,	each	student	will	receive	a	portion	of	the	math	assessment	(30	minutes);	reading	

assessment	(30	minutes);	two	of	the	executive	function	assessments	(10	minutes);	and	the	student	survey	

(20	minutes)	(approximately	90	minutes	total).

School	Recruitment	Approach

The	student	sample	for the	Item	Validation	Field	Test,	while	not	required	to	be	nationally	representative	

for psychometric	analysis,	will include	students	in	the	typical age	range	found in	grades	5–8 in	the	United	

States;	these students	will likely	demonstrate	a	range	of	ability	on	the	constructs	being	measured	by	the	

MGLS	item	pool.	The	sample	will	also	include a	subset	of	students	from	the	three	focal	disability groups	

(learning	disability,	autism,	and	emotional	disturbance)	who	are	able	to	take	standardized	tests	using	

accommodations.	Schools	will	be	recruited	both	directly	and	at	the	district	level.

The	Operational	Field	Test will	require	an	approximately nationally	representative	sample	of	6th-graders	in	

two	groups:	Group	1	is	students who	are	found	on	the	rosters	of	general	education	schools1that	are	sampled	

																																																												

1

Such	students	may	or	may	not	have	an	IEP.	There	will	be	an	effort	to	include	all	students	who	fall	into	the	Group	1	sample	regardless	of	IEP	status.
8

into	the	study, and	Group	2	is	a	purposive	sample	of	students	who have	a	primary	disability	categorization	in	

their IEP of	either	emotional	disturbance	or	autism. The	universe	for	the	MGLS:2017	Operational	Field	Test	

includes	all	children	enrolled	in	6th grade	in	the	2015–16	school	year	in	the	50	states	and	the	District	of	

Columbia.	Schools	will	be	recruited	both	directly	and	at	the	district	level. For	more	information	on	the	

recruitment	approach,	see	Part	B,	sections B.1 and	B.2.

A.3 Improved	Information	Technology	(Reduction	of	Burden)

Where	feasible,	available	technology	will	be	used	to	reduce	burden	and	improve	efficiency	and	accuracy.	For

example,	if	districts	can	provide	information	linking	students	to	their	mathematics	teachers	or	students with	

disabilities	to	their	special	education	teachers	electronically,	we	will	use	this	information	rather	than	asking	

for	it at	the	school	level. The	burden	of	recruitment	on	districts and	schools will	be minimal, with most	

information	gathered	over	the	telephone.	Districts	will	primarily	be	asked	to	provide	confirmation	of	data	

gathered	from	other	sources,	including	school	universe	files	and	district	and	school	websites. Our	collection	

of student	lists	will	accommodate	whatever	format districts	and	schools	find	to	be	the	least	burdensome.	The	

study	will	utilize	the	information	in	any format	it	is	provided.

A.4 Efforts	to	Identify	Duplication

The	MGLS:2017	will	not	be	duplicative	of	other	studies.	While	NCES	longitudinal	studies	have	contributed	to	

our	understanding	of	the	factors	that	influence	student	success	and	failure	in	school,	the	middle	grades	

(grades	6–8)	are	noticeably	absent	from	the	studies	conducted	to	date.	A	majority	of	nationally	

representative longitudinal	studies	have	focused	on	high	school	students	and	on	the	transition	from	

secondary	to	postsecondary	education: e.g.,	the	High	School	and	Beyond	Longitudinal	Study	(HS&B) and	the

Education	Longitudinal	Study	of	2002	(ELS:2002).	The	Early	Childhood	Longitudinal	Study,	Kindergarten	

Class	of	1998–99	(ECLS-K), and	the	National	Education	Longitudinal	Study	of	1988	(NELS:88)	collected	data	

on	students	in	grade	8,	but	neither	included	a	data	collection	in	grades	6	and	7.	The	ECLS-K:2011	will	not	

follow	students	beyond	grade	5, and	the	High	School	Longitudinal	Study	of	2009	(HSLS:09)	began	with	a	

national	sample of	students	in	grade	9.	Thus,	there	is little information	at	the	national	level	about	the	

learning	that	occurs	during	grades	6–8	and	about	the	rates	of	learning	for	different	groups	of	students	who	

may	experience	diverse	school	environments	and	opportunities.

The	MGLS:2017	is	unique	in	that	it	will	assess	students’	mathematics	and	reading	achievement,	as	well	as	

other	student	outcomes	(e.g.,	executive	function	and	socioemotional	development), for	the	same	group	of	

students	over	a	3-year	period.	In	addition	to	the	ECLS-K	and	NELS:88,	other	national studies have	assessed	

some	of	these	outcomes	for	students	in	grade	8,	including	the	National	Assessment	of	Educational Progress	

(NAEP)	and	the	Trends	in	International	Mathematics	and	Science	Study	(TIMSS).	These	studies, however, are	

cross-sectional	and	do	not	include	repeated	measures	of	achievement or assess	multiple	subjects	and	areas	

of	development	for	the	same	sample	of	students.	Therefore,	they cannot	answer	questions	about	students’	

growth	in	mathematics	and	reading	over	the	middle-grade	years,	about	differences	in	the	rates	of	growth	for	

different	populations	(e.g.,	differences	by	gender, by race/ethnicity,	and	for	students	attending	public	and	

private	schools),	and	about	the	school	and	nonschool	factors	that	may	facilitate	or	hinder	this	growth.	Nor	

can	they	explore	questions	about	the	relationships	between	student	achievement	and	other	school	outcomes	

and	executive	functions	(e.g.,	working	memory,	attention,	and	inhibitory	control)	that	work	to	regulate	and	

orchestrate	cognition,	emotion,	and	behavior	to	enable	a	student	to	learn	in	the	classroom. The	MGLS:2017	

will	also	be	unique	in	its	focus	on	obtaining	a	sample	of	students	in	three	disability	categories	that	can	be	

studied	on	their	own	or	compared	to	general	education	students	over	the	three	middle	level	years.
9

Other	adolescent	development	studies	have	been	conducted,	but	they	often	do	not	include	a	grade	6	sample.	

For	example,	the	youngest	children	in	the	National	Longitudinal	Study	of	Adolescent	Health	(Add	Health) and	

the	Maryland	Adolescent	Development	in	Context	Study	(MADICS)	were	in	grade	7	at	baseline.	Many	of	these	

studies	collected	data	on	local	samples,	had	a	primary	focus	on	family	and	child	processes,	and	were	started	

in	the	1990s: e.g.,	MADICS	and	the	Michigan	Study	of	Adolescent	and	Adult	Life	Transitions	(MSALT).	As	

such,	they	do	not	provide	a	contemporary	picture	of	U.S.	students	in	grades 6–8.

A.5 Minimizing	Burden	for	Small	Entities

Although	small	entities	are	not	part	of	this	study,	in	general,	burden	will	be	minimized	wherever	possible.	

For	example,	schools	will	be	selected	so	as	to	avoid	overlap	with	other	NCES	assessments, such	as	the	

concurrently	run MGLS:2017	Operational	Field	Test. During	district	and	school recruitment,	we	will	

minimize	burden	by	training	recruitment	staff	to	make	their	contacts	as	straightforward	and	concise	as	

possible.	The	recruitment	letters	and	materials	(e.g.,	the	study	description	and	FAQs)	are	designed	to	be	

clear,	brief,	and	informative.	In	addition,	contractor	staff	will	conduct	all	test	administration	and	will	assist	

with	parental	notification,	sampling,	and	other	study	tasks	as	much	as	possible	within	each	school.

A.6 Frequency	of	Data	Collection

The	Item	Validation	Field	Test for	the	MGLS:2017	is	a	one-time	data	collection	that	will take	place in	

January	through	March of	winter	2016. The	Operational	Field	Test for	the	MGLS:2017	is	expected	to	take	

place in	January	through	March of	winter	2016,	with	limited	follow-ups	(as	described	in	section	A.2)	in	

January	through	March	of	winter	2017	and	2018. The	follow-ups	will	focus	solely	on	student	tracing	and	

tracking	and	will	not	re-assess	students	or	re-administer	any	survey	components.

A.7	Special	Circumstances

There	are	no	special	circumstances	involved	with	the	recruitment.

A.8 Consultations	Outside	NCES

As	part	of	the	MGLS:2017	design	contract,	content experts	were	consulted	in	the	development	of	the	

assessments	and	questionnaires. These	experts	are	listed	by	name,	affiliation,	and	expertise	in	table	2.

Table	2. Members	of	the	MGLS:2017	Content	Review	Panels

Name Affiliation Expertise

Mathematics	Assessment	Content	Review	Panel (June	18–19,	2013)

Tom	Loveless Brookings	Institution Policy,	math	curriculum

Linda

Wilson

Formerly	with	

Project	2061

Math	education,	math	assessment,	middle	school	assessment,	author	of	NCTM	

Assessment	Standards	for	School	Mathematics	and	NAEP	math	framework,	

teacher

Kathleen	Heid University	of	Florida Math	education,	use	of	technology,	teacher	knowledge,	NAEP	Grade	8	

Mathematics	Standing	Committee	member

Edward	Nolan Montgomery	County	

Schools,	Maryland Math	curriculum	and	standards,	large-scale	assessment	of	middle-grade students

Lisa

Keller

University	of	

Massachusetts,

Amherst

Psychometrics,	former	math	teacher

Paul

Sally University	of	Chicago Math	education,	mathematics	reasoning,	mathematically	talented	adolescents
10

Margie

Hill University	of	Kansas Co-author	of	Kansas	mathematics standards,	former	NAEP	Mathematics Standing	Committee	member,	former	district	math	supervisor

Executive	Function	Content	Review	Panel (July	18,	2013)

Lisa	Jacobson

Johns	Hopkins	

University;	Kennedy	

Krieger	Institute

Development	of	executive	functioning	skills,	attention,	neurodevelopmental	

disorders,	and	parent	and	teacher	scaffolding

Dan

Romer

University	of	

Pennsylvania Adolescent	risk	taking

James	Byrnes Temple	University Self-regulation,	decision	making,	cognitive	processes	in	mathematics	learning

Socioemotional-Student-Family	Content	Review	Panel (July	25–26,	2013)

James	Byrnes Temple	University Self-regulation,	decision	making,	cognitive	processes	in	mathematics	learning

Russell	

Rumberger

University	of	

California,	Santa	

Barbara

School	dropouts,	ethnic	and	language	minority	student	achievement

Tama	

Leventhal Tufts	University Family	context,	adolescence,	social	policy,	community	and	neighborhood	indicators

Susan	Dauber Bluestocking	

Research

School	organization,	educational	transitions,	urban	education,	parent	involvement	

and	family	processes

Scott

Gest

Pennsylvania	State	

University Social	networking,	social	skills,	longitudinal	assessment	of	at-risk	populations

Kathryn	

Wentzel

University	of	

Maryland

Social	and	academic	motivation,	self-regulation,	school	adjustment,	peer	

relationships,	teacher-student	relationships,	family-school	linkages

Richard	

Lerner Tufts	University Adolescent	development	and	relationships	with	peers,	families,	schools,	and	communities

School	Administrator	Content	Review	Panel (August	16,	2013)

Susan	Dauber Bluestocking	

Research

School	organization,	educational	transitions,	urban	education,	parent	involvement	

and	family	processes

George	Farkas University	of	

California,	Irvine Schooling	equity	and	human	resources

Jeremy

Finn

State	University	of	

New	York	at	Buffalo School	organization,	school	dropouts

Edward	Nolan Montgomery	County	

Schools,	Maryland Large	urban	school	system	administrator

Tom	Loveless Brookings	Institution Policy,	math	curriculum

Reading	Assessment	Content	Review	Panel (	April	14,	2014)

Donna	

Alvermann University	of	Georgia Adolescent	literacy,	online	literacy,	codirector of	the	National	Reading	Research	Center	(funded	by	the	U.S.	Department	of	Education)

Joseph	

Magliano

Northern	Illinois	

University

Cognitive	processes	that	support	comprehension,	the	nature	of	memory	

representations	for	events	depicted	in	text	and	film,	strategies	to	detect	and	help	

struggling	readers

Sheryl	

Lazarus

University	of	

Minnesota

Education	policy	issues	related	to	the	inclusion	of	students	with	disabilities	in	

assessments	used	for	accountability purposes,	student	participation	and	

accommodations,	alternate	assessments,	technology-enhanced	assessments,	

teacher	effectiveness,	large-scale	assessments,	school	accountability,	research	

design	(including	cost	analyses),	data-driven	decision making,	rural	education,	the	

economics	of	education

Disabilities	Content	Review	Panel	(April	29,	2014)

Jose	

Blackorby SRI	International

Autism,	specific	learning	disabilities,	special	education,	curriculum	design,	

alternate	student	assessment,	large-scale	studies	of	students	with	disabilities,	

codirector of	the	Special	Education	Elementary	Longitudinal	Study (SEELS)

Lynn

Fuchs Vanderbilt	University Specific	learning	disabilities,	student	assessment,	mathematics	curriculum,	psychometric	models	

Mitchell	L.	

Yell

University	of	South	

Carolina

Autism,	emotional	and	behavior	disorders,	specific	learning	disabilities,	pre-K–12	

instruction	and	curriculum,	special	education,	evidence-based	intervention
11

Sheryl	

Lazarus

University	of	

Minnesota

Special	education	policy,	inclusion	of	students	with	disabilities	in	assessments,	

accommodations,	alternate	assessments,	technology-enhanced	assessments,	

large-scale	assessments,	school	accountability,	research	design	(including	cost	

analyses)

Martha	

Thurlow

University	of	

Minnesota

Specific	learning	disabilities,	reading	assessment,	alternate	student	assessment,	

early	childhood	education,	special	education,	curriculum,	large-scale	studies

Diane	

Pedrotty	

Bryant

University	of	Texas,	

Austin

Educational	interventions	for	improving	the	mathematics	and	reading	

performance	of	students	with	learning	disabilities,	the	use	of	assistive	technology	

for	individuals	with	disabilities,	interventions	for	students	with	learning	

disabilities	and	who	are	at risk	for	educational	difficulties

A.9 Payments	or	Gifts	to	Respondents

High	levels	of	school	participation	are	critical	to	the	success of both	the	Item	Validation	Field	Test and	the	

Operational	Field	Test. School administrator,	mathematics	teacher,	special education	teacher,	parent,	and	

student	data	collection	activities	are	contingent	on	school	cooperation.	NCES	recognizes	that	the	burden	

level	of	the	study	is	one	of	the	factors	that	school	administrators	will	consider	when	deciding	whether to	

participate.	To	offset	the	perceived	burden	of	participation,	NCES	intends	to	continue	to use	strategies	that	

have	worked	successfully	in other	major	NCES	studies	(e.g.,	ECLS-K,	ECLS-K:2011,	HS&B,	NELS:88,	and	

ELS:2002),	including	offering	both monetary	and	non-monetary incentives. Table	3 summarizes the	

proposed incentive	amount	for	each	instrument	and	activity along	with their estimated	administration	

times;	a	brief	justification	for	each	incentive	amount follows	table	3.

Table	3. Item	Validation	and	Operational	Field	Test	Instruments	and	Proposed	Incentive	Amounts

Instrument/Activity Administration	Time* Field	Test Incentives

Item	Validation Operational

Student	Assessments	and	

Survey

(Math,	Reading,	Executive	

Function,	and	Student	Survey)

90	minutes No	monetary	incentive

Choice	of	e.g.	1)	mechanical	pencil,

2)	mobile	device	screen	cleaner,

3)	suncatcher, 4)	slap	bracelet

(average	value	$0.50	each)

Parent	Survey 30	minutes None,	$20,	or	$40

$20 (with	possible	additional	amount	for	

targeted	cases	if	responsive	design	in	

nonresponse	follow-up	is	utilized)

Mathematics	Teacher	

Teacher	Survey 20	minutes $20

Teacher	Student	Report	 10	minutes	per	student $7	per	TSR

Special	Education	Teacher

Teacher	Survey 10	minutes $20

Teacher	Student	Report	 25	minutes	per	student	 $7	per	TSR

School	Administrator	Survey 20	minutes No	monetary	incentive

School	Participation

School	Coordinator

(logistics,	on-site	visit,	consent	

forms,	administrative	records,	

etc.)

6	hours	for	consent	assistance

2	hours	to	schedule	

assessments

2	hours	to	set	up	web	access,	

coordinate	computer	labs

6	hours	to	provide	

administrative	records

$200,	$400,	or	$400	in	material	or	services	for	school

$150	for	coordinator

*Note	that	the	assessment	administration time	may	be	longer	for	students	with	disabilities.

Students

There	is	no	monetary	incentive	in	the	Item	Validation	Field	Test for	students.	In	the	Operational	Field	

Test,	we	plan	a	simple	experiment	on	Token	of	Appreciation	for	Student	Participation.	To	build	goodwill	

toward	the	study	on	the	part	of	students,	in	the main	study	MGLS:2017 plans	to offer	students	the	choice	of	
12

one	of	two	items	valued	at	$0.25	up	to	$1	each.	These	items	will	be	unbranded	with	any	MGLS	logo	in	order	

not	to	identify	the	student	as	having	participated	in	this	particular	study.	NCES	has	experience	with	the	

tastes	of	elementary	and	high	school	students	but	we	are	not	as	familiar	with	what	would	be	an	attractive	

reward	for	middle	grades	students.	By	giving	the	operational	field	test	students	the	choice	of	one	of	four	

items,	we	can	determine which	two	would	likely	be	more	attractive	in	the	national	data	collection.

Parents

Parent	survey	response	rates	have	declined	over	the	past	decade.	The	ECLS-K:2011	baseline	(fall	2010)	

parent survey	response	rate	was	more	than	10	percentage	points lower	(74 percent)2than	the	parent survey	

rate in	the	corresponding	1998	wave	of	the	ECLS-K (85 percent).3 Additionally,	the	9th grade	parent	survey	

response	rate	for	the	HSLS:2009	baseline was	68	percent.4 The	MGLS	parent	survey	is a	key	component	of	

the	data	being	collected.

To	improve	the	chances	of obtaining	higher	parent	participation	rates	in	a	school-based	design,	we will	work	

with	school	personnel	to	recruit	sample	students’	parents	into	the	MGLS:2017 and	will conduct	an	

experiment in	the	Item	Validation	Field	Test to	determine	the	effect	of	different	levels	of	monetary	incentives	

on	parent	participation.

In	the	Item	Validation	Field	Test,	an experiment will	be	used	to	determine	the	effect	on	response	rates and

on	the	cost	and	length	of	nonresponse	follow-up of	offering parents	of	middle-grade	students	a	$0,	$20,	or

$40	incentive for	completing	the	parent	questionnaire.	Additionally,	the	experiment	will	also	evaluate	

whether	parents	of	children	with	disabilities,	who	may	be	more	reluctant to	engage	in	this	study	and	who	

may	require more	frequent	and	extensive	nonresponse	follow-up,	are	influenced	differently	by	the	offer	of	a	

monetary	incentive	than	parents	of	students	without	disabilities.

In	the	Item	Validation	Field	Test,	each	school	will	be	randomly	assigned	to	one	of	the	three	experimental	

conditions. Therefore,	all	parents asked	to	complete	the	parent	survey within	a	school	will	be	assigned	to	the	

same	condition.	Parents	in	one-third	of	the	schools	will	be	asked	to	complete	the	parent	survey,	but	will	not	

be	offered	a	monetary	incentive	for	doing	so; parents	in	another	one-third	of	the	schools	will	be	offered	$20	

to	complete the	survey; and	parents	in	the	remaining	one-third	of	the	schools	will	be	offered	$40 to	complete	

the	survey.	We	will	monitor	the	response	rate	in	each	group	and	document	the	level	of	effort	needed	to	

obtain	the	response	rates	achieved	under	the	different	incentive/no-incentive	options.	All	groups	will	

receive	similar	reminders	and	other	modes	of	follow-up	contact.	The	number	of	contact	attempts	to	achieve	

the	final	response	rates	will	be	measured	to	compute	the	potential	resource	savings,	if	any,	of	each	incentive	

payment	relative	to no incentive	payment.

For	the	Item	Validation	Field	Test,	as	shown	in	Part	B,	section	B.1.1	we	plan	for	3,950	participating	students.	

Assuming	an	80	percent	response	rate	from	students,	this	means	we	will	have	needed	to	consent	4,938	

students.	Therefore,	within	the	Item	Validation	Field	Test,	we	will	be	seeking	parent	surveys	from	4,938	

																																																												

2 Tourangeau,	K.,	Nord,	C.,	Lê,	T.,	Sorongon,	A.G.,	Hagedorn,	M.C.,	Daly,	P.,	and	Najarian,	M.	(2012).	Early	Childhood	Longitudinal	Study,	Kindergarten	

Class	of	2010–11	(ECLS-K:2011),	User’s	Manual	for	the	ECLS-K:2011	Kindergarten	Data	File	and	Electronic	Codebook	(NCES	2013-061).	U.S.	Department	

of	Education.	Washington,	DC:	National	Center	for	Education	Statistics.

3 Tourangeau,	K.,	Nord,	C.,	Lê,	T.,	Sorongon,	A.G.,	Hagedorn,	M.C.,	Daly,	P.,	and	Najarian,	M.	(2001).	Early	Childhood	Longitudinal	Study,	Kindergarten	

Class	of	1998–99	(ECLS-K),	User’s	Manual	for	the	ECLS-K	Base	Year	Public-Use	Data	Files	and	Electronic	Codebook	(NCES	2001-029).	U.S.	Department	of	

Education.	Washington,	DC:	National	Center	for	Education	Statistics.

4 Ingels,	S.J.,	Pratt,	D.J.,	Herget,	D.R.,	Burns,	L.J.,	Dever,	J.A.,	Ottem,	R.,	Rogers,	J.E.,	Jin,	Y.,	and	Leinwand,	S.	(2011). High	School	Longitudinal	Study	of	

2009	(HSLS:09).	Base-Year	Data	File	Documentation	(NCES	2011-328). U.S.	Department	of	Education.	Washington,	DC:	National	Center	for	Education	

Statistics.
13

students’	parents.	Assuming	4,938	are	split	approximately	equally	across	schools	and	conditions;	this	would	

result	in approximately	1,646	cases	within	each	incentive	level.	

As	stated,	the	9th grade	parent	survey	response	rate	for	the	HSLS:2009	baseline	was	68%.	For	a	power	of	

0.80,	a	confidence	level	of	95%,	and	1,646	cases	within	each	condition,	in	this	experiment	a	4.5% point	

difference	in	response rate	should	be	detectable as	statistically	significant	(e.g.,	68.0%	vs.	72.5%).	Formula	

provided	below.5

n	=	(Zα/2+Zβ)

2 *	(p1(1-p1)+p2(1-p2))	/	(p1-p2)2

Where	Zα/2 is	the	critical	value	of	the	Normal	distribution	at	α/2	(e.g.	for	a	confidence	level	of	95%,	α	is	0.05	and	

the	critical	value	is	1.96); Zβ is	the	critical	value	of	the	Normal	distribution	at	β	(e.g.	for	a	power	of	80%,	β	is	0.2	

and	the	critical	value	is	0.84)	and	p1 and	p2 are	the	expected	sample	proportions	of	the	two	groups.

However,	the	Item	Validation	Field	Test	has	a	clustered	design	with	students	nested	in	schools.	Therefore,	

assuming	an	approximate	design	effect	of	4,	which	is	a	similar	design	effect	as	reported	by	the	HSLS:09	for	

parent	respondents6,	which	also	had	a	clustered	design	with	students	nested	in	schools,	the	effective	sample	

size	for	any	condition	would	be	approximately	412	cases	(1,646/4).	For	a	power	of	0.80,	a	confidence	level	of	

95%,	and	412	cases	within	each	condition,	this	experiment	should	be	able	to	detect	approximately	an	8.5%

point	difference	in	response	as	statistically	significant	(e.g.,	68.0%	vs. 76.5%).

In	the	Operational	Field	Test,	given	that	we	will	not	yet	have	the	results	from	the	Item	Validation	Field	Test	

experiment,	to	try	to	achieve	desired	response	rates,	all	parents	will	be	offered	$20.00	for	completing	the	

parent	survey.	A	parent	incentive	responsive	design	may	also	be	employed	in	non-response	follow-up,	the	

details	of	which	would	be	provided	in	late	2015	as	part	of	the	field	test	data	collections	submission.

In	September	through	December,	2015,	school	coordinators	will	distribute	to	sample	parents	a	letter	asking	

them	to	consent	to	their	child’s	participation	in	the	study	(where	explicit	consent	is	required)	and	to	provide	

either	their	email,	their	landline	phone	number,	or	their	cell	phone	number	so	that	they	can	be	contacted	for	

the	parent	survey.	By	providing	this	information,	parents	will	indicate	that	they	are	willing	to	participate	in	

the	parent	survey.	The	consent	form	indicates	that	parents	will	receive	a	$20	incentive	upon	completing	the	

parent	survey.

Teachers

For	both	the	Item	Validation	Field	Test and	the	Operational	Field	Test,	the	incentive	proposed	for	

students’	teachers	is	$20	per	teacher	survey, plus	$7	per	teacher student	report (TSR).	These	amounts	are	

consistent	with	the amounts	used	in current	NCES	studies, such	as	the	ECLS-K:2011.	While	it	is	estimated	

that	the	mathematics	teacher	survey will take	longer	to	complete	(20	minutes)	than	the	special	education	

teacher	survey (10	minutes),	the	reverse	is	true	for	the	individual	student	reports.	The	individual	student	

reports	will	require	approximately	10	minutes	per	student	to	complete	for	mathematics	teachers	and	25	

minutes	per	student	to	complete	for	special	education	teachers	(including 5	minutes	for	an	indirect	

assessment	of	student’s	skills,	the	SPERS).

																																																												

5

Retrieved from http://www.select-statistics.co.uk/sample-size-calculator-two-proportions.

6

Ingels,	S.J.,	Pratt,	D.J.,	Herget,	D.R.,	Burns,	L.J.,	Dever,	J.A.,	Ottem,	R.,	Rogers,	J.E.,	Jin,	Y.,	and	Leinwand,	S.	(2011). High	School	Longitudinal	Study	of	

2009	(HSLS:09).	Base-Year	Data	File	Documentation	(NCES	2011-328). U.S.	Department	of	Education.	Washington,	DC:	National	Center	for	Education	

Statistics.
14

Schools and	School	Coordinators

For	both	the	Item	Validation	Field	Test and	the	Operational	Field	Test,	we	propose	to conduct	an	

experiment	for	schools	that	works	across	both	field	tests.	Each	school	will	be	randomly	assigned	to	one	of	

the	three	experimental	conditions. In	Condition	1,	the	baseline	condition, we	will	offer	schools	a	$200	

incentive	for	their	participation.	Given	the	many	demands	and	outside	pressures	that	schools	already	face,	it	

is	essential	that	they	see	that	MGLS:2017	staff	understand	the	additional	burden	being	placed	on	school	staff	

when	requesting	their	participation.	The	study	asks	for	many	kinds	of	information	and	cooperation	from	

schools,	including	a	student	roster	with	basic	demographic	information	(e.g.,	date	of	birth,	sex,	and	

race/ethnicity);	information	on	students’	IEP	status;	permission	for	field	staff	to	be	in	the	school	for	up	to	a	

week;	space	for	administering	student	assessments;	permission	for	students	to	leave	their	normal	classes	for	

the	duration	of	the	assessments;	and	information	about	the	students’	teachers	and	parents.

For	sample	students	with	disabilities,	on	average,	5 students	in	each	school	will	be	selected	based	on

disability category, and	many	will	require	accommodations	and	different	assessment	settings,	such	as	

individual	administration	and	smaller	group	sessions.	Working	with	the	data	collection	contractor	to	assess	

these	students	will	place	even	more	of	a burden	on	the	participating	schools.	A	baseline	school	incentive	

amount	of	$200	is	consistent	with	the	amount	that	NCES	offers for	participation	in	other	studies, such	as	the	

ECLS-K,	ECLS-K:2011,	TIMSS,	and	the	Program	for	International	Student	Assessment	(PISA). However,	based	

on	previous	difficulties	in	recruiting	schools	for	the	originally	approved	MGLS	field	test	recruitment	and	the	

general	decline	in	school	participation	in	NCES	longitudinal	studies	over	the	years,7 we	propose	to also	test	

offering	one	third	of	the	sample	schools $400 (Condition	2), and	one	third	of	schools	a choice	of	one	of	seven	

non-monetary	incentives equivalent	to	$400 (Condition	3).	The	list	of	the non-monetary	incentive	choices is

provided in	Table	4.

Table	4. Non-Monetary	Incentive Choices for	Schools in	Experimental	Condition	3

Incentive Value

Registration	for	Association	for	Middle	Level	Education	(AMLE) or	Regional	Annual	Meeting $400

Two-Year	School	Membership	in	AMLE $400

Membership	in	Regional	ML	Organization	plus	Subscriptions to	Professional	Journals $400

Professional	Development	Webinar $400

Custom	Report	of	School	Results* $400

School	Supplies	 $400

Library	of	Middle	Level	Publications $400

*Report	type	will	depend	on	student	and	parent	response	rate	for	the	school.

School	coordinators	will	be	offered	a	$150 monetary	incentive.	They	play	an	especially	important	role	in	the	

study	and	are	critical	to	its	success.	The	coordinator	in	each	participating	school	will	coordinate	logistics	

with	the	data	collection	contractor;	compile	and	supply	to	the	data	collection	contractor	a	list	of	eligible	

students	for	sampling;	communicate	with	teachers,	students,	and	parents	about	the	study	to	encourage	their	

participation;	distribute	and	collect	parental consent	forms;	and	assist	the	test	administrator	in	ensuring	that	

the	sampled	students	attend	the	testing	sessions.

																																																												

7 For	example,	in	1998–99,	the	Early	Childhood	Longitudinal	Study	had	a	weighted	school-level	response	rate	of	74	percent,	whereas	12	years	later,	

the	complementary	ECLS-K:2011	study	had	a	weighted	school-level	response	rate	of	63	percent.
15

A.10 Assurance	of	Confidentiality

NCES	is	authorized	to	conduct	this	study	under	the	Education	Sciences	Reform	Act	of	2002	(20	U.S.	Code,	

Section 9543).	By	law,	the	data	provided by	schools,	staff,	parents,	and	students may	be	used	only	for	

statistical	purposes	and	may	not	be	disclosed	or	used	in	identifiable	form	for	any	other	purpose	except	as	

required	by	law	(20	U.S.	Code,	Section 9573). The	laws	pertaining	to	the	collection	and	use	of	personally	

identifiable	information	will	be	clearly	communicated in	correspondence	with	states,	districts,	schools,	

teachers,	students,	and	parents.	Letters	and	informational	materials	will	be	sent	to	parents	and	school	

administrators	describing	the	study,	its	voluntary	nature,	and	the	extent	to	which	respondents	and	their	

responses	will	be	kept	confidential. A	request	for	a	list	of	middle	grade	students	with	IEPs	will	be	requested	

from	school	districts	and/or	schools	under	FERPA	exception (34	CFR	Part	99.31).	This	information	will	be	

used	for	sampling	purposes	only and	will	be	securely	destroyed	once	student	samples	are	drawn.

The	confidentiality	plan	developed	for	the	MGLS:2017	requires	that	all	contractor and	subcontractor	

personnel	and	field	workers	who	will	have	access	to	individual	identifiers sign	confidentiality	agreements	

and	notarized	nondisclosure	affidavits.	The	plan	also	requires	that all personnel	receive	training	regarding	

the	meaning	of	confidentiality,	particularly	as	it	relates	to	handling	requests	for	information	and	providing	

assurance	to	respondents	about	the	protection	of	their	responses. NCES	understands	the	legal	and	ethical	

need	to	protect	the	privacy	of	the	MGLS:2017 respondents	and	has	extensive	experience	in	developing	data	

files	for	release	that	meet	the	government’s	requirements	to	protect	individually	identifiable	data	from	

disclosure.	The	data	files,	accompanying	software,	and	documentation	will	be	delivered	to	NCES by	the	data	

collection	contractor at	the	end	of	the	project.	Neither	names	nor	addresses	will	be	included	in	any	data	file.

A.11 Sensitive	Questions

The	recruitment	effort	does	not	involve gathering	information	considered	to	be	of	a	sensitive	nature.	A	list	of	

middle-grade	students	with	IEPs	will	be	requested	from	school	districts, or	schools	where	appropriate,	

under	FERPA	exception	(34	CFR	Part	99.31).	This	information	will	be	used	for	sampling	purposes	only and	

will	be	destroyed	once	student	samples	are	drawn.	All	district	and	school	personnel	facilitating	the	conduct

of	the	study	and	developing	the	sampling	frame	will	be	informed	of	the	privacy	and	confidentiality	protocols	

required	for	the	study,	including	those	having	to	do	with	the	sample	lists	of	schools	and	students.

A.12 Estimates	of	Burden

Table	5 shows	the	expected	burden	for	districts,	schools, and	parents	during	the	Item	Validation	Field	Test	

and	Operational	Field	Test	recruitment	activities.

For	the	Item	Validation	Field	Test,	as shown in	Part	B,	section	B.1.1 we	anticipate	contacting	approximately	

92	schools	to	reach	the	approximately	51	schools	needed	for	participation,	and	contacting	the	parents	of	

approximately	6,172	students	to	yield	approximately	3,950	participating	students.	For	the	Operational	

Field	Test,	as	shown	in	Part	B,	section	B.2.1,	we	anticipate	contacting	approximately	90	schools	to	reach	the	

approximately	50	schools	needed	for	participation,	and	contacting	the	parents	of	approximately	1,750	

students to	yield	approximately	1,120	participating students.	In	both	field	tests,	in	order	to	draw	samples	of	

students	with	disabilities,	we	may	need	to	obtain	student	records	information	from	up	to	four	districts per	

field	test.	We anticipate	needing to	contact	up	to	12	districts	to	gain	participation	from	four.

We	estimate	that	it	will	take 10 minutes	on	average	for	school	and	district	administrators	to	review	the	

materials	and	either	agree	or	decline	to	participate.	For	those	participating,	we	estimate	an	additional	4	
16

hours	for the	provision	of student	rosters (including	information	about	students	for	sampling,	contact	

information	for	the	sample	students’	parents,	and	sample	students'	math	and	special	education	teachers;	see	

Appendices S and	T). For	student’s	parents,	we	estimate	that	it	will	take	up	to	10 minutes	to	review	the	

recruitment	materials	and either	consent	or	refuse	to	participate (on	behalf	of	their	student	and	themselves).

The	provision	of	student	rosters	and	the	parents’ consent forms	will serve	as	sources	for	parents	contact	

information,	which	during	the	data	collection	period	can	be	used	for	nonresponse	follow-up.

Table	5. Recruitment	Burden	Estimates for Schools	and	Parents

Recruitment

Number	

contacted

Average	

burden	time	

(minutes)

Total	burden	

(hours)

Respondent	

average	

hourly	wage1

Estimate	of	

respondent	

labor	cost

Item	Validation	Field	Test

Nonparticipating	districts 8 10 2 $44.18	 $88.36	

Participating	districts 4 250 17 $44.18	 $751.06	

Nonparticipating	schools 41 10 7 $43.67	 $305.69	

Participating	schools 51 250 213 $43.67	 $9,301.71	

Students’	parents 6,172 10 1,029 $22.76	 $23,420.04	

Total 6,276 - 1,268 - $33,866.86	

Operational	Field	Test

Nonparticipating	districts 8 10 2 $44.18	 $88.36	

Participating	districts 4 250 17 $44.18	 $751.06	

Nonparticipating	schools 40 10 7 $43.67	 $305.69	

Participating	schools 50 250 208 $43.67	 $9,083.36	

Students’	parents 1,750 10 292 $22.76	 $6,645.92	

Total 1,852 - 526 - $16,874.39	

Total 8,128 - 1,794 - $50,741.25	

1 Average	wages	use	the	Bureau	of	Labor	Statistics	(BLS)	May	2013	National	Occupational	and	Employment	Wage	Estimates.	Salaries	estimated	by	the	

BLS	are	computed	assuming	2,080	hours	per	year.

A.13 Total	Annual	Cost	Burden

There	are	no	respondent	costs	other	than	the cost	associated	with	response time burden.

A.14 Annualized	Cost	to	Federal	Government

The	estimated	cost	to	the	federal	government	for	contractor	and	subcontractor	work	to	conduct	the	Item	

Validation	Field	Test and	the	Operational	Field	Test will	be	provided	by	early	May	2015.

A.15 Program	Changes	or	Adjustments

In	June	2014,	NCES	obtained	OMB	clearance	for	district and	school	recruitment	for	the	MGLS:2017	Item	

Validation	Field	Test.	The	recruitment	was	scheduled	to	begin	in	the	summer	of	2014	with	data	collection	

beginning	in	February	2015.	NCES	has	since	revised	the	schedule	and	plan	for	the	MGLS:2017	developmental	

work.	Both	the	item	validation	field	test	and	the	operational	field	test	will	be	conducted	starting	in	January	

2016,	with	school	recruitment	beginning	in	the	summer	of	2015,	as	soon	as	OMB	approval is	received.	As	a	

result,	NCES	requested	to	discontinue	the	original	approval	for recruitment	(OMB#	1850-0911	v.1),	and	this	

submission	seeks reinstatement	of	the	1850-0911	OMB	number	with approval	for	recruitment	for	both	

MGLS:2017	field	tests.
17

A.16 Plans	for	Tabulation	and	Publication

The results	from	the	Item	Validation	Field	Test	and	the	Operational	Field	Test	will	be	presented	in	a	single	

field	test	report	released	approximately	6	months	after	the	completion	of	the	field	tests.

Table	6.		Schedule	for	Item	Validation	Field	Test	and	Operational	Field	Test

Activity Start	date	 End	date

Recruitment	of	school	and	districts June	2015 December	2015

Recruitment	of	students	and	parents	through	requesting	

parent	consent	from	parents

September	2015 December	2015

Item	Validation	Field	Test January	2015 March	2015

Operational	Field	Test January	2015 March	2015

Field	Test	Report ‒ September	2015

A.17 Display	OMB	Expiration	Date

The	OMB	expiration	date	will	be	displayed	on	all	recruitment	materials.

A.18 Exceptions	to	Certification	Statement

No	exceptions	to	the	certification	statement	are	requested	or	required.