Middle	Grades	Longitudinal	Study	of	2016–17	(MGLS:2017)

Recruitment for

2016	Item	Validation	and	Operational	Field	Tests

Request	for	OMB	Clearance

OMB#	1850-0911	v.3

Supporting	Statement	Part	B

National	Center	for	Education	Statistics

U.S.	Department	of	Education

Institute	of	Education	Sciences

Washington,	DC

February 2015
ii

Table	of	Contents

B.	Collection	of	Information	Employing	Statistical	Methods................................................................................. 1

B.i Item	Validation	Field	Test......................................................................................................................................... 1

B.i.1	Respondent	Universe ......................................................................................................................................... 1

B.i.2 Procedures	for	the	Collection	of	Information .......................................................................................... 2

B.i.3 Maximizing	Participation.................................................................................................................................. 5

B.i.4 Purpose	of	Field	Test	and	Data	Uses............................................................................................................ 6

B.ii Operational	Field	Test............................................................................................................................................... 6

B.ii.1	Respondent	Universe........................................................................................................................................ 6

B.ii.2 Procedures	for	the	Collection	of	Information......................................................................................... 7

B.ii.3 Maximizing	Participation................................................................................................................................ 7

B.ii.4 Purpose	of	Field	Test	and	Data	Uses .......................................................................................................... 7

B.5 Individuals	Consulted	on	Study	Design............................................................................................................ 7
1

B.	Collection	of	Information	Employing	Statistical	Methods

Part	B	of	this	submission	presents	information	on	the	collection	of	information	employing	statistical	methods.	

As	the	methodological	approaches	for	the	Item	Validation	Field	Test	(IVFT)	and	the	Operational	Field	Test	

(OFT)	are	distinct,	part	B	presents information	pertaining	to	IVFT in	sections B.i.1-B.i.4 and	to	OFT in	sections

B.ii.1-B.ii.4. This	OMB	clearance	request	is	limited	to	the	recruitment	of	schools	and	participants	for	the	two	

field	tests.	A	separate	OMB	clearance	request	with	details	of the	field	test	data	collections will	be	submitted

later	in	2015. Please	note	that	the sampling	methodology	provided	in	this	document	may	be	subject	to	change	

upon	award	of	the	MGLS:2017	data	collection	contract.	Any	revisions	will	be	submitted	to	OMB	for	approval	as	

a	change	request	as	soon	as	they	become	available.

B.i Item	Validation	Field	Test (IVFT)

B.i.1	Respondent	Universe

The	MGLS:2017	Item	Validation	Field	Test (IVFT)	will	include 5th through	8th grade	students and	will	be	

conducted	during	the	2015-2016	school	year,	with	data	collection	scheduled	to	begin in	January	2016.	The	

IVFT	will	use	a	convenience	sample	of schools representing the	variability	in	student	achievement	that	is	

required	for	robust	psychometric	analysis	of	the	MGLS	reading,	mathematics,	and	executive	function	

assessment,	and	will	also	aim	to	maximize	geographic	diversity	to	the	extent	possible. There	may	be	some	

overlap	in	the	districts	and	states	in	which	sample	schools	will	be	recruited	for both	IVFT	and	OFT,	but	no	

school	will	be	asked	to participate	in	both	field	tests.

While	not	required	to	be	nationally	representative for	psychometric	analysis,	the	IVFT will	include	students	in	

the	typical	age	range	found in	grades	6–8 in	the	United	States and	in	the typical range	of	abilities on	the	

constructs	being	measured	by	the	MGLS	item	pool.	To	help	ensure	that	the	assessments	that	will be	used	

during	the	2017	MGLS	base	year	will	be appropriate	for	6th grade	students with	lower	achievement	level	in	

mathematics, the	field	test	sample	will	also	include	a	selection	of	5th graders.	In	addition,	the	sample	will	

include a	subset	of	students	from	the	three	focal	disability	groups	(specific	learning	disability,	autism,	and	

emotional	disturbance)	who	are	able	to	take	standardized	tests	using	accommodations. Schools	will	be	

selected	for	recruitment	from	the	most	recent	available universe	files:	the	2012-13 Common	Core	of	Data	

(CCD)	to	develop	the	public	school	list	and	from	the	2012-13	Private	School	Universe	Survey	(PSS)	to	develop	

the	private	school	list.	The	less restrictive	sampling	requirements	of the IVFT mean	that	schools	that	require	

extensive	research	applications may not	be	included.

Students

For	IVFT to	support	psychometric	analyses, at	minimum	approximately 400	student	responses	should	be	

obtained	per	direct assessment	item	and	200	per	survey item.	To	support	the	number	of	responses	per	item,	

the	sample	yield	needs	to	be at	least	1,200	students	from	each	of	6th,	7th,	and	8th	grades.	The	sample	will also	

include	schools	with	a	grade	5	so	that	an	additional	5th	grade	sample	of	350	students	can	be	selected	in	order	

to	examine	possible	floor	effects	of	the	mathematics	items. Within	the yield	of	3,950 students,	the	sample	will	

include at	least 350	students	in	the	three	focal	disability categories (as	defined	in	IDEA	section	300.8):	150	

students	with	a	specific	learning	disability,	100	students	with	autism, and	100	students	with	an	emotional	

disturbance. In	the	IVFT,	unlike	in	the	OFT, the	students	with an Individualized	Education	Program	(IEP)

required	for	Differential	Item	Function	study	will	not	be	sampled	separately	but	will	be	sought	first	as	part	of	

the	sample	drawn	in	general	education	schools.	If	the	numbers	of	students	with IEPs are	not	adequate,	

districts may	be	contacted directly,	similarly to	the	procedures	outlined below	for	the	OFT. Disability	codes

will	be	either	obtained	from	schools	as	part	of	the	initial	student	roster	collection	or	from	districts	(see	

appendix	S-T) when	the	information	is	not	available	or	easily	obtainable	from	the	school,	or	when	additional	

students	in	particular	disability	categories	are	needed	to	complete	the	sample.	If	needed,	in the	IVFT,	special	

schools	(where more	than	80%	of	students	have	an IEP) may	be	recruited to	reach	the	target sample	sizes in	

the	three	disability categories.

The	field	test	data	collection	will	be	conducted	with	as	many	students	as	each	school is willing	to	make	

available,	with	an	estimated	average	of	120	students	per	school. Based	on	the	number	and	grades	of	students	
2

the	school	will	allow	to	participate	in	MGLS,	the	study staff	will	collect	rosters	and	draw	a	sample	of	students	

within	grades	5-8	and	of	appropriately	aged	students	in	ungraded	classrooms to	obtain	the	maximum	number	

of	students	permitted.	Purposively,	all	eligible	students	identified	as	having	specific	learning	disability,	autism,

or	emotional	disturbance	will	be	included	in	the	sample.

Depending	on	each	school’s	policy,	either	active	(explicit) or	passive (implicit)	parental consent	materials	

(appendices	F-G)	will	be	distributed	to	the	sampled	students,	with	an	estimated 80 percent rate	of	granted	

consents. Among	those	with	granted	consent,	an	estimated	80	percent	of	students	are	expected	to	be	present	

and	take	the	assessments.	Therefore, to	get	to	a	yield	of	3,950	assessed	students,	the	parents	of	approximately	

6,172	students	will	need	to	be	contacted	for	consent	(6,172*0.8*0.8=3,950).

Schools

To	send	consent	materials	to parents	of	6,172 students,	with an	average of	120	students	per	school	for	grades	

6,	7,	and	8,	we	will	need	to	recruit approximately	51	schools.	The	High	School	Longitudinal	Study	of	2009	

(HSLS:09)	reported a	base-year	weighted	school response	rate	of	55.5	percent.	Assuming the	same	response	

rate in	MGLS,	a sample	of	approximately	92	schools	will	need	to	be	selected.

If	a	selected	school	offers	grade 5,	the	goal	will	be	to	assess	all	of	the	5th-graders	within	the	school.	If	the	

participating	schools	do	not	provide	sufficient numbers	of	5th-grade	students	to	meet	the	sample	targets,	we	

will	supplement	the	initial	school	sample	with schools	serving 5th-grade	students.	We	will	first	seek	schools	

with	5th	grades	in	close	proximity	to	other schools participating in	MGLS.	If	additional	schools	need	to	be	

recruited	to	obtain	the	5th-grade	targets,	they	will	be	selected for	cost	effectiveness	of	field	administration.

Also,	if	the	schools	do	not	provide	sufficient	numbers	of	students	within	the	three	focal	disability categories to	

meet the	sample	yield	targets,	we	will	supplement	participating	schools with	a	sample	of	special	schools	

where more	than	80	percent	of	the	students	have	been	identified	as	having	an	IEP.	We	will	initially	seek	to	

identify	these schools	in	or	near districts	that	already	contain schools	that	have	agreed	to	participate.	If	we	

need	to	recruit	additional	special	schools,	we	will	choose	schools	that	allow the	most	cost	effective	field	

administration, given	the	geographic	deployment	of	the	field	assessment	staff.

B.i.2 Procedures	for	the	Collection	of	Information

School	Recruitment	Approach

Gaining	schools’	cooperation	in	voluntary	research	is	increasingly	challenging.	For	example,	in	1998–99	the	

Early	Childhood	Longitudinal	Study	had	a	weighted	school-level	response	rate	of	74	percent,

1 whereas	12	

years	later,	the	complementary	ECLS-K:2011	study	had	a	weighted	school-level	response	rate	of	63	percent.2

Additionally,	there	is	evidence	that	response	rates	may	be	lower	for	schools	that	serve	older	students,	as	in	the	

High School	Longitudinal	Study	of	2009,	which had	a	weighted	school-level	response	rate	of	56 percent.3

Therefore,	effective	strategies	for	gaining	the	cooperation	of	schools	are	of	paramount	importance. Schools	

will	be	recruited	both	directly	and	at	the	district	level.

Direct	School	Recruitment. In	the	interest	of	time,	a sample	of	schools	that	do	not	require	district	level	

approval will	be	identified	for	direct	recruitment. As	part	of	the	school	recruitment,	we	will	send	an	advance	

notification	letter	to	their	state	(appendix	B)	and	district	(appendix	C)	education	agencies,	and	then	a	trained	

school	recruiter	will	initiate	contact	with	the	school.	Schools	will	receive	an	attractive,	introductory

information	packet	via	FedEx	that	includes	a	cover	letter	(appendix	D),	a	colorful	recruitment-oriented	

brochure	(appendix	H),	and	a	sheet	of	Frequently	Asked	Questions	(FAQs)	about	the	study	(appendix	I) with

																																																												

1 Tourangeau,	K.,	Nord,	C.,	Lê,	T.,	Sorongon,	A.G.,	Hagedorn,	M.C.,	Daly,	P.,	and	Najarian,	M.	(2001).	Early	Childhood	Longitudinal	Study,	Kindergarten	Class	

of	1998–99	(ECLS-K),	User’s	Manual	for	the	ECLS-K	Base	Year	Public-Use	Data	Files	and	Electronic	Codebook	(NCES	2001-029).	U.S.	Department	of	

Education.	Washington,	DC:	National	Center	for	Education	Statistics.

2 Tourangeau,	K.,	Nord,	C.,	Lê,	T.,	Sorongon,	A.G.,	Hagedorn,	M.C.,	Daly,	P.,	and	Najarian,	M.	(2012).	Early	Childhood	Longitudinal	Study,	Kindergarten	Class	

of	2010–11	(ECLS-K:2011),	User’s	Manual	for	the	ECLS-K:2011	Kindergarten	Data	File	and	Electronic	Codebook	(NCES	2013-061).	U.S.	Department	of	

Education.	Washington,	DC:	National	Center	for	Education	Statistics.

3 Ingels,	S.J.,	Pratt,	D.J.,	Herget,	D.R.,	Burns,	L.J.,	Dever,	J.A.,	Ottem,	R.,	Rogers,	J.E.,	Jin,	Y.,	and	Leinwand,	S.	(2011).	High	School	Longitudinal	Study	of	2009	

(HSLS:09).	Base-Year	Data	File	Documentation	(NCES	2011-328). U.S.	Department	of	Education. Washington,	DC:	National	Center	for	Education	

Statistics.
3

links for	accessing	the	MGLS:2017	recruitment	website,	following the	study on	NCES’s	Twitter	account,	and

joining the	study’s	Facebook	page.	The	Twitter	and	Facebook	accounts	will	provide	an	outreach	service,	

offering	updates	on	the	progress	of	the	study and posting	information	on	recent	articles	and other	material	

relevant	to	education	in	the	middle	grades.

Within	3 business	days of the	information	packet	delivery (confirmed	via	FedEx),	a	school	recruiter	will	follow	

up	with	a	phone	call	to	secure	the	school’s	cooperation	and	answer	any	questions	the	school	may	have.	During	

this	call,	the	recruiter	will	establish	who	from	the	school’s	staff	will	serve	as	the school	coordinator	for	the

study.	The	MGLS:2017	study	team	will	then	work	with	the	school	coordinator	to	schedule	IVFT activities	at	the

school,	including	gathering	student	rosters,	distributing	consent	materials	to	parents	of	sample	students,	and	

arranging	the	onsite	assessments.	In	early	communications, the	recruiter	will	also	gather	information	about	

what	type	of	parental	consent	procedures	need	to	be	followed at	the	school; any requirements	for	collecting	

data	on	the	IEP	status of	students	and	student-teacher	rosters;	hours	of	operation,	including	early	dismissal	

days,	school	closures/vacations,	and	dates	for	standardized	testing;	and	any	other	considerations	that	may	

impact scheduling	student	assessments (e.g.,	planned	construction	periods,	school	reconfiguration,	or	planned	

changes	in	leadership). Participation refusals	will	be	referred	to	an	MGLS:2017	data	collection	task	leader	and	

examined	for	possible	approaches	to	refusal	conversion.

District-Level Recruitment. In	addition	to	school	recruitment,	some	districts	may	be	asked	for	student	rosters

to	enable	MGLS	staff	to	recruit	enough	students	in	the	three	focal disability	categories,	or	they	may	be	asked	to	

endorse	the	study	to	aid	the	study’s	school	recruitment	efforts.	Generally,	for	district-level	recruitment,	a

similar	protocol	will	be	followed	as	with	direct	school	recruitment.	Districts	will	receive	an	introductory

information	packet,	including	a	cover	letter	(appendix	C),	brochure	(appendix	H),	and	FAQs	(appendix	I).	

Three	days	after	mail	delivery of	the	packet,	a	recruiter	will	make	a	call	to	secure	the	district’s	cooperation,	

answer	any	questions	the	superintendent	or	other	district	staff	may	have,	review	the	list	of	schools	in	the	

district	serving	grades	6–8,	confirm	key	information	about	the	schools	(e.g.,	grades	served,	size	of	enrollment,	

enrollment	of	students	with	disabilities),	and	discuss	obtaining	students’	IEP	information that	is	necessary	for	

drawing	the	IVFT	student	sample.	Information	collected	during	this	call	will	be	used	to	confirm	which	schools

in	the	district	are eligible for	participation	in	the study, and	to	obtain contact	and	other information	helpful	in

school	recruitment.

Once	district	cooperation	has	been obtained	and	the	school	sample	has	been	selected,	we	will	complete	any	

required	additional district-level	paperwork,	such	as research	applications.	Recruitment	of	schools	in	these	

districts	will	follow	the	same	protocol	as	outlined	above	under	“Direct	School	Recruitment”,	with	the	

additional	inclusion	in	the	school	introductory	information	packet of	the	district’s	letter	of	approval	and	

support. Participation refusals	by	districts	or	individual	schools	will	be	referred	to	an	MGLS:2017	data	

collection	task	leader	and	examined	for	possible	approaches	to	refusal	conversion.

Private	and	Charter	School Recruitment.	If	a private	or	charter	school	selected	for	the	field	test	operates under	

a	higher	level	governing	body	such	as	a	diocese, a	consortium of	private	schools,	or	a	charter	school	district,	

we	will	use	the	district-level	recruitment approach with	the appropriate	higher	level	governing	body.	If	a

private	or	charter	school	selected	for	the	field	test does not	have	a	higher	level	governing	body,	we	will	use	the	

direct	school	recruitment	approach outlined	above.

Student	Rosters	Collection.	Once	a	school	or	district	has	agreed	to	participate,	the	MGLS	contractor will	gather	

student	rosters	with	the	assistance	of	the	school	coordinator.	Or	in	the	case	of	the	OFT, where	districts	may	be	

directly	recruited	to	identify	students	with	disabilities, the	appropriate	district	personnel will	be	contacted.	

Gathering	information	on	the	students	will	be	a	two-step	process.	First,	a	complete	roster	of	all	students	

eligible	for	sampling will	be	requested,	including	key	student	characteristics,	such	as:	name;	ID	number;	date	

of	birth;	grade	level;	gender;	race/ethnicity;	and	IEP	status	with	disability	code(s),	when applicable.	Based	on	

this	information	the	student sample	will	be	drawn. Next,	to complete	the	roster information, within	3-5

business	days,	the	MGLS	contractor	will	request	from the	school	coordinator	or	designated	district	personnel	

the	following	information	for	each	student	selected	into	the	sample:	student’s	parent	and/or	guardian	contact	

information	(e.g.,	mailing	address;	landline	phone	number;	cell	phone	number;	e-mail	address);	student’s	

math	teacher;	and	student’s	special	education	teacher,	when applicable. If	the	school	and/or	district	is	

unwilling	to	provide	parent	contact	information,	the	MGLS	contractor	will	work	with	the	school	and/or	
4

district	to	determine	the	best	way	to	contact	parents	(e.g.,	the	school	coordinator	or	designated	district	

personnel	would	facilitate	contacting	parents	and/or	mail	the	required	materials	to	parents	using	the	contact	

information	they	have	on	file).

Schools	and	districts	will	be provided with	a	template	and	secure	transfer	options	to	deliver the	rosters

(appendices	S	and	T).	The	data	quality	of	the	student rosters	will	be	then	evaluated	by:

 reviewing	and	assessing	the	quality	and	robustness	of	student	and	parent	information	available	at	each	

school,	including	contact	information	for	parents;

 reviewing	and	assessing	the	quality	of	the	data	on	student-teacher	linkages;

 reviewing	and	assessing	the	quality	of	the	data	on	IEP	status;

 addressing any	incompleteness	or	irregularities	in	the	roster	file;

 requesting additional	information	as	needed	from	the	school	coordinator or	designated	district	

personnel;	and

 (re)verifying	that	the	sampled	students are currently	in	attendance in	the	school.

Parent	Recruitment

Information	about	schools’	procedures	for	obtaining	consent	for	students	to	participate	in	the	study will	have	

been	gathered during	school	recruitment.	Schools	generally	require	one	of	two	types	of	consent:	implicit	or	

explicit.	Both	types	of	consent	require	that	parents	be	notified	that	their	children	have	been	selected	for	the	

study.	With	implicit	consent,	the	school	does	not	require	verbal or	written	consent	for	a	student to	participate

in	the	study	– parents	are	asked	only	to	notify	the	appropriate	person	if	they	do	not	want	their	child	to	

participate	(appendix	F).	With	explicit	consent,	children	may	participate	only	if	their	parents	provide	written	

or	oral	consent	for	their	children	to	do	so	(appendix	G). In	the	field	test,	proactive	parent	recruitment	will	be	

focused	on	maximizing	the	number	of	parents	(1)	returning	signed	explicit	consent	forms and	(2)	completing

the	parent	survey.

After the	student sample	is	drawn	within	a	school,	the	initial	communication	with	parents	consisting	of	

introductory	and	consent	materials	(Appendixes	F-I)	will	be	distributed	to	parents	in	a	way	each	school	

believes	to	be	most	appropriate	and	effective	(e.g.,	sending	the	materials	home	with	students;	the	school	or	

district	sending	the	materials	directly	to	parents;	and/or	trained	MGLS	recruitment	staff	contacting	parents	

directly	by	mail,	email,	and/or	phone).	The	initial	materials	will	introduce	the	study,	explain	the	study’s

purpose	and	the	importance	of	student	and	parent	participation,	describe what	is	involved	in	participation,	

and	specify	the	consent	procedure	that	is	being	used	by	their	school. The	materials	will	include	a	consent

seeking	letter	to	all	parents plus	a	consent	form	where	explicit	consent	is	required (appendices	F-G),	a	colorful	

recruitment-oriented	brochure	(appendix	H),	and	a	sheet	of	FAQs	about	the	study	(appendix	I) with links for	

accessing	the	MGLS:2017	recruitment	website,	following the	study on	NCES’s	Twitter	account4,	and joining the	

study’s	Facebook	page.	The	Twitter	and	Facebook	accounts	will	provide	an	outreach	service,	offering	updates	

on	the	progress	of	the	study and posting	information	on	recent	articles	and other	material	relevant	to	

education	in	the	middle	grades. Additionally,	in	schools	using	explicit	consent,	the	parental	consent form	for	

student’s participation,	which	will	be	included	in	the	initial	communication	materials,	will	ask	parents	to	

provide	their	contact	information (appendix	G).

The	IVFT	(but	not	the	OFT)	will	include	a	parent	incentive	experiment.	All	parents	of	sampled	students	within	

the	same	school	will	be	assigned	to	the	same	incentive	level.	Within	the	schools	selected	for	a parent	incentive

other	than	$0,	the	$20	or	$40	incentive	amount	will	be	indicated	in	the	letter	that	will	accompany	the	initial	

parent	contact	materials	and	will	be	referenced	on	the	explicit	consent	form	(appendices	F-G).

In	schools	that	use	an	explicit	consent	procedure,	the	methods	that	will	be	used	to	follow-up	with	parents	who	

do	not	respond	to	the	initial	contact	materials	will	depend	on	whether	the	school	or	district	provided	contact	

information	for	these	parents	and on	the	ways	each	school	considers	to	be	appropriate	for	contacting	parents	

(either	through	the	school’s	staff	or	by	MGLS	trained	staff	directly).	The	non-response	follow-up	will	focus	on	

																																																												

4 Note that Twitter and Facebook accounts can be set to disallow for responses or re-tweeting, thereby avoiding participant 

disclosure.
5

assuring	that	parents	receive	the	introductory	and	consent	materials,	encourage	them	to	sign	and	return	the	

consent	form,	and	answer any	questions	they	may	have.	In	schools	that	use	an	implicit	consent	procedure,	no	

follow-up	will	be	required	given	that	absence	of	parents’	reply	indicates	consent.

If	parents	indicate	that	they	would	prefer	not	to	participate	at	a	rate	that	suggests	refusal	conversation	is	

necessary	for	the	success	of	the	field	test,	trained	recruiters	may	contact	parents	in	an	attempt	to	convert	

refusals	by	addressing any	concerns	parents	may	have	over	participation	in	the	study.	Depending	on	the	mode	

of	initial	contact,	parents	may	be	contacted	by	phone,	email,	or	a	mailing	sent	home	from	the	contractor	or	

from	the	school	or	district	personnel. Refusal	conversion	experiences	in	the	IVFT and	OFT	field	tests	may	

inform	parent	recruitment	materials	and	procedures	that	will be	used	in the	main	study.

B.i.3 Maximizing	Participation

Studies	increasingly	experience	challenges	in	obtaining	the	cooperation	of	districts	and	schools.	Loss	of	

instructional	time, competing	demands	(such	as	district	and	state	testing	requirements), lack	of	teacher	and	

parent	support, and	increased demands	on	principals	impede	gaining	permission	to	conduct	research	in	

schools.	MGLS:2017 recruitment	teams	will	be	trained	to	communicate	clearly	to	districts,	dioceses,	private

school	organizations,	schools,	teachers,	parents,	and	students the	benefits	of	participating	in	the	field	test	and	

what	participation	will	require	in	terms	of	student	and	school	personnel	time.	MGLS	staff will	seek	to	utilize

conferences	to	educate	middle	grades	professionals	about	the	study and	its	field	tests	and	to	increase	MGLS

name	recognition.	Recruiters	will	be	trained	to	address	concerns	that	districts	and	schools	may	have	about	

participation,	while	simultaneously	communicating	the	value	of	the	study and	the	school’s key	role	in	

developing	instruments	that	ensure	high-quality	data	focusing	on	middle-grade	students.	Early	engagement	of	

districts	and	school	administrators	will	be	important.	The	study	will	also	offer	monetary	incentives	to	schools	

(see Part	A), which	have	proven	in	other	NCES	studies	to	increase school	participation	rates.

Along	with	offering	monetary	incentives,	our	plan	for	maximizing	district, school	administrator,	and	parent

engagement	includes the	following:

 Experienced	recruiters.	The	recruiting	team	will	include	staff	with	established	records	of	successfully	

recruiting	school	districts	and	schools.	To	maximize	district	approval,	senior	staff	will	make	the	initial	

district	telephone	contacts.	Their	familiarity	with	the	study	and	its	future	impact,	as	well	as	their	

experience	in	working	with	districts	to	overcome	challenges	to	participation,	will	be	crucial	to	obtaining	

district	approval.	Recruiters	contacting	schools	will	be	equally	adept	at	working	with	school	

administrators	and	providing	solutions	to	the	many	obstacles associated	with	student	assessments,	

including	conflicts	related	to	scheduling	and	assessment	space,	minimizing	interruption	to instructional	

time,	and	obtaining	teacher	and	parent	buy-in.

 Persuasive	written	materials.	Key	to	the	plan	for	maximizing	participation	is developing	informative	

materials and professional	and	persuasive	requests	for	participation.	The	importance	of	the	study	will	be	

reflected	in	the	initial	invitations	from	NCES	(appendices	C-G)	sent	with	a	comprehensive	set	of	FAQs

(appendix	I)	and	a	colorful	recruitment-oriented	brochure	describing	the	study	(appendix	H).	Reviewing	

these	study	materials should provide	districts	and	school	administrators with	a	good understanding	of	the	

study’s	value,	the	importance	of	the	field	test,	and	the	data	collection	activities	required	as	part	of	the	

study.	A	full	understanding	of	these	factors	will	be	important	both	to	obtain	cooperation	and	ensure	that	

schools	and	districts	accept	the	data	collection	requests	that	follow.

 Persuasive	electronically	accessible	materials.	In	addition	to	written	materials,	we	will	develop	a	

recruitment-focused	website which, drawing heavily	on	the	written	materials,	will	present	clear	and	

concise	information	about	the	study	and	convey	the	critical	importance	of	participating	in	it.	The	study’s	

Twitter	and	Facebook	accounts	will	provide	an	outreach	service,	offering	updates	on	the	progress	of	the	

study and posting	information	on	recent	articles	and other	material	relevant	to	education	in	the middle	

grades.

 Buy-in	and	support	at	each	level.	During	district	recruitment,	the	study	team	will	seek	not	just	

permission	to	contact	schools	and	obtain	student	rosters	but also	to	obtain support	from	the	district.	This	

may	take	the	form	of	approval	of a	research	application	and a	letter	from	the	district’s superintendent	
6

encouraging	schools	to	participate.	Active	support	from	a	higher	governing	body	or	organization,	such	as	a	

district	or	a	diocese,	encourages	cooperation of	schools.	Similarly,	when	principals	are	interested	in	the	

research	activity,	they	are	more	likely	to	encourage	teacher	participation	and	provide an	effective	school	

coordinator.

 Avoiding	refusals.	MGLS	recruiters	will	work	to	avoid	direct	refusals	by	focusing on	strategies	to	solve	

problems	or	meet	obstacles to	participation faced	by	district	or	school	administrators.	They will	endeavor	

to	keep	the	door	open	while	providing	additional	information	and seeking	other	sources	of	persuasion.

B.i.4 Purpose	of	Field	Test	and	Data	Uses

The	IVFT is	designed	to	obtain	the	cooperation	of	enough	schools	in	order	to	sample	and	collect	sufficient	data	

from	students,	parents,	teachers,	and	administrators	to evaluate the	psychometric	properties	of	items	and	the	

predictive	potential	of	scale	variables.	This	will	allow valid, reliable	and	useful	assessment	and	survey

instruments to	be	composed	for	the	main	study. Additionally,	as	described	in	Part	A	section	A.9,	the	IVFT	will	

include	a	school	and	a	parent	incentive	experiment	to	determine	whether	different	levels	and/or	types	(in	

case	of	schools)	of	incentives	can	significantly	improve	participation	rates.

B.ii Operational	Field	Test (OFT)

B.ii.1	Respondent	Universe

The	MGLS:2017	Operational	Field	Test (OFT) will	be	conducted	during	the	2015-2016	school year,	with	data	

collection	scheduled	to	begin in	January	2016.	The	OFT	will be conducted	in	5-10 geographic	locations	that	

adequately	simulate	the	diversity	of	the	50	States	and	the	District	of	Columbia. The	OFT	may	include	up	to	two

quasi-nationally	representative	samples of	students	in	the	United	States.	The	Sample	1	universe	would	include

all	students	enrolled	in	6th grade	and	attending	general	education	schools,	and	Sample	2	would	include	

students	in	the	three	focal	disability	categories	who	are enrolled	in	6th grade	or	are	of	equivalent age	in	an	

ungraded	classroom.	Sample	2	would	be	used	if	Sample	1	proves	insufficient	to	meet	the	target student	sample	

yields for	the	three	focal	disability	categories,	and would	be	selected	from	district-level student	rosters.

The	MGLS:2017	will	employ	a	multi-stage	probability	proportional	to	size	sampling	design,	with	schools	

selected	in	one	stage,	and	students	selected	within	schools.	For	OFT	Sample	1,	schools	would be	selected	for	

recruitment	from	the	most	recent	available Common	Core	of	Data	(CCD	2012-13)	to	develop	the	public	school	

list	and	from	the	Private	School	Universe	Survey	(PSS	2012-13)	to	develop	the	private	school	list. In Sample	1,	

once	schools	are	selected and	recruited,	students	enrolled	in	6th grade	will	be	selected	from	the	provided	

student	rosters.	For	OFT	Sample	2,	once	the	geographic	regions	are	identified,	districts	would be	contacted	to	

provide	rosters	of	students	enrolled	in	6th grade	along	with	their	disability	codes	(see	appendixes	S-T).	

Students	in	the	autism,	specific	learning	disability,	and	emotional	disturbance categories	would be	sampled

from	the	provided	rosters,	and	their schools	would be	contacted	for	recruitment.

During	the	2016	administration	of	the OFT,	students	will	participate	in	the	full	protocol	as	designed	for	

implementation	in	the	main	study,	including student	assessments;	student	surveys;	and	surveys	of	the	

students’	parents,	math	teachers,	special	education	teachers	(when	applicable),	and	school	administrators.	In	

the	subsequent	two	OFT	follow-ups planned	for	2017	and	2018, when	most	students	will	be	in	7th	grade	and	

then	8th grade,	respectively,	only	student	tracing	and	tracking	activities	will	be	conducted	to	test	approaches,	

materials,	and	procedures	needed	to	locate	the sample students	as	they	move	and	change	schools.

Students

The	specified	desired	yield	for	OFT is	1,120 students	enrolled	in	6th grade	(included	in	the 1,120 cases is	a	

sample	of 120	students	in the three	focal	disability	categories). As	in	IVFT,	depending	on	each	school’s	policy,	

either	active	(explicit)	or	passive	(implicit)	parental	consent	materials	(appendices	F-G)	will	be	distributed	to	

the	sampled students,	with	an	estimated	80	percent	rate	of	granted	consents.	Among	those	with	granted	

consent,	an	estimated	80	percent	of	students	are	expected	to	be	present	and	take	the	assessments.	Thus	to	get	

to	a	yield	of	1,120	assessed	students,	the	parents	of	approximately	1,750	students	will	need	to	be	contacted	

for	consent	(1,750*0.8*0.8=1,120).
7

Schools

The	MGLS:2017	main	study	design	estimates	that	approximately	20–25 students	from	each	school	will	

participate,	which is	similar	to	the	number	of	students	who	participated	in each	school	in	the	ECLS-K:2011	

and	the	HSLS:09.	Therefore,	for	OFT to	mimic	procedures	in	the	main	study,	assuming	an average	

participation	of	22 students	in each	school,	and	a	target	yield	of	1,120	students total, we	will	need	to	recruit	

approximately	50	schools. As	in	IVFT,	based	on	the HSLS:09	base-year	weighted	school	response	rate	of	55.5

percent,	the	MGLS:2017	OFT	would	need	to	draw	a	sample	of	approximately	92	schools.

B.ii.2 Procedures	for	the	Collection	of	Information

For	sample	schools	that	do	not	require	research	approval	at	the	district	level,	recruitment	will	begin	at	the	

school	level;	otherwise	it	will	begin	with	a	research	application	at	the	district	level.	Additionally,	if	student	

Sample	1	does	not	yield	sufficient	numbers of	students	in	the	three	focal	disability	categories,	requests	for	

student	rosters	will be	sent	to	districts to	draw	student	Sample	2	that	supplements	Sample	1	such	that	target	

yields	in	the	three	focal	disability	categories	can	be	obtained.	In	all	other	ways,	the	recruitment	approach	for	

OFT will	be	the	same	as	that	described	for	IVFT in	section	B.i.2.	The	OFT	field	test	communication	and	

recruitment	materials	use	slightly	different	language	than	their	IVFT	equivalents	(see	Appendices A	through	I)

in	order	to reflect	the	fact	that: (a)	OFT	will	only	sample	6th grade	students	and their	ungraded	equivalents;	(b)

sample	students	and	parents	in	OFT	will	be	contacted	again	in	the	future	for	a	two-year	follow-up	study	of	

tracing	and	tracking;	and	(c)	all	parents	in	OFT	will	be	offered	a	$20	incentive.

Also,	in	the	OFT	(but	not	IVFT),	as	part	of	the	parent	survey,	we	will	ask	parents	for	additional	contact	

information	that	can be	used	to	reach	them	at	a	later	date	and	for the	contact	information	of	a	friend	or	

relative	who	may	know	how	to	contact	them	in	the	event	their	contact	information	is	no	longer	effective.	

While	the	tracking	and	tracing	activities	in	the	OFT	will	largely	rely	on	school	records,	this	contact	information	

may	be	used in	locating	the	student	and	family	over	time.	The	tracking	and	tracing	activities	in	the	OFT	will	

inform the	procedures	and	materials	for	tracing	and	tracking	sample	members	in	the	MGLS	main study.

B.ii.3 Maximizing	Participation

Similar	tactics	to	maximize	sample	members’	participation	will	be	used	in	OFT	as	in IVFT (see section	B.i.3).

B.ii.4 Purpose	of	Field	Test	and	Data	Uses

A	main	goal	of	the	OFT is	to	better	understand	the	recruitment	strategies	necessary	for	a	large-scale	nationally	

representative	effort to	obtain	the	targeted	sample	yield	of	6th-grade	general	education	students	and	students	

with	disabilities,	and	the subsequent	tracing	and	tracking	strategies	necessary	to	maintain	the	student	sample	

from	the	base	year	(when	sample	students will	be	in	6th	grade) to	the	first	follow-up	(when	most	of	the	

students will	be	in	7th grade) and	the	second	follow-up	(when	most	will	be	in	8th	grade).	As	described	in	Part	

A	section	A.9,	the	OFT	will	include	a	school	and	a	student	incentive	experiment	to	determine	whether	different	

types	and/or	levels	(in	case	of	schools)	of	incentives	can	significantly	improve	participation	rates.

B.5 Individuals	Consulted	on	Study	Design

The	following	individuals	are	responsible	for	the	MGLS:2017:

Carolyn	Fidelman,	NCES (202)	502-7312

Jeff	Owings,	NCES (202)	502-7423

Chris	Chapman,	NCES (202)	502-7414