# National Center for Education Statistics **(NCES) Middle Grade**  **s Longitudinal Study**  **of 2016**  **- 2017 (MGLS:2017)**  **Field Test 2015** 

**Recruitment**

**OMB #1850 -**  **NEW v.**  **1 Supporting Statement**  **Part A** 

March 6 , 2014 Revised May 28, 2014

## Ta **ble	of	Contents**

**PART A: Supporting Statement for Paperwork Reduction Act Submission ..........................**  **1 A.**  **Justification ................................**  **................................ ................................**  **........................ 1**  A.1 Circumstances Necessitating Collection of Information

................................ ................ 1 District and School Recruiting

................................ ................................ ............................. 4 Recruiter Training

................................ ................................ ................................ ................ 6 Monitoring

................................ ................................ ................................ ........................... 7 A.2 Purposes and Uses of Data

................................ ................................ .............................. 7 A.3 Use of Technology to Reduce Burden

................................ ................................ ............ 8 A.4 Efforts to Avoid Duplication of Effort

................................ ................................ ........... 8 A.5 Methods of Minimizing Burden on Small Entities

................................ ......................... 9 A.6 Consequences of Not Collecting Data

................................ ................................ ............ 9 A.7 Special Circumstances

................................ ................................ ................................ .... 9 A.8 Consultations outside the Agency

................................ ................................ ................... 9 A.9 Pay

ment or Gift to Respondents

................................ ................................ ................... 11 A.10 Confidentiality

of the Data

................................ ................................ ........................... 11 A.11 Additional Justification for Sensitive Questions

................................ .......................... 13 A.12 Estimates of Hour Burden

................................ ................................ ............................. 13 A.13 Estimate of Total Annual Cost Burden to Respondents or Recordkeepers

.................. 14 A.14 Annualized Cost to the Federal Government

................................ ................................ 14 A.15 Tabulation, Publication Plans , and Time Schedules

................................ ..................... 14 a. Tabulation Plans

................................ ................................ ................................ ............ 14 b. Publication Plans

................................ ................................ ................................ ............ 14 c. Schedule

................................ ................................ ................................ ......................... 15 A.16 Reasons for Program Changes or Adjustments

................................ ............................ 15 A.17 Exception to the Certification Statement

................................ ................................ ...... 15 REFERENCES

................................ ................................ ................................ ......................... 15 

## Ta **bles** Table 1. MGLS:2017 Field Test Instruments, Target Sample Y ield, Estimated Administration Time and Mode

................................ ................................ ................................ ............... 4 Table 2. Members of the MGL S:2017 Content Review Panels

................................ ............... 11 Table 3. Respondent Burden Estimates for MGLS:2017 Field Test Recruitment Activities

... 14 Table 4. Field Test Study Timeline of Activities

................................ ................................ ...... 15 

## Appendices Appendix A: Recruitment Letter for School Districts/Diocese Appendix B : Recruitment Letter for School Administrators Appendix C : Frequently Asked Questions Appendix D : Field Test Summary Appendix E : Guide for Recruitment Calls Appendix F : Project Staff Confidentiality Agreement1 

### PART	A: **Supporting	Statement	for	Paperwork	Reduction	Act Submission**  This package requests clearance to recruit school districts and schools for the spring 2015 field test for the Middle Grades Longitudinal Study of 2016

- 2017 (MGL S:2017). MGLS:2017 will be the first study sponsored b y the National Center for Educ ation Statistics (NCES) , within the Institute of Education Sciences (IES) of the U.S. Department of Education (ED) , to follow a nationally

- representative sample of students as they

enter and move through the middle grades (grades 6

- 8). The data collected through repeated measures of key constructs will provide a rich descriptive picture of the academic experiences and development of students during these critical years and will allow researchers to examine associations between contextual factors and student outcomes. There is a wealth of research highlighting the importance of mathematics and literacy skills for success in high school and subsequent association with later education and career opportunities, thus the study will focus on student achievement in these areas along with measures of student socioemotional wellbeing and other outcomes. The study will also include an oversample of students with different types of disabilities that will provide descriptive information on their outcomes, educational exp eriences, and special education services. Baseline data for the MGLS:2017 will be collected from a nationall y- representative sample of 6 th grade students in spring of 2017 with annual follow

- ups in spring 2018 and spring 2019 when most of the students in the sample will be in grades 7 and 8, respectively . In preparation for the national study , the data collection instruments and procedures mu st be field tested. NCES has contracted with Decision In formation Resources, Inc. and its partners Mathematica Policy Research and Education al Testing Service to conduct the field test in school year 2014

- 2015. NCES requests permission to contact and recruit public school districts and public and private schools to participate in the field test. Districts will be contacted for the purpose of obtaining their cooperation and assistance in sampling schools for the field test. Schools that are chosen to participate will be contacted to obtain their cooperation and to gather information that will be used to facilitate the selection of students and to plan for data collection visits to the field test schools. In this package, we present the field test sample design, including the plans for selecting districts, schools, and students, and the procedures for recruiting district s and schools in 2014. This package also provides an overview of the field test, including its design and data collection procedures. A separat e package will be submitted to request clearance for the field test data collection, including selecting students for the study and gathering parental consent for students to participate, securing the participation of parents, teachers, and school administrators, data collection ins truments and procedures, anal ys i s plan for field test data , and schedule for reporting of field test findings. A third package requesting permission to conduct cognitive laboratories for each field test instrument was submitted to OMB for approval on March 6 , 2014 (OMB# 1850

- 0803 v. 97 ). Findings from the cognitive laboratories will inform the instruments submitted with the field test data collection package. 

### A. **Justification A.1**  **Circumstances	Necessitating	Collection	of	Informati on**  The years between ages 10 and 14 are marked b y dramatic chang es in multiple domains of development. At that same time, students are experiencing many changes in their educational environments, for example, chang ing schools when transitioning from elementary to middle school , being taught b y multiple discipline

- specific teachers, and aging out of after

- school child care programs. However, most research focused on this developmental stage and on the education that adolescents receive during this period of life has relied on data from small, often non

- representative samples. These e xisting studies provide consistent evidence of declines in academic motivation, interest in school, and achievement across the early adolescent years
—

particularl y as adolescents m ake the transition to middle school or junior high school (see Eccles et al., 1993; Wigfield, et al, 2006) , marking this as an important educational juncture. The decline in academic motivation and disengagement from school is particularly2

notable for youth and adolescents who have already experienced educational or socioemotional troubles and individuals coming from lower socioeconomic and disadvantaged communities and families (see Wigfield et al., 2006). Despite growing evidence attesting to the importance of this developmental stage for understanding the perpetuation and production of educational and subsequent social inequality , there is generall y a lack of diverse, nationall y representative data to assist researchers and policy makers in understanding the individual, social, and contextual factors that influence youth outcomes. The federal government is uniquely

positioned to undertake the needed comprehensive large

- scale longitudinal study of a nationall y representative sample of youth that includes measures of known critical influences on adolescents’ academic and socioemotional trajectories. NCES is authorized to conduct MGL S:2017 under U
. S

. C

. 20 , Section 9543 (Education Sciences Reform Act of 2002)

. **National Study Design.** The MGL S:20 17 aims to obtain an understanding of students’ development and learning that occurs during the middle grade ye ars (grades 6 through 8) and that is predictive of future success, along with the individual, social, and contextual factors related to that positive development. A key goal of the study

is to provide researchers and policy makers with the information to better understand the school and non

- school influences associated with better mathematics and reading success, socioemotional health, and positive life development during this age period and beyond (for example, high school graduation, college and career readiness, healthy lifestyl es). Examples of the key questions the MGL S:2017 is being designed to answer include:

 How do students develop academically in the middle grades, and what factors are related to this development?

 What is the nature and extent of m athematics learning in the middle grades?

 What is the nature and extent of reading acquisition in the middle grades?

 What factors ( e.g., poverty , low parent education, and family structure ) are related to academic success of students in the middle grades ?

 What instructional and school factors are associated with more positive growth in m athematics ad reading in the middle grades?

 How is academic achievement in m athematics and reading related to other student outcomes, such as socioemotional well

- being and executive function? The study is also being designed to answer the above

- set of questions for students with different ty pes of disabilities. For example, do es the strength of the associations between mathematics and reading achievement and students’ executive function differ b y disability type? Inaddition, the study will provide answers to questions that are specific to students with disabilities , such as:

 To what extent are students with disabilities included in general education classrooms in middle school , and to what extent do they participate in general education curricula? Do these characteristics of their education change ov er time, and how?

 To what extent do parents participate in the IEP process and how satisfied are they with the services their children are receiving?

 What are students’ IEP goals, and how is progress towards these goals measured?

 For students who cannot p articipate in the standard MGL S:2017 assessments, how do their teachers rate their life skills (e.g., adaptive behaviors and independent functioning) and academic functioning at grades 6, 7, and 8 ? MGL S:2017 serves two main purposes: (1) to provide a rich descriptive picture of the experiences and lives of middle grades students and (2) to examine the associations between student outcomes and contextual factors during this time of transition from elementary to high school

. To achieve this broad set3

of goals , MGL SL :2017 requires a set of longitudinal and complementary instruments across several ty pes of respondents

. Collecting data over time and from multiple sources will provide information on the outcomes, experiences, and perspectives of students across grades 6, 7, and 8; their families and home lives; their teachers, classrooms, and instruction (with a focus in mathematics); and the school settings, programs , and services available to them. At each wave of data collection in the national study , students’ mathematics and reading achievement, socioemotional development, and executive function will be assessed. Students will also complete a survey that asks about their engagement in school, out

- of

- school experiences, peer group relationships , and identity dev elopment. Parents will be asked about their background, family

resources, and involvement with students ’ education and their schools. Students’ mathematics teachers will complete a two

-part survey
. In part 1, teachers will be asked about their background and classroom instruction . In part 2, teachers will be asked to report on the academic behaviors, mathematics performance, and classroom conduct of each study child in their classroom

. For students receiving special education services, their special educati on teacher or provider will also complete a survey

questionnaire similar in structure to the two

-part m athematics t eacher instrument, consisting of a t eacher

- level questionnaire and s tudent

- level questionnaire, but with questions specific to the special ed

ucation experiences and services of the study student

. School administrators will be asked to report on school programs and services, as well as school climate. The Common Core of Data (CCD) and Private School Survey

(PSS) will be used as source s for infor mation on school (e.g., school ty pe, grade configuration) and student

- body characteristics (e.g., percent minority enrollment and percent of students receiving free

- or reduced

-price lunch). S tudent information such as attendance, grade retention, and scores on state and national standardized assessments will be abstracted from school records. Lastly , field staff will complete an observation checklist on the physical condition and general upkeep of the school, the presence of different security measures, and handicap access. **Field Test Study Design .**  Prior to the spring 2017 national data collection, a MGL S:2017 field test will be conducted to : (1) evaluate the proposed assessments and instruments , and (2) evaluate the data collection procedures in a real world setting , identify potential challenges that could hinder the national effort, and test various procedures and strategies to inform plans for the national study . The field test will provide in formation about: (a) whether the selected assessment items and key non - assessment measures will allow us to answer the intended research questions, (b) whether the measures of key constructs are psychometricall y sound , and (c) whether the survey instrument s are appropriate for use across diverse samples of middle grade students (including students from different language groups and language proficiencies and those with different disabilities) , teachers, administrators, and parents. It will guide final item selection and revisions and help meet the goal of developing valid and reliable assessments to measure middle grade students’ mathematics and reading achievement and executive function skill s, as well as their socioemotional well

- being and other developmental and contextual factors . Finally , the field test will serve to evaluate procedures to be used to select, recruit , and gather data from schools, students, teachers, and parents , and determine if modifications to these procedures are needed to avoid any challenges identified during the field test when the national study is undertaken. The field test will be used to identify

effective procedures for use in the national study to identify , sample, and obtain reliable data on students with different disabiliti es. The MGL S:2017 field test will include students in grades 5 through 8 attending public or private schools in the contiguous United States. Itwill also include an oversample of students with three ty pes of disabilities (specific learning disability , em otional disturbance, and autism). We expect to conduct the field test in approximately 50 schools in five sites ( a site is comprised of a geographic area with one or more counties or county equivalents in the contiguous United States or the District of Columbia and its surrounding counties

. ) and to collect data from an estimated 4,075 students, including 250 students in the targeted categories of students with disabilities. The field test will also surv ey up to 450 mathematics teachers, 150 special education teachers, 819 parents, and 50 school administrators

. Field test data collection will include the same instruments and activities that are planned for the national study. Students will be administered a set of standardized assessments or item sets to measure their mathematics and4

reading achievement, as well as executive function, and will be asked to complete a student questionnaire 1 to gather data on their socioemotional well

- being, social behavior, as well as other school and outside

- of

school experiences. Their parents/guardians will be asked to complete a questionnaire that captures information about the students’ families, and their engagement in their child’s education. Mathematics and special ed

ucation teachers will be asked to complete a two

-part questionnaire: a teacher

- level questionnaire about the teacher’s backgrounds and experience, and a student

- level questionnaire about the teacher’s rating of the study student’s performance and skills

. S chool administrators will be asked to answer questions about the characteristics of their school’s population, staffing, programs , and academic support resources. S tudent transcripts will be collected to gather information on student outcomes, such as grades, academic honors, and disciplinary actions. Field staff will record their observation of the school facilities and grounds. Table 1 includes a summary of the field test instruments, with expected sample sizes, modes of administration, and estimated administration times. However, this request is only for recruiting districts and schools to participate in the field test. **Table 1. MGLS:2017 Field Test Instruments,**  **Target Sample Yield , Estimated Administration Time**  **and Mode**
**Instrument Target**  **Sample Yield**  **Target Disability**  **Yield Administration Time**  **Mode** Mathematics Assessments grades 6, 7, 8 a 4,075 250 30 minutes; 45 minutes for students with disabilities Group adm inistered with com puter Reading Assessments grades 6, 7,8 a Executive Function Measures 819 300 10 minutes Student Questionnaire (including SE measure) 25 minutes

Parent Interview 819 300 30 minutes We b, paper, telephone follow
- up Mathematics Teacher Questionnaire 450 150 23 minutes Mathematics Teacher Student Report (TSR) 819 300 10
- 13 minutes per student review Special Education Teacher Questionnaire 150 150 20 minutes Special Education Teacher Student Report 300 300 10 minutes per student review School Administrator Survey 50 50 33 minutes a The student sample for the mathematics and reading assessments will include 350 fifth graders to guard against floor effects

. 

### District **a nd	School**  **Recruiting** Recruiting schools for the MGL S:2017 field test will be a two

- step process. First, for public schools and some charter schools we will secure agreement from the district, and then proceed to recruit individual schools. For most private schools, recruitment will begin at the school level, although if the school operates under a diocese or other administrative unit or consortium , we will contact that administrative unit prior to contacting schools. 

### District Recruitment The field test wil l be conducted at 5 sites with each site containing at least one regular public school district and a private school serving grades 6

- 8

. 2 ( If there are multiple districts in the site that are large and diverse enough to satisfy

the minimum requirements of the field test, one district will be designated as the

1 Not every student will complete the full student questionnaire because of limits on the time that schools will allow us to wo rk with students, and given that the field test is evaluating a larger set of measures and items than will be included in the nati onal study. Students will be asked to complete different mathematics item sets, different measures of executive function and socioemotional development, and different sets of items from the student questionnaire. 2 Sites must also have at least 10 schools serving grades 6

- 8.5

“main” district to recruit for the study and the o thers w ill be used as replacements if the main district declines to participate. Upon receipt of OMB approval for th e recruitment activities , the superintendent in each of the selected public school districts (or the main district) in the 5 sites will be s ent a field test introductory packet that will include an introductory letter from ED (Appendix A), Frequentl y Asked Questions about the MGL S:2017 field test (Appendix C ), and a MGL S:2017 Field Test Summary (Appendix D )

. Senior project staff leading district recruitment will conduct a follow

- up call to each district to confirm receipt of the recruitment package, provide additional context for the study and data collection requirements, and to secure the district’s permission to contact schools

. During the call, the recruiter will answer any questions the superintendent or other district staff have about the field test, review the list of schools in the district serving grades 6

- 8 , and confirm key information about the schools (e.g., grades serv ed, size of enrollment). Information collected during this call will be used to confirm schools’ eligibility for participation in this study , and collect relevant information about school leaders that might be useful for recruitment. These recruitment call s will also be used to collect information about school enrollments of students with disabilities and to identify any barriers we may face in collecting student’s IEP information, which is needed for the field test oversample of this group of students. 3 NC ES has obtained a FERPA exemption that will allow us access to information on students ’ IEPs for sampling purposes. Recruiters will also determine if there are any additional requirements at the district level to conduct research in their schools (i.e., IRB review process) and whether the district has any restrictions on providing school staff with incentives to participate in the study

. Wherever possible, recruiters will approach the district with a goal of gaining both permission and support. They

will as k superintendents to let their principals know that a study representative will be contacting them and that the district has approved this contact , as well as ask for a letter of support from the district to be sent with the recruitment packet to schools

. This ty pe of district level support, where sincere and timely , greatly increases school support. Recruiters will make multiple attempts by telephone and email to reach the s uperintendent over a two
week period

. If we are unable to reach the superintendent after multiple attempts or if we make contact with the superintendent and the district refuses to participate and the site has more than a single school district , we will contact a second district and begin the district recruitment process again. If the second district refuses, we will attempt to convert the district whose refusal was soft er or where our contact remained amicable. Where this fails, we will break

- off contact with the superintendents and begin again the recruitment process using one of the two replacement sites

. Once district cooperation is obtained and the school sample is selected, we will complete any district

- level paperwork , including completing a district IRB application, and begin the school recruitment process once we have final approval from the district to do so

. The approach to recruiting private schools and some charter schools may or may not follow the same approach outlined for district recruitment. If the private schools sel ected for the field test operate under a governing body such as a diocese, private school administrator or a consortium of private schools, we will use the same approach that is used for public schools. Similarly , if a charter school operates under the dis trict, we will first contract the district for approval prior to contacting the school. However, many private schools and some charter schools will not have such a higher level governing body. In these cases, recruitment into the study will begin with scho ol recruitment as outlined below.

3 If we are able to get this information from sources other than districts, and can move forward with selecting the field test sample, we will share information about the schools selected in the districts during these initials calls.6 

### School Recruitment Once district approval (or the approval of another governing body in the case of private schools) has been acquired and the field test sample of schools chosen, a field test information packet will be sent to administrators of the field test schools. The packet will include a letter of introduction (Appendix B ), a copy of the Field Test Summary (Appendix D ), Frequently Asked Questions (Appendix C ), and a district (or another governing body) letter of approval and support (if the district or governing body is willing to prepare one ). The introductory letter will inform the school administrator that the school is being invited to participate in a field test for an important and unprecedented national study that will provide information about the middle grade ye ars and students’ progress during this time

. Itwill also provide an overview of what participation entails, and that it includes the participation of students in grades 6

- 8 (and in some schools , grade 5 students ), their mathematics teachers and parents. The letter will also inform the administrator of the study’s interest in assessing and collecting data from students with disabilities, their mathematics teachers (when applicable) and special education teacher or provider, and parents. The letter will mention that a DIR staff member will contact them to discuss their school’s participation in the field test and to answer any questions they may have. Within one week , a s chool recruiter will follow up with a call to request the school’s cooperation and to answer any questions about the field test and what participation in the field test entails

. Drawing on our experien ce, we plan to conduct our initial discussions with the school ’s administrator (e.g., principal, headmaster) who may designate another staff member as the school coordinator, who will serve as the study ’s primary contact for all future communication. If th e school administrator names a school coordinator, the recruiter will contact that person to introduce the study , and discuss the role of the school coordinator as facilitator. Moving forward , the school coordinator will work with the study team to schedul e data collection and field test activities at that school, including student recruitment, onsite assessments, and transcript abstraction. During our initial contact with the school, we will inform the administrator that a $500 incentive has been reserved for the school. Depending on school policy , schools may want to use part of this money to compensate the school coordinator for the key role this person will p l ay during the field test. During the initial school recruitment call, we will ask how many teachers provide math ematics instruction in the selected grades and confirm what type of parental consent procedures or requirements will need to be followed in order to collect data on students at the school, including for collecting information on students’ activities, grade

- level enrollment, and student

- teacher rosters

. Information will also be collected on any considerations that may impact data collection (e.g., planned construction periods, school reconfiguration, or planned changes in leadership) ; and ab out sources for missing IEP status to supplement information collected at the district level , as we identify students with disabilities and their special education teachers. We will also gather information about hours of operation, including early dismissal days, school closures/vacations, and dates for standardized testing. This information will be important to know when scheduling the student assessments. By design, the initial sample will include more schools in each site than we need to recruit. School s will be randoml y assigned to the initial sample (to be contacted first) and to the replacement sample (in a designated selection order). Replacement schools will be used if one of the initially sampled schools is determined to be ineligible (e.g., focal study grades no longer offered at the school) or refuses to participate. The plan for replacement will vary with the number of schools in each site; however, the plan will allow for replacement of initial sample schools with schools sharing similar characteristics (e.g., a private school would be replaced b y another private school and a rural school would be replaced with another rural school ). 

### Recruiter	Training Initial district contacts will be made b y senior staff that have experience recruiting district s for other education studies

. This staff can easily present the study

background, purpose, and data requirements to district personnel and district research review panels. Recruiters, who will follow

- up with schools and7

school coordinators, will also be w ell prepared and experienced in recruiting schools and educators for large

- scale studies. They

will be selected from a group of experienced research staff with past success recruiting schools for research studies , including studies sponsored by the Institute for Education Sciences (IES) . They

will participate in an all

- day training conducted by

the study design team structured to ensure that staff have a good understanding of the MGLS:2017 and the field test, what school participation entails, and that they are able to respond to questions raised during the recruitment process. The training will use a mix of lecture , group discussion, and exercises (e.g., responses to questions that school personnel may

ask about the study

and what participation involves). Among the topics to be covered are the purposes and design of the MGL S:2017 and the MGL S:2017 field test ; the field test samples of schools, students, parents , and mathematics and special education teachers and how they are chosen ; field test data collectio n instruments an d procedures ; overview of field staff’s visit to a sampled schools ; the burden participation places on schools and the steps that the study team will take to reduce this burden ; and common concerns raised by schools and how these will be addressed. 

### Monitoring To monitor the recruitment process, team members will update a spreadsheet on the MGL S:2017 SharePoint site. In formation about contact attempts (including details such as day , time, number called, and whether a message was left) and recruitment notes will help to track the status of each district and school. Disposition codes (no contact made, in process, agreed, refused) and detailed notes for all contacts will be used to monitor progress and determine whether a replacement district or school is needed , or if one of the five initial sites needs to be replaced with another one

. Recruiters will be trained to provide enough detail in the notes section for the recruitment leader to assess the likely outcome while the district and school is still in process. Enough detail will be provided to enable another recruiter or the recruitment leader to follow

- up. This information will be uploaded to Mathematica’s Survey Management System (SMS). The SMS will be used to track all field test recruitment and data collection activities. DIR’s recruitment leader will meet with the recruiters once a week by phone to review the status of each district and school and identify any questions or prob lems that have come up in the recruitment process. Group meetings will allow recruiters to share challenges and successful strategies. 

### A.2 **Purposes	and	Uses	of	Data** The purpose of the field test is to evaluate a battery of student assessments ( i.e., mathem atics and reading achievement and executive function skills ) and non - assessment survey instruments (e.g., student, mathematics and special education teacher, and parent questionnaire) for use in the MGL S:2017 national study

. The field test will also be use d to test procedures and operations that will be used in the national study . Data gathered during the field test recruitment process will be used to validate district and school characteristics used for sample selection, including school

- level information on the prevalence of students in specific disability

categories ( e.g. , emotional disturbance and autism). The field test will provide important information about the viability of sampling these students at the district level. It will also inform recruitment strategies for the national study

. The field test study team will revise recruitment materials and procedures for the national study based on the field test experience. The results of the recruitment effort will be provided in the field test report along with challenges and lessons learned. During district recruitment, districts will be asked to review key district and school

- level characteristics we have gathered for accuracy and completeness. While other sources will be us ed to develop current and complete data (including universe files and school and district websites), districts will have an opportunity to provide new information and correct outdated information that we collected from existent sources. For example, the re cruitment team will check for recent school mergers, changes in school leadership, temporary

but extended school closings for construction, and changes in grade configuration or size of enrollment. Once districts are recruited into the study , they will be asked to provide a list of students with IEPs on record , which will be used to oversampl e students with disabilities for the field test

. If districts8

are willing to provide contact information (e.g., email addresses and direct telephone numbers) for principals in sample schools, we will record this information and use it during school

- level recruitment. School s will be asked to provide guidance on the best way to gather accurate class roster information (for mathematics classes) and to identify special educ ation teachers or providers for sampled students with disabilities

. This information will be used to develop the student sample and procedures for linking students and their mathematics and special education teachers or providers

. 

### A.3 **Use	of	Technology	to Reduce	Burden**  Where feasible, available technology will be used to reduce burden and improve efficiency and accuracy

. For example, if districts can provide information linking students to their mathematics teachers or students with disabilities to their special education teachers electronicall y, we will use this information rather than asking for this information at the school level. The burden of district and school recruitment is minimal and most information is gathered over the telephone. Districts will primarily be asked to provide confirmation of data gathered from other sources , including school universe files and district and school websites. Our request to provide student lists will accommodate whatever form districts find to be the least burden some . The study will utilize the information in whatever format it is provided

. 

### A.4 **Efforts	to	Avoid	Duplication	of	Effort** The MGL S:2017 will not be duplicative of other studies. While NCES longitudinal studies have contributed to our understanding of the factors that influence student success and failure in school, the middle grades (grades 6

– 8) are noticeabl y absent from the studies conducted to date. A majority of national longitudinal studies have focused on high school students and on the transition from s econdary to postsecondary

education (for example, High School and Beyond [HS&B], Education Longitudinal Study of 2002 [EL S:2002]). The Earl y Childhood Longitudinal Study, Kindergarten Class of 1998

–99 (ECLS

K) and the National Education Longitudinal Study of 1988 (NELS:88) collected data on samples students attending grade 8, but neither included any data collection on grades 6 and 7. The ECL S

- K:2011 will not follow students beyond grade 5 and the High School Longitudinal Study of 2009 (HSL S:09) began with a national sample of students in grade 9. Thus, we have little information at the national level about the learning that occurs during grades 6

- 8 and about the rates of learning for different groups of students who may experience diverse school environment s and opportunities. The MGL S:2017 is unique in that it will assess student’s mathematics and reading achievement, as well as other student outcomes (for example, executive functions and dimensions of socioemotional development) for the same group of students over a three

- ye ar period. In addition to the ECL S

- K and NEL S:88, other national studies have assessed some of these outcomes for students in grade 8, including the National Assessment of Education Progress (NAEP) and Trends in International Mathematics and Science Study

(TIMSS). These studies however are cross - sectional and do not include repeated measures of achievement, and do not assess multiple subjects and areas of development for the same sample of students. Therefore, they

cannot answe r questions about students’ growth in mathematics and reading over the middle grade ye ars, differences in the rates of growth for different populations (for example, differences b y gender, race

- ethnicity , and for students attending public and private schoo ls) , and the school and non

- school factors that may facilitate or hinder this growth. Nor can they explore questions about the relationships between student achievement and other school outcomes and executive functions (for example, working memory , attenti on, and inhibitory control), that work to regulate and orchestrate cognition, emotion, and behavior to enable a student to learn in the classroom. Other adolescent development studies have been conducted, but these often do not include a grade 6 sample. F or example, the youngest children in the National Longitudinal Study of Adolescent Health (Add Health) and the Maryland Adolescent Development in Context Study (MADICS) were in grade 7 at baseline. Many of these studies collected data on local samples, had a primary focus on family

and child processes, and were started in the 1990s (for example, MADI CS and the Michigan Study of Adolescent9

and Adult Life Transitions [MSALT]). As such, they do not provide a contemporary picture of grade 6

- 8 students in the U. S. Within the MGL S:2017, to the extent possible, we will use existing data to develop the school and student sample s rather than duplicate data collection and reporting efforts. However, while many school characteristics will be gathered from existing sources such as the CCD and PSS , districts have a unique ability to provide updates based upon more current information reflecting recent school and student enrollment information not available elsewhere. The most recent CCD and PSS data that will be available at the time the field test sample is drawn (summer 2014) are for school ye ars 201 1

- 201 2 and 20 09

- 201 0 , respectivel y. Districts can provide information on school

- level enrollments of students with disabilities that is either not available or would require significant resources to collect. There is no universal grade 6

- 8 student sampling frame. We will develop the frame from information provided by districts and schools. 

### A.5 **Methods	of	Minimizing	Burden	on	Small	Entities** During district and school recruitment, we will minimize burden by training recruitment staff to make their contacts as straightforward and concise as possible. The recruitment letters and materials (e.g., study description and FAQs) are designed to be clear, brief, and informative. We will include all relevant staff at any district

- and school

- level meetings so that the district superintendents and principals will not be required to convey

the information individually to their staff members. At the district level (or at a private school organization or administrative office) , we will attempt to arrange a conference call that includes representatives of the superintendent’s or director’s office and research approval office; and officials who can discuss the availability of student records. For the school

- level recruitment, we will offer to organize a conference call or webinar for administrators and teachers who might be included in the study

. To avoid imposing an additional time commitment and travel requirements on students participating in the field test student assessment, we will attempt to conduct all of the field test data collections at the school during the regular school day

. The approaches that will be used to minimize burden for parents, teachers, students, and school administrators will be described in the second field test OMB package

– Fiel d Test Data Collection Package. 

### A.6 **Consequences	of	Not	Collecting	Data** A field test to ensure that we have reliable instruments and effective procedures for the national study will greatl y impro ve the quality of and enhance efficiencies in the MGL S: 2017

. The recruitment effort is critical , as it will offer the best opportunity to get first

- hand , up - to
- date information on schools to include in our field test, such as changes in school grade levels or closures and identifying students with IEPs and schools with different concentrations of these students . Collecting this information is necessary to determine school eligibility and finalize the schools that will be included in the field test sample. 

### A.7 **Special	Circumstances** There are no special circumstances involved with the recruitment and data collection. 

### A.8 **Consultations	outside	the	Agency** In addition to the DIR, Mathematica, and ETS study team members, several consultants have provided expert advice during the instrument development process. **William Schmidt** (Michigan State University) has assisted with the development of the student mathematics assessment and recommended measures and survey items for the teacher

- level and student

- level Mathemati cs Teacher Questionnaire. **Jacquelynne Eccles**  (University

of Michigan) has assisted with the selection of the executive function assessments, measures of socioemotional development, and identified content and recommended survey items for the student and parent questionnaires. **Donald Rock** (independent consultant, previously at ETS) has assisted with the development of the student math assessment and provided expert advice on structure of the mathematics assessment and the design of the field test sample. **Kath y Terry**  (Houston Independent School District) has assisted with the parent and school administrator questionnaires . She is also assisting10

with the approaches used to secure cooperation from districts and schools for the field test. **Martin Frankel**  ( Baruch C ollege ) has assisted with the field test sample design. The MGL S:2017 design team also convened a series of Content Review Panels (CRPs) to gather input on key constructs to measure in the MGL S:2017 and identify possible item sources. The MGL S:2017 Mathematics Assessment CRP first met in Washington DC on June 18

- 19, 2013 , to provide input during the assessment development process and ensure that the math assessment resulted in reliable and valid data on students ’ mathematics knowledge and skills. Specifically , the Math ematics CRP helped to guide decisions about which domains of mathematics are most critical to sample for the assessment. The panel also provided input on the amount of assessment time that should be devoted to different domains of mathematics and offered advice f or the best ways to assess these domains gi ven the time constraints of the MGL S:2017 assessment. Following the meeting, members of the Math ematics CRP were asked to review a revised version of the assessment framework and test blueprint and provide a short written reaction to the domains and their definitions, the learning progressions, the allocation of items within domains , and depth of knowledge within categories in the framework and blueprint. The panel members participated in a 3

- hour webinar on September 13, 2013 , to discuss these topics. The MGL S:2017 Executive Function CRP met via webinar on July 18, 2013 , to provide input during the assessment development process and help guide decisions about which constructs are most critical to examine. The pane l provided input on potential measures and scales for assessing middle grades students’ executive functioning (for example, inhibitory control, cognitive flexibility, and working memory). The MGL S:2017 Socioemotional

- School

- Family CRP met in person for a 2

- day meeting in Washington DC on July

25

- 26, 2013 , to provide input during the instrument development process and ensure that the assessment and questionnaires will result in reliable and valid data on students’ socioemotional well

being , school experience s , and home life. The panel provided input on potential items and scales as well as the best reporter for various aspects of students’ socioemotional well

- being (e.g., perceived competence, internalizing behavior, conscientiousness, and engagement). T he panel also provided input on the student and parent questionnaires that the MGL S:2017 will use to collect data on student s ’ family , school, and out

- of

- school experiences. Finally , the MGL S:2017 School Administrator CRP met via webinar on August 16, 2013 , to provide input on constructs to measure school

- level characteristics. The panel provided input on potential items and scales as well as the best reporter for various constructs. Following the CRP meetings, a revised construct matrix was sent out to panelis ts for comment. They were asked to provide written feedback on constructs they believed are high priority and should be included in the field test questionnaires , in which waves ( i.e. in grade 6, 7, and/or 8) the construct should be measured , and as report ed by which ty pe of respondent. The instruments being evaluated in the cognitive laboratory

interviews reflect recommendation s made by the various CRPs. On October 31, 2013, NCES convened a panel of experts to discuss a range of issues associated with sampling, assessing, and collecting data about students with different disabilities. Another expert panel will be convened in spring 2014 to discuss these issues further with more attention on the content of the Special Education Teacher Questionnaire and student assessment accommodations

. Additionally, a panel of reading expert will be convened in spring 2014 to review the plans for assessing students’ reading achievement. Input from this panel will be used to finalize the reading assessment field test instrument. The recommendations of th e s e group s will be reflected in the second OMB package

– Field Test Data Collection Package. See Table 2 for the list of the members on each CRP that has been formed and met to date. The membership in the follow

- up Disabilities CRP and the Reading CRP has not yet been finalized.11

**Table 2. Members of the MGLS:2017 Content Review Panels**  **Name Affiliation**  **Expertise** Mathematics Assessment Content Review Panel Tom Loveless Brookings Institute Policy, Math curriculum Linda W ilson Formerly with Project 2061 Math education, math assessment, and middle school assessment, author for Assessment Standards for School Mathematics (NCTM) and

NAEP math framework, teacher Kathleen Heid University of Florida Math education, use of technology, teacher knowledge, member of the NAEP Grade 8 mathematics standing committee Edward Nolan Montgomery County Schools Math curriculum and standards, large
- scale assessment of middle grade students Lisa Keller UM ass Amherst Psychometrics, former math teacher Paul Sally University of Chicago Math education, mathematics reasoning, mathematically talented adolescents Margie Hill University of Kansas Co
- author of Kansas math standards, former NAEP Math Standing Committee member , former district math supervisor Executive Function Content Review Panel Lisa Jacobson Johns Hopkins University; Kennedy Krieger Institute Executive functioning; attention; neurodevelopmental disorders; parent and teacher scaffolding development of executive functioning skills Dan Romer University of Pennsylvania Adolescent risk taking James Byrnes Temple University Self
- regulation; decision making; cognitive processes in mathematics learning Socioemotional
- School
- Family Content Review Panel James Byrnes Temple University Self
- regulation; decision making; cognitive processes in mathematics learning Russell Rumberger University of California, Santa Barbara School dropouts and ethnic and language minority student achievement Tama Leventhal Tufts University Family context; adolescence; social policy; communities and neighborhood indicators Susan Dauber Bluestocking Research School organization; educational transitions; urban education; parent involvement and family processes Scott Gest Penn sylvania State University Social networking, social skills, and longitudinal assessment of at
- risk populations. Kathryn W entzel University of Maryland Social and academic motivation; self
- regulation; school adjustment; peer relationships; teacher
- student relationships; family
- school linkages Richard Lerner Tufts University Adolescent development and relationships with peers, families, schools, and communities School Administrator Content Review Panel Susan Dauber Bluestocking Research School organization; educational transitions; urban education; parent involvement and family processes George Farkas University of California, Irvine Schooling equity and human resources Jeremy Finn State University of New York at Buffalo School organization, school dropouts Edward Nolan Montgomery County Schools Large urban school system administrator Tom Loveless Brookings Institute Policy, Math curriculum Disabilities Considerations Content Review Panel Lynn Fuchs Vanderbilt University Student assessment, mathematics curriculum, psychometrics models, learning disabilities Margo Mastropieri George Mason University Learning disabilities, retention strategies, cognitive strategies, special education, research methods Cara Laitusis ETS Research and Development Curriculum
- based assessment and the diagnosis and treatment of students with learning disabilities. Leslie Scott University of Michigan Quality/cost factors related to design of multimode school studies; web
based data collection methodol ogies; use of multimedia approaches 

### A.9 **Payment	or	Gift	to	Respondents** High levels of school participation are critical to the success of the MGLS:2017 field test. Schoo l, mathematics teacher, special education teacher, parent, and student data collection activities are contingent o n school cooperation. NCES recognizes that the burden level of the study is one of the factors that school administrators will consider when agreeing to participate. To offset the perceived burden,12

NCES intends to continue its use of strategies that have worked successfully in the past for other major NCES studies ( e.g. , ECLS

- K , ECL S

- K:2011, HS&B, NELS:88, and ELS:2002). Offering schools a financial incentive helps to offset the burden schools are asked to take on as a result of their participation. The study asks a lot of schools , including requesting a student roster with basic demographic information ( e.g. , date of birth, gender, and race/ ethnicity) ; information on student s’ IEP status; permission for field staff to be in the school for up to a week ; administering an extensive battery of assessments to 84 students in grades 6

- 8 ; making available space for administering student assessments; allowing students to leave their normal classes for the duration of assessments; and providing information about the student’s teachers and parents. A lso, on average, approximately 10 students in each school will have one of the disabilities that are of special interest to the study and many will require accommodations and different assessment settings , such as individual administration and smaller group sessions. Working with the data collection contractor to assess these students will place further burden on the participating schoo ls. Given the many demands and outside pressures that schools already face, it is essential that they see that MGL S:2017 staff understand the additional burden being plac ed on the school staff when requesting their participation. We propose to offer school s a $200 incentive for their participation. This amount is consistent with the amount of the school incentive offered b y other NCES studies such as the ECL S

- K, ECL S

- K:2011, TIMSS , and the Program of International Assessment (PI SA). MGL S:2017 will also ask participating schools for assistance in administering the student assessments by allowing access to school computer resources and equipment, and b y helping to establish internet connections to the web

- based instruments. Therefore, like the HSL S:09, MGL S:20 17 will offer a $50 incentive to the school’s information technology (IT) coordinator. School coordinators play an especially important role in the study and are critical to its success. The coordinator in each participating school will coordinate logistic s with the data collection contractor; compile and supply

a list of eligible students for sampling to the data collection contractor; communicate with teachers, students, and parents about the study to encourage participation; distribute and collect parent consent forms; and assist the test administrator in ensuring that the sampled students attend the testing sessions. For this reason, coordinators will be offered $150 as a baseline incentive plus an additional $50 incentive if the student participation rate for the school meets or exceeds 85 %

. In comparison, the PI SA 2015 field test offered coordinators a $ 2 00 incentive , while HSL S:09 offered $100

- $150 incentive conditional on the percentage of participating students

. The MGL S:2017 will sample and assess 84 students per school compared to the 42 students in PI SA 2015 field test and 36 students per school in HSL S:09. The MGL S:2017 will also involve the additional effort associated with assess ing an oversample of students with disabilities. The supplemental conditional $50 incentive is meant to encourage school coordinators to make the extra effort needed to overcome the fact that obtaining parent consent for middle grade students in general and for students with disabilities in particular is often challengin g . The actions of the school coordinator can be central to minimizing these challenges. All incentives will be paid by check. An honorarium check will be mailed to each school, school IT coordinator , and school coordinator at the end of spring data collection along with a note thanking each for their assistance. 

### A.10 **Confidentiality	of	the	Data** DIR and its research partners, Mathematica Policy Research and Educational Testing Service, will conduct all data collection, anal ys is and reporting activities in accordance with all relevant regulations and requirements, consistent with the Education Sciences Reform Act of 2002 ( Title I , Part E, Section 183 ) , the Family

Educational and Rights and Privacy Act (FERPA) (20 U. S. C. 1232g;34 CFR Part 99), and the Protection of Pupil Rights Amendment (PPRA) (20 U.S. C. 1232h ; 34 CFR Part 98)

. The research team will protect the confidentiality of all data collected for the study and will use it for research purposes only

. Only DIR and Mathematica Policy Research will be required to handle13

personall y- identifying information (PII) and their project directors will ensure that all PII about respondents will be protected . All data will be kept in secured locations and identifiers will be destroyed as soon as they

are no longer required. All members of the study team having access to the data will be trained and certified on the importance of confidentiality an d data security. When reporting the results, data will be presented only in aggregate form, such that individuals and institutions will not be identified. In cluded in all requests for data will be the following statement: NCES is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002), 20 U.S. Code, § 9543. By law, the data provided by your schools, staff, and students may only be used for statistical purposes and may not be disclosed, or used, in identifiable form for a ny other purpose except as required by law (20 U.S. Code, § 9573). The following safeguards will be used to carry out confidentiality

assurances for the study :
 All em

ployees at DIR and their partners sign confidentiality agreement s that em phasize the impo rtance of confidentiality and specify employees’ obligations to maintain it (see Appendix F ).

 Personally identifiable information (PII) is maintained on separate forms and files, which are linked only by sample identification numbers.

 Access to the file linking sample identification numbers with the respondents’ ID and contact information is limited to a small number of individuals who have a need to know this information

 Access to hard copy documents is strictly limited. Documents are stored in locked file s and cabinets. Discarded materials are shredded.

 Access to computer data files is protected by secure usernames and passwords, which are only available to approved users. Access to identifying information for sample members is limited to those who have direct responsibility for providing and maintaining sample contact information. At the conclusion of the study, these data are destroyed.

 Sensitive data are encrypted and stored on removable storage devices that are kept physically secure when not in use.

 T he plan for maintaining confidentiality includes staff training regarding the meaning of confidentiality, particularly as it relates to handling requests for information and providing assurance to respondents about the protection of their responses. Itals o includes built
- in safeguards concerning status monitoring and receipt control systems. Responses will only

be made available in tabular form. Under no condition will information be made available to school personnel. District and school staff responsible for assisting DIR in the data collection will be fully

informed of the study ’s confidentiality policies and procedures for ensuring protection and confidentiality

of individual sample members and responses. The study will also adhere to state and local laws and school board policies as appropriate. 

### A.11 **Additional	Justification	for	Sensitive	Questions** The recruitment effort is not gathering information considered to be of a sensitive nature. A request for a list of middle grade students with IE P s will be requested from school districts under FERPA exemption (February 7, 2014 )

. This information will be considered confidential and used for sampling purposes only

. All district and school personnel facilitating the conducting of the study and develop ing the sampling frame will be informed of the privacy and confidentiality

protocols required for the study

. This will include the sample lists of schools and students. 

### A.12 **Estimates	of	Hour	Burden** Table 3 shows the expected burden during the field test recruitment activities. We anticipate contacting up to 15 districts/diocese and 100 school administrators to recruit the final 50 schools. The expected burden for school districts/diocese is 30 minutes for those that decline and 5 hours for districts/dioces e that agree14

to participate for a total of 35 hours of burden for school districts/diocese. The estimates for the recruitment phase of the field test for school administrators includes an estimated 30 minutes for up to 100 schools that decline or are deter mined to be ineligible, and 1 hour for the 50 schools that agree to participate. This is a total of 100 hours of burden for schools. The number of district and school administrators who will need to be contacted about their participation and their recruitm ent time is based on estimates from similar school based sample studies (e.g., ECL S

- K:2011). The total number of burden hours for the field test district and school recruitment is estimated at 140 hours. **Table 3. Respondent Burden Estimates for MGLS:2017 F ield Test Recruitment Activities** 

**Recruitment Number of**  **Respondents Number of** 

**Responses*** **Average burden**

**tim e (minutes)**  **Mode Total Burden**  **(Hrs)** Nonparticipating districts (diocese) 10 10 30 Mail, e
- mail, telephone 5 Participating districts** (diocese) 7 7 300 35 Nonparticipating schools 100 100 30 50 Participating schools 50 50 60 50 Study Total*** 167 167 N/A 140 * Responses include both refusals and acceptances to participate in the field test. ** For the purposes of this burden table, we have assumed one site will include 2 districts and that one site will include a district and a diocese for a total of 7 participating districts/diocese across 5 sites. We have also assumed that the 7 participating distri cts or dioceses will require a district l evel review of the proposed data collection activities by their IRB, research department, or school board. It is assumed that discussions around these requirements, review of the research application submission, and preparation of the approval notific atio n, will total approximately 4 hours per district with an additional hour spent during the initial contact determining interest and reviewing eligibility upon acceptance. ** * N/A

– Not Applicable

. 

### A.13 **Estimate	of	Total	Annual	Cost	Burden	to	Respondents	or	R ecordkeepers**  There are no additional respondent costs associated with this data collection other than the time estimated as burden. 

### A.14 **Annualized	Cost	to	the	Federal	Government** The estimated cost to the government to recruit districts and schools for the field test is $ 253,094

. 

### A.1 **5 Tabulation,	Publication	Plans,	and	Time	Schedules**  **a. Tabulation	Plans**  There are no plans to tabulate information obtained from district or school personnel during school recruitment. In formation about school changes or students with IEPs will be used internall y for sampling purposes and to inform field test plans. The usefulness of the information requested, its accuracy , and reporting source will be documented and findings shared in the field test report. 

### b. **Publication	Plans** The field test is expected to be completed in approximately two years. School recruitment will begin in fall 2014 with data collection planned for spring 2015. Final instruments and field test reports will be pro vided in 2016 for the 2017 national study

. When field test anal ys es are complete, the study team will include lessons learned during district and school recruitment in the field test report

. Findings will include discussion of the information districts were able to update regarding school changes and the percentage of student enrollment with IEPs. We will report whether the information was provided b y districts or schools, the timeliness with which it was obtained , and school or district burden associated with the reporting of this information

. This information will be shared with and used by research staff developing procedures for the national study

.15 

### c. **Schedule** Table 5 shows the schedule for the field test

. **Table 4**  **. Field Test Study Timeline of Activi** **ties** Activity Time Period Instrument item pool development Spring 2013

- Summer 2014 Conduct cognitive testing for instruments* Spring 2014

- Summer 2014 Identify sites and select school sample Summer

- Fall 2014 Recruit districts (and diocese) and schools Fall 2014 Collect student rosters & Sample students (and their parents and teachers) Winter 2014

– Spring 2015 Collect parent consent for student testing Winter 2014

– Spring 2015 Data Collection Spring 2015 Draft Field Test Report Spring 2016 Final report and instruments for the 2017 national survey Summer 2016 *Request for Clearance for MGLS:2017 Cognitive Laboratory Activities was submitted in March , 2014

. 

### A.1 **6 Reasons	for	Program	Changes	or	Adjustments**  This is a new project. 

### A.1 **7 Exception	to	the	Certification	Statement**  No exceptions to the certification statement are requested or required. 

### REFERENCES Eccles, J.S, Midgley, C., Wigfield, A., Buchanan, C.M., Rueman, D., Flanagan, C., & Iver, D. (1993). Development during adolescence : The impact of stage

- environment fit on young adolescents’ experiences in schools and in families. American Psychologist, 48 (2), 90
– 101. Gibbons, R.D. & Hedeker, D. (1992). Full

- information item bi

- factor analysis. Psychometrika , 57, 423
- 436. Holzinger, K. J., & Swineford, F. (1937). The bi

- factor method. Psychometrika, 2 , 41
– 54. Wigfield, A., Eccles, J. S., Schiefele, U., Roeser, R., & Davis

- Kean, P. (2006). Motivation. In N. Eisenberg (Ed.), Handbook of Child Psychology (Vol. 3, 6th ed., pp. 933
– 1002). New York, NY: Wiley. Wigfield, A., & Eccles, J.S. (1989). Test anxiety in elementary and secondary school students. Educational Psychologist, 24, 159
– 183.