# NATIONAL CENTER FOR

**EDUCATION STATISTICS (NCES) NATIONAL ASSESSMENT OF**  **EDUCATIONAL PROGRESS WAVE**  **1 SUBMITTAL FOR**  **2014**

**VOLUME I**  **SUPPORTING STATEMENT OMB# 1850** 

– 

### 0790 **v.37**

**Grade 4** Student Core, Science Survey Questions Teacher Questionnaire s School Questionnaire s **Grade 8** Student Core, Civics , Geography, U.S. History , Science Survey Questions Teacher Questionnaire s School Questionnaire s **Grade 12** Student Core, Science Survey Questions School Questionnaire s

May 16 , 201 3201 4 Wave 1 Supporting Statement ii

Contents 1. Explanation for This Submittal ................................ ................................ .............................. 3 Table 1: List of Components in Wave 1 ................................ ................................ ................................ ............ 3 2. Overview of Wave 1 NAEP 2014 Assessments ................................ ................................ .... 3 3. Sampling Information ................................ ................................ ................................ ............ 4 4. Information Pertaining to the 2014 Questionnaires in This Submittal ................................ .. 4 Table 2: School Background Questionnaires for the NAEP 2014 Assessments ................................ ................ 5 5. 2014 Wave 1 Burden Information ................................ ................................ ......................... 6 Table 3: Total Burden for Wave 1 ................................ ................................ ................................ ..................... 6 Table 4: Wave 1 Questionnaires - Per Respondent Burden Time ................................ ................................ ...... 7 Table 5: Estimated Burden for NAEP 2014 Questionnaires Contained in This Submittal (Wave 1) ....... **Error! Bookm**  **ark not defined.** 6. How, by

Whom, and for What Purpose the Data Will Be Used ................................ ............ 8 7. Estimates of Costs to the Federal Government ................................ ................................ ...... 8 Table 6: Administration Cost Estimates (Wave 1 subjects) ................................ ................................ ............... 8 Appendix A: Committee Lists ................................ ................................ ................................ ... 9 Appendix B: Parent/Guardian Notification Letter ................................ ................................ ... 12 Appendix C: Science Cognitive Interview Reports, List of New Questions **Error! Bookmark not defined.** Appendix D: 2014 Sampling Design Memo ............................ **Error! Bookmark not defined.**201 4 Wave 1 Supporting Statement : Ap pendix A 3 **1. Explanation**  **for This Submittal** This document contains supplemental information pertaining to the 2014 - 2016 National Assessment of Educational Progress (NAEP) s ys tem clearance proposal. NAEP is administered by the National Center for Education Statistics (NCES) in the I nstitute for Education Sciences of the U.S. Department of Education. The National Assessment Governing Board (henceforth referred to as the Governing Bo ard) sets policy for NAEP and determines the content framework for each assessment. The system clearance package was submitted in November 2012 and approved in March 201 3 (OMB #1850 - 0790 v. 36 ). The terms of clearance for OMB approvals state that NCES will publish a notice in the Federal Register allowing a 30 - day public comment period on the details of each collection concurrent with submission to OMB. The 2014 submittal is divided into two waves to meet scheduling and question development requirements. Th e first wave contains the core , 1 social studies ( c ivics, g eography, U.S. history), and science (paper - and pencil) descriptions, burden , and questionnaires . W ave 2 will contain technology and engineering literacy (TEL) and science ICT (interactive computer tasks) descriptions, burden, and materials, as well as information regarding school coordinator activities, including the collection of information on students with disabilities (SD) and English language learners (ELL). This Wave 1 submittal contains burden information and survey questions (referred to as background or noncognitive questions) for the components of the NAEP 201 4 assessments listed in Table 1 below. **Table 1: List of Components in Wave**  **1 Component**  **Part in Vo lume II**  **Grade 4** Student

— Core , Science Part 1a Teacher

— Background, Education, Training (BET) - Science , Classroom Organization and Instruction (COI) - Science Part 1b School

— School Characteristics and Policies (SCP), Science, Charter School Part 1c **Grade 8** Student

— Core, Civics, Geography, U.S. History, Science Part 2a Teacher

— BET - Social Studies (Civics, Geography, U.S. History) , C OI - Social Studies (Civics, Geography, U.S. History), Teacher

— BET - Science, COI - Science Part 2b School

— SCP, Social Studies (Civics, Geography, U.S. History), Science, Charter School Part 2c **Grade 12** Student

— Core, Science Part 3a School

— SCP, Science, Charter School Part 3b **2. Overview of Wave 1**  **NAEP 201**  **4 Assessments**  Much of the following broad overview of the 2014 NAEP assessments was included as part of the 201 4

– 201 6 system clearance submittal. The Governing Board determines NAEP policy and assessment schedule, and future Governing Board decisions may result in changes to some aspects of an assessment (e.g., which subjects are assessed in which years). However, the overall methodology and assessment process remain s

1 NCES collects student question data , referred to as core questions , that are required by law ( 20 U.S.C. § 9622 ; i.e., race, ethnicity, and socioeconomic status) and provide a context for reporting student performance .201 4 Wave 1 Supporting Statement : Ap pendix A 4 consistent. The national NAEP assessments contain two kinds of questions: cognitive ( assessment question s that measure student knowledge of an academic subject) and survey (background questions that gather information on demographic as well as classroom instructional procedures ) . The Governing Board is responsible for selecting and approving all of NAEP’s su rvey questions, as well as the cognitive items. The NAEP assessments are currentl y supported by an alliance of organizations under contract with NCES, within the U.S. Department of Education. The main NAEP assessment reports current achievement levels and trends in student achievement at grades 4, 8, and 12 for the nation and , for certain assessments, states and select urban districts. These assessments follow subject - area frameworks devel oped b y the Governing Board and use the latest advances in assessment methodology . The subject area frameworks evolve to match instructional practices. **3. Sampling Information** The 2014 Sampling Design Memo 2 (see appendix D ) provides specific sampling info rmation regarding the 2014 NAEP assessments. It includes information on:

• Assessment ty

pes • Primary

Sampling Units Selection • Stratification and Oversampling • Overlap Control of Samples • Substitute Samples • Student Sampling Information • Weighting Requirements **4 . Information Pertaining to**  **t he**  **2014 Questionnaires in This Submittal**  **Student Question**  **naires** Core

— the first survey question block for all students contains core questions , many of which are related to demographic information. Civics

— the grade 8 operational civics subject - specific questions are existing trend questions. These questions were last administered as part of the 2010 civics assessment. Geography — the grade 8 operational geography subject - specific questions are existing trend questions. These questions were last administered as part of the 2010 geography assessment. U.S. History

— the grade 8 operational U.S. history subject - specific questions are existing trend questions. These questions were last administered as part of the 2010 U.S. history assessment. Science

— the grades 4, 8, and 12 pilot science subject - specific questions consist of trend and newl y developed questions. The new questions were pre - tested in cognitive interviews ( also referred to as cog

2 The 2014 Sampling Memo (dated 3 - 25 - 2013) contains information related to the grades 4 and 12 social studies assessments, which were subsequently dropped.201 4 Wave 1 Supporting Statement : Ap pendix A 5 labs) in 2012. Refer to appendix C for the reports associated with the science cog labs and for a list of the new questions contained in the science pilot questionnaires (student, teacher, and school) . Membership of the c ommittees that helped define the cognitive and survey questions pertaining to the 2014 subjects in Wave 1 is provided in this document, in appendix A . All of the assessments covered by Wave 1 will be administered b y paper- and -pencil. **Teacher Questionnaires** Teacher questionnaires are comprised of multiple sections , which include: Background, Education, and Training (BET; also referred to as the teacher core)

— the operational BET questions are trend or previously piloted questions . Science pilot consists of trend and pilot questions. Classroom Organization and Instruction (COI)

— the operational COI questions in the social studies subjects are trend questions . The COI section for the science pilot questionnaires contains both existing trend and new questions. All t eacher questionnaires will be completed online in 2014. If a grade 8 teacher teaches both social studies and science, the online questionnaire s ys tem will administer the appropriate BET questions and both COIsections to that teacher. **School Questionnaires** School questionnaires are comprised of multip le sections, which include: School Characteristics and Policies (SCP; also referred to as the school core)

– One version of SCP questions will be given at each grade . The SCP questions are comprised of trend or previousl y piloted questions . Social Studies (Civics, Geography, and U.S. History)

– The operational school questions in the social studies subjects are trend questions that were administered in the 2010 assessments. Science

– The pilot science questionnaires contain both existing trend and new questions. Charter School

– Charter school questions are comprised of trend questions that were administered in previous assessments. Note that the charter school section is only completed if the school is a charter school. All 2014 s chool questionnaires will be completed online . Table 2 depicts the composition of the 2014 school questionnaires. **Table 2**  **: School Background Questionnaires for the NAEP 2014 Assessments** Grade Section 1 Section 2 Section 3 Section 4 **4** SCP Science Pilot Charter School **8** SCP Social Studies Science Pilot Charter School **12** SCP Science Pilot Charter School Note: S CP

– School Characteristics and Policies201 4 Wave 1 Supporting Statement : Ap pendix A 6 **5 .**  **Burden Information for 2014 Wave 1**  The nature of NAEP is that samples of students alternate from a relativel y low volume in national - level only administration years to a substantial volume increase in state - level administration years that include one or more assessments that support the reporting of results by state and certain urban districts. In 2014 , NAEP will conduct national - level onl y assessments, so the estimated burden is much lower than in 201 3 NAEP. The total combined burden for the 2014 Wave 1 materials is provided in table 3. **Table 3**  **: Total B**  **urden for Wave 1**  **Respondent Cate gory**  **Number of Respondents Total Burden Hours**  Student s 30 ,500 12,708 Teachers 2,894 988 School s 533 267 **Total 33,**  **927 13,**  **963** Note: School coordinator activities, such as e - filing and pre - assessment activities, along with their related burden, will be included in the Wave 2 submittal. The burden include s the following categories: Students

– Students in eighth grade complete civics, geography, or U.S. history assessment booklets that contain two 25 - minute cognitive blocks , and two survey question sections that require a total of 15 minutes to complete. The social studies subjects are all paper -based assessments . The science pilot s associated with Wave 1 will be administered in paper - and -pencil and hands - on tasks (HOTs) formats. The paper- and -pencil version contain s two 25 - minute cognitive blocks . The science hands - on tasks have students work with lab materials and other equipment to perform experiments. These hands - on tasks may

require a total of an additional 15 minutes of student time for the cognitive portion of the assessment. The paper- and -pencil and HOTs versions will also include survey questionnaire sections . Additional student burden is included for time to read directions and distribute test booklets . F or paperand -pencil and HOTs assessments, this additional burden is estimated at 10 minutes , thus the total burden for students is 25 minutes 3 for the paper- and -pencil and HOTs assessments. Appendix B includes a sample parental notification letter regarding NAEP. The letter will be ad a pted for each grade/subject combination and the school principal may edit it. However, the information regarding confidentiality

and the appropriate law reference will remain unchanged . Teachers

– In 2014, teachers will be asked to complete questions about their teaching background, education, training, and classroom organization and instruction. At fourth - grade, onl y science teachers will complete teacher questionnaires and this burden is estimated at 20 minutes . The teachers of eighth grade students participating in the NAEP civics, geography, U.S. history , or science assessments will complete questionnaires. The eighth - grade teacher burden is estimated to be 20 minutes for most teachers,

3 The 25 - mi nute burd en calculation is comprised of 15 minutes for survey questions and 10 minutes for set up and directions. The time for the cognitive sections is not part of the burden calculation.201 4 Wave 1 Supporting Statement : Ap pendix A 7 and 30 minutes for a small percent (5%) who may teach both social studies and science . Note there is no teacher questionnaire for the teachers of students taking the twelfth - grade assessments . Schools

– The school administrators in the sampled schools will be asked to complete a school questionnaire. The school questionnaire burden is estimated to be 3 0 minutes at all three grades . **Table 4**  **: Wave 1**  **Question naire**  **s -**  **Per Respondent Burden Time Respondent Category**  **Time per Respondent Grade 4**  Student Core + Science (paper and pencil, including HOTs) 15 mi nutes Student Directions and Set - up 10 minutes Teacher Questionnaire 20 mi nutes School Questionnaire 30 minutes **Grade 8** Student Core + Civics 15 mi nutes Student Core + Geography 15 mi nutes Student Core + U.S. History 15 mi nutes Student Core + Science (paper and pencil, including HOTs) 15 mi nutes Student Directions and Set - up 10 minutes Teacher Questionnaire (single subject) 20 minutes Teacher Questionnaire (social studies and science) 30 minutes School Questionnaire 30 minutes **Grade 12** Student Core + Science (paper and pencil, including HOTs) 15 mi nutes Student Directions and Set - up 10 minutes School Questionnaire 30 minutes **Table 5**  **: Estimated Burden for NAEP 2**  **01 4**  **Questionnaire s Contained in This Submittal (Wave 1)**  **.**

**Subjects # of** 

**Students Avg. time** 

**per respondent**  **[m inutes)**  **Student**

**Burden (hours)**  **# of**

**Teachers Avg. time** 

**per respondent**  **[m inutes)**  **Teacher**

**Burden (hours)**  **# of**

**Schools Avg. time** 

**per respondent**  **[m inutes)**  **School**

**Burden (hours)**  **4th Grade** Science pilot (P&P, HOTs) 1,500 25 625 152 20 51 38 30 19 **8th Grade** Operational (US history, civics, geography); Pilot (science P&P, science HOTs) 27,500 25 11,458 2,742 20 or 30 1 937 457 30 229 **12th Grade** Science pilot (P&P, HOTs) 1,500 25 625 - - - 38 30 19 **Total 30,500**  **N/A 12,708**  **2,894 N/A**  **988 533**  **N/A 267**  **Total number of respondents:**  **33,927 Total burden hours:**  **13,963** Note: Due to rounding, some totals may differ slightly from sum of subtotals.

1 It is estimated that 95% of the grade 8 teachers teach one subject and, thus, have 20 minutes of burden, while about 5% of the grade 8 teachers teach both social studies and science and, thus, have 30 minutes of burden.201 4 Wave 1 Supporting Statement : Ap pendix A 8 **6 . How,**  **by Whom, and for What Purpose the Data Will Be Used** The purpose of NAEP is to collect and report assessment data on student achievement in the subject areas assessed for use in monitoring education al progress. In addition to reporting overall results of student performance and achievement, NAEP also reports student performance results for various subgroups of students and on various educational factors. The Governing Board sets guidance for what is asked in the questions . NCES is responsible for developing the question s and for selecting the final set of questions. Th e questions are designed to (a) provide the information for disaggregating data according to categories specified in the legislation , 4 (b) provide contextual information that is subject specific (e.g. reading, mathematics) and has an impact and known relationship to student achievement, and (c) provide policyrelevant information specified by the Governing Board. In the original request to OMB for s ys tem clearance, NCES requested approval of the data - gathering instruments to be used in the 2014

–2016 national - and state - lev el NAEP assessments. An expanded description of the purposes of the data is provided in the system clearance submittal. **7 . Estimates of Costs to the Federal Government**  Table 6 provides estimates for the Wave 1 portion of the 2014 administrations. **Table 6**  **: Administration Cost Estimates**  **(Wave 1 subjects) Activity**  **Provider Estimated Cost**  P rinting, packaging, and distribution phases of the administration including: • prepare and package the assessment and all auxiliary materials • distribute assessment booklets and materials to the test administrators for each school Pearson (the Materials, Distribution, Processing and Scoring [ MDPS ] contractor) $ 1.3 million Field administration including: • administration of assessments data collection from the field Westat (the Data Collection contractor and the NAEP Support and Service Center [NSSC ] contractor) $ 1.6 million **Total s**  **$ 2.9**  **million**

4 Education Sciences Reform Act of 2002 (ESRA), National Assessment of Educ ational Progress (20 USC § 9622).201 4 Wave 1 Supporting Statement : Ap pendix A 9 **Appendix A: Committee Lists NAEP Civics Standing Committee**  **Name Affiliation**  Patricia Avery University of Minnesota , Minneapolis, MN Steve Camicia Utah State University , Logan, UT Michael Fischer Consultant, Encino, CA Daphne Greene Doe Elementary School, Mountain City, TN Lewis Huffman South Carolina Department of Education Peter Levine Tufts University , Medford/Somerville, MA Kimberly Eggborn Howard County Public Schools, Ellicott City, MD Ayo Magwood Cesar Chavez Public Charter School, Washington, DC Rick Miller Princeton Regional Schools, Princeton, NJ Richard Ochoa Alta High School, Sandy, UT Clarissa Peterson DePauw University , Greencastle, IN Joseph Stewart, Jr. Clemson University , Clemson, SC Marcie Taylor- Thoma Maryland State Department of Education , Baltimore, MD Kim Tetzlaff Banyan Elementary School, Newbury Park, CA Jackie Viana Hialeah Gardens M iddle School, Miami, FL **NAEP Geography Standing Committee Name**  **Affiliation** Sarah Bednarz Texas A&M University , College Station, TX Osa Brand National Council for Geographic Education, Washington, DC Karen Thomas - Brown University of Michigan

–Dearborn , Dearborn, MI Charlie Fitzpatrick ESRISchools, Arlington, VA Donna Fontaine Westside Elementary School, Smithfield, VA Paul Gray Russellville High School, Russellville, AR Harry Hanna Frederick County Public School, Walkersville, MD Briavel Holcomb Rutgers University , New Brunswick, NJ Niem Huy

nh Association of American Geographers, Washington, DC Lauren Mitterman Gibraltar Middle School, Fish Creek, WI201 4 Wave 1 Supporting Statement : Ap pendix A 10 David Palmer Eaglecrest High School, Centennial, CO Harris Pay

ne Omaha Public Schools, Omaha, NE Joe Stoltman Western Michigan University , Kalamazoo, MI Kelly Swanson Johnson Senior High, St. Paul, MN Fred Walk Illinois State University , Normal, IL Sonja Yow Eastern Kentucky University, Richmond, KY **NAEP Science Standing Committee Name**  **Affiliation** Alicia Cristina Alonzo Michigan State University , East Lansing, M I Carlos Ayala Sonoma State University, Rohnert Park, CA Mary Thandi Buthelezi Wheaton College, Wheaton, IL Susan Craft Hanahan High School, Hanahan, SC George Deboer American Association for the Advancement of Science, Washington, DC Alex Decaria Millersville University , Millersville, PA Crystal Edwards Lawrence Township Public Schools, L awrenceville, NJ Ib ari Igwe Shrewd Learning, Elkridge, MD Michele Lombard Swanson Middle School, Arlington , VA Ellen Mingione Delaware Department of Education , Dover, DE Brett Moulding Utah Partnership for Effective Science Teaching and Learning, Ogden, UT Amy Pearlmutter Littlebrook Elementary School, Princeton, NJ Steve Semken Arizona State University , Tempe , AZ Gerald Wheeler National Science Teacher Association, Arlington, VA David White Consultant, Berlin, VT **NAEP U.S. History Standing Committee Name**  **Affiliation** Keith Barton Indiana University , Bloomington, IN Konstantin Dierks Indiana University , Bloomington, IN Cosby Hunt B ell Multicultural High School, Washington, DC201 4 Wave 1 Supporting Statement : Ap pendix A 11 Linda Levstik University of Kentucky , Lexington, KY Louisa Moffitt Marist School, Atlanta, GA Alana Murray Loiederman Middle School, Silver Spring, MD Caleb Perkins International Education Curriculum, Instruction, & Assessment, Oly mpia, WA Kim Rasmussen Auburn Washburn Unified School District, Topeka, KS Tina Rey nolds Woodfill Elementary School, Fort Thomas, KY Larry Starr Nebraska Department of Education , Lincoln, NE Kathleen Anderso n Steeves National Council for History Education, College Park, MD Diana Turk New York University , New York, NY Rebecca Valbuena Stanton School, Glendora, CA Bruce Vansledright University of Maryland , College Park, MD201 4 Wave 1 Supporting Statement : Appendix D 12

**Appendix B**  **: Parent** **/Guardian Notification Letter** 

**NAEP (NAEP Assessment Year)**  **PARENT/GUARDIAN NOTIFICATION LETTER** (School Letterhead) **(Insert Date Here)** Dear Parent or Guardian: We are pleased to notify you that (school name) has been selected to represent schools across the nation by participating in the National Assessment of Educational Progress (NAEP). First administered in 1969, NAEP is the largest nationally representative a ssessment of what students know and can do in various subject areas. It is administered by the National Center for Education Statistics, within the U.S. Department of Education. NAEP is different from state assessments because it provides a common measure of student achievement across the country. The results of NAEP are released as The Nation's Report Card, which provides information about student achievement to educators, parents, policymakers, and the public. In our school, the NAEP assessment will be gi ven on (date) in (subject) . Your child (may be/has been) selected to take the assessment. In addition to subject area questions, students will be asked some questions about themselves and their educational experience. These questions provide contextual inf ormation for the assessment, as well as information that may be related to students’ learning. If you would like to view sample subject area and student questions, please visit http://nationsreportca rd.gov/parents.asp . It will take about (assessment time) minutes for most students to participate in the assessment. The results are completely confidential (in accordance with the Confidential Information Protection provisions of Title V, Subtitle A, Public Law 107 - 347), and the information provided will be us ed for statistical purposes only. Your child’s grades will not be affected. Your child may be excused from participation for any reason, is not required to complete the assessment, and may skip any question. While NAEP is voluntary, we depend on student pa rticipation to provide an accurate measure of student achievement that will inform improvements in education. Your child will represent many other students, so participation is very important. However, if you do not want your child to participate, please n otify me in writing by (date). There is no need to study in preparation for NAEP. We do, however, ask parents to encourage their children to do their best and get plenty of rest the night before the assessment. A brochure that explains what participation in NAEP means for you and your child is available at http://nces.ed.gov/nationsreportcard/pdf/parents/2012469.pdf . Should you have questions, please contact me at (telephone numbe r) or at (e - mail address) . We are excited that our school will be participating in NAEP and pleased that your child (may be/has been) selected. We know that (school name) 's students will help us show what our nation’s students know and can do. Sincerely, School Principa l