# N

ATIONAL 

# C

ENTER FOR 

# E

DUCATION 

# STATISTICS 

# N ATIONAL 

# A SSESSMENT OF 

# E DUCATIONAL 

# P

ROGRESS 

# SUPPORTING STATEMENT PART A Request for System Clearance for NAEP Assessments for 2014 - 2016 OMB# 1850- 0790 v.36

November 1 , 2012 (revised 1 - 17 - 13)**Table of Contents**

A. JUSTIFICATION ................................ ................................ ................................ ................................ ............... 1 1a. Circumstances making the collection of information necessary ................................ ................................ ......... 1 1b. Overvie w of

NAEP assessments ................................ ................................ ................................ ........................ 2 1c. Overvie w of

2014

–2016 NAEP assessments ................................ ................................ ................................ ..... 4 1d. Rationale for OMB System Clearance ................................ ................................ ................................ ............... 7 2. How , by

whom, and for what purpose the data will be used ................................ ................................ .............. 8 3. Use of techniques to reduce burden ................................ ................................ ................................ .................... 9 4. Efforts to identify duplication ................................ ................................ ................................ ........................... 1 0 5. Burden on small businesses or other small entities ................................ ................................ .......................... 1 1 6. Consequences of collecting information less frequently ................................ ................................ .................. 1 1 7. Consistency with 5 CFR 1320.5 ................................ ................................ ................................ ....................... 1 1 8. Consultations outside the agency ................................ ................................ ................................ ..................... 1 1 9. Payments or gifts to respondents ................................ ................................ ................................ ...................... 1 2 10. Assurance of confidentiality ................................ ................................ ................................ ............................. 1 3 11. Sensitive questions ................................ ................................ ................................ ................................ ........... 1 6 12. Estimation of respondent reporting burden (2014

–2016) ................................ ................................ ................. 1 6 13. Cost to respondents ................................ ................................ ................................ ................................ ........... 2 1 14. Estimates of cost to the federal government ................................ ................................ ................................ ..... 2 1 15. Reasons for changes in burden (from last System Clearance submittal) ................................ .......................... 2 2 16. Time schedule for data c ollection ................................ ................................ ................................ ..................... 2 2 17. Approval for not displaying OMB approval expiration date ................................ ................................ ............ 2 2 18. Exceptions to Certification Statement ................................ ................................ ................................ .............. 2 2 B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS ................................ ....... 1 Appendix A Statute Authorizing NAEP ................................ ................................ ................................ .................. A - 1 Appendix B External Consultants NAEP Design and Analysis Committee ................................ ................................ ................................ ........ B - 1 NAEP Validity Studies Panel ................................ ................................ ................................ ......................... B - 2 NAEP Quality Assurance Technical Panel ................................ ................................ ................................ ..... B - 3 NAEP Socio - Economic Status Panel ................................ ................................ ................................ ............. B - 4 NAEP National Indian Education Study Technical Revie w Panel ................................ ................................ . B - 5 NAEP Civics Standing Committee ................................ ................................ ................................ ................. B - 6 NAEP Economics Standing Committee ................................ ................................ ................................ ........ B - 7 NAEP Geography Standing Committee ................................ ................................ ................................ ........ B - 8 NAEP Mathematics Standing Committee ................................ ................................ ................................ ..... B - 9 NAEP Reading Standing Committee ................................ ................................ ................................ ........... B - 10 NAEP Science Standing Committee ................................ ................................ ................................ ........... B - 11 NAEP Technology and Engineering Literacy Standing Committee ................................ ........................... B - 12 NAEP U.S. History Standing Committee ................................ ................................ ................................ .... B - 13 NAEP Writing Standing Committee ................................ ................................ ................................ ............ B - 14 Appendix C Example of Sample Design Document (2013 Assessment) ................................ ................................ . C - 1 Appendi x D Sam

ple Parental Notification and School Coordinator Responsibiliti e s ................................ .............. D - 1 Appendi x E Sam

ple School Coordinator Responsibilities Brochure ................................ ................................ ...... E - 1NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 1 

### A. **JUSTIFICA**

**TION 1a.**  **Circumstances making the collection of information necessary**  The National Assessment of Educational Progress (NAEP) is a federally authorized survey

of student achievement at grades 4, 8, and 12 1 in various subject areas, such as mathematics, reading, writing, science, U.S. history , civics, geography, economics, technology and engineering literacy (TEL) , and the arts. In the current legislation that reauthorized NAEP , the National Assessment of Educational Progress Authorization Act (Public Law 107 - 279 Title III, section 303 ) , Congress mandate s the collection of national education survey data through a national assessment program. The National Assessment of Educational Progress Authorization Act also requires the assessment to collect data on specified student groups and characteristics , including information organized by race/ethnicity , gender, socio - economic status, disability, and limited English proficiency . It requires fair and accurate presentation of achievement data and permits the collection of background , noncognitive, or descriptive inf ormation that is related to academic achievement and aids in fair reporting of results. The intent of the law is to provide representative sample data on student achievement for the nation, the states, and subpopulations of students and to monitor progress over time. NAEP is administered by the National Center for Education Statistics (NCES) in the Institute for Education Sciences of the U.S. Department of Education. The National Assessment Governing Board (henceforth referred to as the Governing Board) s ets policy for NAEP and determines the content framework for each assessment. As a result of the National Assessment of Educational Progress Authorization Act , the Governing Board has final authority on the appropriateness of all cognitive and noncognitive assessment items. The NAEP assessments are conducted b y an alliance of organizations under contract with the U.S. Department of Education . 2 The se assessments contain different kinds of item s — “cognitive” assessment item s, which measure student knowledge of an academic subject , and “noncognitive” or “background” item s, which gather factual information such as demographic variables, as well as construct - related information, such as courses taken . A cop y of the current statute is included in a ppendix A.

1 As described in section 1b, main NAEP assesses students in grades 4, 8, and 12, and long - term trend NAEP assesses students at ages 9, 13, and 17. 2 The current contract expires on March 26, 2013. A new contract will be awarded prior to that date .NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 2 

### 1b. **Overview of NAEP assessments**  The following provides a broad overview of NAEP assessments. Please note that the Governing Board determines NAEP policy and assessment schedule , 3 and future Governing Board decisions may result in changes to s ome aspects of an assessment (e.g., which subjects are assessed in which years). However, the overall methodology

and assessment process will remain consistent. NAEP consists of two assessment programs: the NAEP long - term trend (LTT) assessment and the main NAEP assessment. The LTT assessments are designed to give information on the changes in academic performance and have been administered for over 40 years . They are administered nationall y every four ye ars (but are not reported at state or district level) and report student performance at ages 9, 13, and 17 in mathematics and reading. The main NAEP assessment s report current achievement levels and short - term trends in student achievement at grades 4, 8, and 12 for the nation and, for certain assessments, states and select urban districts. These assessments follow subject - area frameworks developed by the Governing Board and use the latest advances in assessment methodology . The subject - area frameworks evolve to match instructional practices. NCES is responsible for developing the cognitive items and for selecting the final set of items. In addition to reporting overall results of student performance and achievement, NAEP also reports student performance results for various subgroups of students and on various educational factors. Guidance for what is asked in the questionnaires is set by the Governing Board. NCES is responsible for developing the noncognitive items and for selecting the final set of items . T h e items are designed to (a) provide the information for disaggregating data according to categories specified in the legislation, (b) provide contextual information that is subject - specific (e.g. reading, mathematics) and has an impact and known relationship to achievement, and (c) provide policy relevant information specified by the Governing Board. Q uestionnaires are generally administered to students at grades 4, 8, and 12; to teachers at grades 4 and 8 ( and at grade 12 for economics ); and to school administrators at grades 4, 8, and 12. Students with disabilities (SD) and English language learners (ELL) worksheets are completed by teachers or

3 The Governing Board assessment schedule can be found at http://www.nagb.org/naep/assessment - schedule.htm .NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 3

administrators of students identified as SD or as ELL , in order to determine if individual students can meaningfully

participate and, if so, which accommodation ( s ) they should receive . Assessment ty

pes listed in this document are described as follows: Operational An assessment whose results will be used for NAEP reporting

purposes. Pilot An assessment that contains a pretest of items and possible assessment

conditions to obtain information regarding clarity, difficulty levels, timing, and feasibility of items and conditions. Special Study A study to investigate topics s uch as content issues, delivery options,

accommodations, linking to other surveys, or reporting variables. All three ty

pes of activities may

simultaneousl y be in the field during any given data collection effort. Each is described in more detail below: 1) **O perational assessments**  **:** “ Operational ” NAEP administrations, unlike pilot administrations, collect data to publicl y report on the educational achievement of students. The NAEP results are reported in the Nation’ s Report Card , which is used by policy makers, state and local educators, principals, teachers, and parents to inform educational policy decisions. 2) **Pilot testing :**  Pilot testing of cognitive and noncognitive items is carried out in all subject areas. In addition to ensuring that items measure what is intended, the data collected from pilot tests serve as the basis for selecting the most effective items and data collection procedures for the subsequent operational assessments. Pilot testing is a cost - ef fective means for revising and selecting items prior to an operational data collection because, while fewer numbers of students participate, the items are administered to a nationally representative sample of students and data are gathered about performanc e that crosses the spectrum of student achievement. It ems that do not work well can be dropped or modified before the operational administration. For example, when pilot data indicate very low response percentages in some response categories, the number of response categories provided can be reduced.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 4 Prior to pilot testing, many new items are pre - tested with small groups of sample participants (cleared under the NCES pretesting generic clearance agreement; OMB #1850 - 0803 ) . All noncognitive items undergo one - on - one cognitive interviews, which is useful for identifying questionnaire and procedural problems before larger scale pilot testing is undertaken. Select cognitive items also undergo pre -pilot testing, such as item tryouts or cognitive interviews, in order to test out new item ty pes or formats or challenging content. 3) **Special studies :**  Special studies are an opportunity for NAEP to investigate particular aspects without impacting the reporting of NAEP results. Previous special studies have focused on linking NAEP to other assessments or linking across NAEP frameworks, investigating alternative sets of items for particular samples, evaluating specific accommodations, investigating administration modes (such as computer- del ivered assessment alternatives), and providing targeted data on specific student populations . In addition to the overarching goal of NAEP to provide data about student achievement at the national, state, and district levels, NAEP also provides speciall y t argeted data on an as needed basis. At times, this may onl y mean that a special anal ys is of the existing data is necessary . At other times the targeting may include the addition of a short add - on questionnaire targeted at specified groups. For example, in the past, additional student, teacher, and school questionnaires were developed and administered as part of the National Indian Education Study

(NIES) that NCES conducted on behalf of the Office of Indian Education. Through such targeted questionnaires, important information about the achievement of a specific group is gathered at minimal additional burden. These typ es of special studies are intentionally

kept to a minimum and are designed to avoid jeopardizing the main purpose of the program. 

### 1c. **Overview of**  **2014 –** **2016 NAEP**  **assessments** The basic methodology

and procedures described under this system clearance will be employed for the 2014, 2015, and 2016 NAEP assessments. Subsequent clearance packages submitted for the specific assessment years will describ e the scope of the assessment, including additional studies, school and student samples, estimated burden, and items requiring clearance. The 2014 data collection consists of the following:NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 5

 Operational national assessments for U.S. history , civics, and geography at grades 4, 8, and 12 ;  Operational national assessment for TEL 4 at grade 8 ;  Pilot assessments for 201 5 science , including interactive computer tasks ( ICT s) 4 and hands - on tasks (HOTs ) , at grades 4, 8, and 12 ; and  Pilot assessments for 201 5 reading and mathematics at grade 12 . The 2015 data collection consists of the following:  Operational n ational and state assessment s in reading, math ematics , and science at grades 4 and 8 ;  Operational national , plus 15 states , assessments in reading and mathematics at grade 12;  Operational n ational assessments in science ICT s 4 and HOT s at grades 4, 8 and 12 ;  Operational Puerto Rico mathematics assessment at grades 4 and 8; and  Pilot assessments for 2017 reading, mathematics , and writing4 at grades 4, 8 , and 12 . The 2016 data collection consists of the following:  Operational national arts assessment at grade 8;  Operational national LTT assessments in reading and mathematics for ages 9, 13, and 17;  Pilot assessments for 2020 LTT reading and mathematics for ages 9, 13, and 17;  Pilot assessments for 201 8 U.S. history , geography , and civics for grades 4, 8, and 12 ; and  Pilot assessment for 2018 economics at grade 12 . Special Studies Special studies are conducted in accordance with the assessment development, research, or additional reporting needs of NAEP. For example , over recent assessment cycles the following special studies were conducted:  NAEP

–Trends in International Mathematics and Science Study **(** TIMSS ) Li nking Study ;  Reading Accessible Blocks Study ;

4 TEL, Science ICT, and w riting will be administered by computer. All other asses sments are currently planned as a paper- and pencil administration.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 6

 Knowledge and Skills Appropriate (KaSA) Study (Mathematics) ;  NAEP

–Lexile © Study (Reading) ;  NAEP

–Program for International Student Assessment (PI SA) Linking Study (Mathematics) ; and  NAEP

–High School Longitudinal Study (HSLS) Study . Over the c ourse of the 2014

–2016 NAEP assessments, special studies will be conducted. The specifics of future studies will be included in the subsequent OMB submittals. Though an exhaustive list of special studies cannot be provided given the real - time nature of these projects, two special studies are currently being considered :  High School Transcript Study (HSTS)

– The NAEP HSTS, planned for 2015, periodicall y surveys th e curricula being followed in our nation’s high schools and the course - taking patterns of high school students through a collection of transcripts. The HSTS information is provided by a sample of the public and private schools and the sample of students is representative of the U.S. population of graduating seniors . The transcript study includes onl y those students whose transcripts indicate that they graduated the year that the study was conducted. Most of the students sampled in the transcript study are in schools that participated in NAEP. The data collected from those students that participated in NAEP make it possible to link course - taking patterns to academic performance, as measured b y NAEP.  National Indian Education Survey (NIES)

– NIES is planned as part of the 2015 assessments in reading and mathematics. The national sample includes students from both public and nonpublic schools that have both large and small American Indian/Alaska Native (AI/AN) student populations. The administration of the NAEP assessment will be followed with the administration of a questionnaire specifically designed for the NIES study . Questionnaire data will be linked to NAEP performance data. In addition, NAEP is in the process of transitioning assessments from paper- and -pen cil to computer delivered assessments . Therefore , the NAEP program will most likely undertake some special studies to explore aspects associated with the transition to computer- delivered assessments, such as item ty pes, hardware, s ys tem performance , and comparability across assessment modes.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 7 

### 1d. **Rationale for OMB System Clearance** NCES is requesting s ys tem clearance for the NAEP assessments to be administered in the 2014

–2016 timeframe, similar to the system clearance approval that was granted for the 2011

–2013 and 2008

–2010 NAEP administrations (OMB 1850 - 0790). The primary reason for the s ys tem clearance request is that it enables NAEP to meet its large and complex assessment reporting schedules and deliverables through a more efficient clearan ce process. Since the passage of the No Child L eft Behind Act in 2001 (P.L . 107 - 110), the numbers of assessments, volumes of participants, and complexity of the development and production processes have all increased dramaticall y. Because of the National Assessment of Educational Progress Authorization Act requirements, every

state participates in the reading and mathematics assessments at grades 4 and 8 and most states participate in the state - level writing and science assessments. These state - level asse ssments require that samples be selected to permit reporting of results for each state as well as for the nation, resulting in a much larger sample size than would otherwise be necessary . In addition, because of the complex matrix sampling plan used by NAE P, upwards of 1,000 distinct assessment booklets or forms may need to be printed or assembled for one assessment year alone. Given the cyclical nature of the NAEP assessments and their complexity and scope, there is a relatively short and fixed time perio d in which production can occur. Prior to the finalization of the content, many activities must be conducted, including quality reviews, anal ys is of any prior data, assembl y of forms, and approval b y the Governing Board and OMB. To meet these demanding tim elines, NCES is requesting a waiver of the 60 - day

federal register notice for clearances of the NAEP assessments and studies described in this system clearance package. Parts A and B of this system clearance package contain supporting information for the assessments to be given in 2014

–2016 . This submittal does not contain any volume II (specific questionnaire) material. Specific yearl y questionnaires will be submitted as they are developed over the course of the s ys tem clearance period. Clearance of three ye ars duration (beginning April 1 , 2013 ) is requested for the 2014

– 2016 assessments. At the end of this system clearance period (i.e., prior to March 31 , 2016 ), NCES will submit a new request for sy stem clearance to cover future NAEP administrations.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 8 

### 2. **How , by w**  **hom, and for w hat purpose the data will be used**  Under this request for s ystem clearance, NCES is asking for approval of the planned assessments, including the various questionnaires (e.g., student, teacher, and school) and data collection efforts that will be part of the NAEP 2014

–2016 national, state, and district assessments, as well as any special studies that are planned to be conducted in the 2014

–2016 timeframe. Results will be reported on the 2014 operational assessments in U.S. history , civics, geography, and TEL ; the 2015 operational assessments in reading , mathematics , and science , including ICT s and HOT s ; and the 2016 operational arts assessment and long - term trend assessment s in reading and mathematics. The schedule for subjects to be a ssessed has been established by the Governing Board; however, it is subject to modification. Therefore, NCES is requesting some leeway with regard to the specific subject assessments that will be administered, while holding the methodology and general proc edures constant. NAEP uses the results from the pilot tests and some special studies to inform future assessments and procedures. Item s are evaluated for their effectiveness and appropriateness to be used in an operational assessment. Results from the op erational assessments and some special studies are released to the public. The NAEP results are reported in the Nation’s Report Card, which is used by policy makers, state and local educators, principals, teachers, and parents to help inform educational pol icy decisions . The NAEP Report Cards provide national results, trends for different student groups, results on scale scores and achievement levels, and sample item s. In reports with state or urban district results, there are sections that provide overview information on the performance of these jurisdictions. NAEP does not provide scores for individual students or schools. In addition to the Report Card , more infor mation is available online (http://nationsreportcard.gov/) and in one -page summary

reports , called snapshots , for each participating state or urban district. Additional data tools are available online for those interested in:  anal yz ing NAEP data and creating tables and graphics (http://nces.ed.gov/nationsreportcard/naepdata/);  comparing state performance b y various demographic groups (http://nces.ed.gov/nationsreportcard/statecomparisons/);  comparing districts (http://nces.ed.gov/nationsreportcard/distric ts/) ;NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 9

 seeing the knowledge and skills demonstrated by students performing at different scale scores (http://nces.ed.gov/nationsreportcard/itemmaps/);  seeing NAEP performance results and student demographics for each state (http://nces.ed.gov/nationsreportcard/states/); and  searching, sorting, and providing data for sample NAEP item s (http://nces.ed.gov/nationsreportcard/itmrlsx/). In addition to contributing to the reporting tools mentioned above, data from the questionnaires are used as part of the marginal estimation procedures that produce the student achievement results. In addition, questionnaire data is used to perform quality control checks on school - reported data and in special reports, such as the Black

– White Achievement Gap report. Finally , there are numerous opportunities for secondary data anal ys is because of NAEP ’ s large scale, the regularity of its administrations, and its stringent quality control processes for data collection and anal ys is. NAEP data are used by researchers and educators who have diverse interests and varying levels of anal ytical experien ce. 

### 3. **Use of techniques to reduce burden** For all operational and pilot assessments, NAEP will continue to take advantage of proven, modern measurement techniques, which greatl y enhance the power and value of the NAEP data collected. Through the use of a partial balanced incomplete block (BIB) spiraling variant of matrix sampling, a variety of anal ys es are feasible because the data are not booklet bound. Covariances are computed among all item s in a subject area, so that  composites of item s can be appraised empiricall y for coherence and construct validity ;  the dimensional structure of each subject area can be determined anal ytically as reflected in student performance consistencies;  item response theory

(IRT) scaling can be applied to unidimensional sets of items or tasks, regardless of which boo klet they appear in;  IRT scales can be developed having common meaning across exercises, population subgroups, age levels, and time periods;  powerful trend anal ys es can be undertaken b y m eans of these common scales;NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 10

 performance scales can be correlated with background, attitudinal, and program variables to address a rich variety

of educational and policy issues; and  public - use electronic files can be made much more useful because secondary anal ys es are also not booklet -bound. To minimize the burden for participating schools and students, the following procedures will be used:  No student takes the complete assessment and, given that NAEP results are reported for groups of students, using a BIB spiral allows the program to administer a lengthy overall assessment without overburdening any one student.  Trained administrators will conduct the assessments at all grades.  Assessment administrations will be limited, whenever possible, to approximately 90 minutes to facilitate school scheduling (note, computer- delivered assessments may be scheduled for up to 120 minutes). The various questionnaires (student, teacher, and school) are constructed to minimize respondent burden using the information gathered from pilot testing. Based on pilot test data, items that do not yi eld useful information or that produce redundant information can be eliminated. Also, to reduce respondent burden, NAEP may

employ some or all of the following techniques:  maintaining items across administrations rather than piloting new ones;  rotating some non - required items across assessments;  spiraling items across respondents to provide a broader range of content coverage while minimizing individual respondent burden; and  reducing the number of new item s piloted for existing frameworks. Another technique for improving the response process consists of pre - testing items using cognitive interviews, item tryouts, and usability studies on a smaller number of students to maximize the success of items. These techniques allow items to be refined before they go to larger groups of students. 

### 4. **Efforts to identify duplication** The proposed assessments, including the questionnaires, do not exist in the same format or combination in the U.S. Department of Education or elsewhere. The noncognitive data gathered by NAEP comprise theNAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 11

only comprehensive cross - sectional survey performed regularl y on a large - scale basis that can be related to extensive achievement data in the United States . No other federall y funded studies have been designed to collect data for the purpose of regularl y assessing trends in educational progress and comparing these trends across states . None of the major non - federal studies of educational achievement were designed to measure changes in national achievement. In short, no existing data source in the public or private sector duplicates NAEP. 

### 5. **Burden on small businesses or other small entities** The school samples for NAEP contain small - , medium - , and large - size schools, including private schools . Schools are included in the sample proportio nal to their representation in the population , or as necessary to meet reporting goals . It is necessary to include small and private schools so that the students attending such schools are represented in the data collection and in the reports . 

### 6. **Conseque nces of collecting information less frequently**  Under the National Assessment of Educational Progress Authorization Act , Congress has mandated the on - going collection of NAEP data. Failure to collect the 2014

–2016 assessment data on the current schedule would affect the quality and schedule of the NAEP assessments, and would result in assessments that would not fulfill the mandate of the legislation. 

### 7. **Consistency with 5 CFR 1320.5** No special circumstances are involved. This data collection observes all re quirements of 5 CFR 1320.5. 

### 8. **Consultations outside the agency** In addition to the contractor s responsible for the development and administration of the NAEP assessments, the program involves many consultants and is also reviewed by specialists serving on various technical review panels . These consultants and special reviewers bring expertise concerning students of different ages, ethnic backgrounds, geographic regions, learning abilities, and socioeconomic levels ; the specific subject areas being assessed ; the anal ys is methodologies employed; and large - scale assessment design and practices . Contractor staff and consultants have reviewed all item s for bias and sensitivity

issues, grade appropriateness, and appropriateness of content across states .NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 12

In particular , subject area standing committees play a central role in the development of NAEP assessment instruments and have been essential in helping create assessment content that is appropriate for the targeted populations, and that meets the expectations outlined in the Governing Board frameworks. One of the most important functions of the committee s is to contribute to the validation of the assessments. Through detailed reviews of items, scoring guides, tasks, constructed- respo ns e item training sets for scorer s , and other materials, the committees help establish that the assessments are accurate, accessible, fair, relevant, and grade - level appropriate and that each item measures the knowledge and skills it was design ed to measure. When appropriate, members of subject area standing committees will also review the questionnaires with regards to appropriateness with existing curricular and instructional practices. The following examples of outside personnel , based on current membership, are provided in a ppendix B:  NAEP Design and Anal ysis Committee  NAEP Validity

Studies Panel  NAEP Quality

Assurance Technical Panel  NAEP Socio - Economic Status Panel  NAEP National Indian Education Study Technical Review Panel  NAEP Civics Standing Committee  NAEP Economics Standing Committee  NAEP Geography Standing Committee  NAEP Mathematics Standing Committee  NAEP Reading Standing Committee  NAEP Science Standing Committee  NAEP Technology

and Engineering Literacy Standing Committee  NAEP U.S. History

Standing Committee  NAEP Writing Standing Committee As has been the practice for the past few ye ars, OMB representatives will be invited to attend the technical review panel meetings that are most informative for OMB purposes. 

### 9. **Payments or gifts**  **to respondents**  In general, there will be no gifts or pay ments to respondents, although students do get to keep the NAEP pencils or earbuds used in the paper - and -pencil and computer- delivered assessment s, respectively . On occasion, NAEP will leave educational materials behind at schools for their use ( e . g ., science kits from the science HOTs assessments). Schools participating in the High School Transcript Study are paid the established fee for providing student transcripts. Given that the study pays schools the prevailing rate to perform a standard service, estimates of school - level burden for that function are not included in thisNAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 13

volume. Special studies (i.e., cognitive lab studies and background item studies) sometimes include remuneration for respondents and will be explicated in the specific - ye ar clearance package submittals. Some schools also offer recognition parties with pizza or other perks for students who participate ; however , these are not reimbursed by NCES or the contractor. If any incentives are proposed as part of a future special study , they would be justified as part of that future clearance package. In addition, as appropriate, the amounts would be consistent with amounts approved in other studies with similar conditions. 

### 10. **Assurance of confidentiality**  NAEP has policie s and procedures that ensure privacy , security , and confidentiality , in compliance with the legislation (Confidential Information Protection provisions of Title V, Subtitle A, Public Law 107 - 347 and the National Assessment of Educational Progress Authorization Act). Specificall y for the NAEP project, this ensures that privacy , security , and confidentiality policies and procedures are in compliance with the Privacy

Act of 1974 and its amendments, NCES Confidentiality Procedures, and the Department of Educati on ADP Security manual. The National Assessment of Educational Progress Authorization Act requires the confidentiality of personall y identifiable information [ 20 U.S.C. §9622 (c) (3) ] : (A) IN GENERAL. -- The Commissioner for Education Statistics shall ensure that all personall y identifiable information about students, their academic achievement, and their families, and that information with respect to individual schools, remains confidential, in accordance with section 552a of title 5 , United States Code . (B) PROHIBIT ION. -- The Assessment Board, the Commissioner for Education Statistics, and any contractor or subcontractor shall not maintain any s ys tem of records containing a student ’ s name, birth information, Social Security number, or parents ’ name or name s, or any other personall y identifiable information. Each contractor develops a Data Security Plan and NCES ensures that all current contractor policies and procedures are in compliance with all NAEP security and confidentiality

requirements. In addition, all NAEP contractor staff with access to confidential NAEP information are required to sign an affidavit of nondisclosure that affirms, under severe penalty for unlawful action, that they will protect NAEP information from non - authorized acce ss or disclosure. The affidavits are in keeping with the NCES Standard for Maintaining Confidentiality (Standard 4 - 2). All contractors must also comply with directive OM: 5 - 101, which requires that all staff with access to data protected b y the Privacy Act and/or access to U.S. Department of Education sy stems and who will work on the contract for 30 day s or more go through the security screening procedures.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 14

An important privacy

and confidentiality

issue is the protection of the identity of assessed student s, their teachers, and their schools. To assure this protection, NAEP has established security procedures, described below, that closely control access to potentially identifying information. Students’ names are submitted to the Sampling and Data Collection contractor for selecting the student sample. This list also includes the month/year of birth, race/ethnicity , gender, and a status code for students with disabilities , English language learners , and participation in the National School Lunch Program. The student sample is selected and the data for selected students are submitted to the Materials Preparation, Distribution, Processing and Scoring (MDPS) contractor, who includes the data in the P ackaging and D istribution system for the production of the Administration Schedule and student labels, which are then forwarded to field staff and used to manage and facilitate the assessment. These data are also added to the School Control System (SCS) used by field staff to print materials used by the schools. Student information is deleted from the Packaging and Distribution system after the assessment begins. All labels are removed from the assessment materials and destro yed at the schools, upon completion of the assessment. The section of the Administration Sche dule with names is removed and placed in the school storage envelope . The school storage envelope contains all of the forms and materials with student names. It is kept at the school until the end of the school year and then destroyed b y s chool personnel. In addition to student information, teacher and principal names are collected and recorded on a roster, which is used to keep track of the distribution and collection of NAEP teacher and school questionnaires. The roster is kept at the school until questio nnaires are collected. At that time, questionnaires and the portion of the roster without names is returned for processing. The portion of the roster with names is kept at the school, in the school storage envelope , which is destroyed at the end of the sch ool year. The completed Administration Schedules and teacher and school questionnaire rosters (without student, teacher, or principal names) are returned to the MDPS contractor for processing. Furthermore, to ensure the confidentiality of respondents, NAEP staff will use the following precautions:  Data files will not identify individual respondents.  No personall y identifiable information, either b y s chools or respondents, will be gathered or released b y third parties. No permanent files of names or other direct identifiers of respondents will be maintained.  Student participation is voluntary .NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 15

 NAEP data are perturbed. Data perturbation is a statistical data editing technique implemented to ensure privacy

for student and school respondents to NAEP’s assessment questionnaires for assessments in which data are reported or attainable via restricted - use licensing arrangements with NCES . The process is coordinated in strict confidence with the IES Dis closure Review Board (DRB) , with details of the process shared onl y with the DRB and a minimal number of contractor staff. After the components of NAEP are completed in a school, neither student- nor teacher- reported data are retrievable b y personal identi fiers. We emphasize that confidentiality is assured for individual schools and for individual students, teachers, and principals. The following text is printed on all student, teacher, and school questionnaires: The information you provide will be used for statistical purposes only. In accordance with the Confidential Information Protection provisions of Title V, Subtitle A, Public Law 107 - 347 and other applicable Federal laws, your responses will be kept confidential and will not be disclosed in identifiab le form to anyone other than employees or agents. By law, every NCES employee as well as every agent, such as contractors and NAEP coordinators, has taken an oath and is subject to a jail term of up to 5 years, a fine of up to $250,000, or both if he or sh e willfully discloses ANY identifiable information about you. In addition, parents are notified of the assessment. Appendix D includes a sample parental notification letter regarding NAEP. The letter is ada pted for each grade/subject combination and the school principal may edit it. However, the information regarding confidentiality and the appropriate law reference will remain unchanged . The NAEP state coordinators 5 work with the school coordinators in obtaining student and school participation in NAEP. A sample brochure from the NAEP state coordinators to the participating schools describing these school coordinator activities is included as appendix E. For the HSTS component of NAEP, student transcripts are collected from schools for sampled students, an d school staff members complete a School Information Form that provides general information about class periods, credits, graduation requirements, and other aspects of school policy . To maintain the privacy of student and school identities, students’ names are removed from the transcripts and questionnaires at the school and given a unique identification number, which is used to match the transcript records to the NAEP questionnaire and performance information, on an individual basis. NCES ensures that the data collected from schools and students are used for statistical purposes onl y.

5 The NAEP State Coordinator serves as the liaison between the state education agency and NAEP, coordinating NAEP activities in his or her state.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 16 

### 11. **Sensitive questions** The National Assessment of Educational Progress emphasizes voluntary respondent participation, assures confidentiality

of individual responses, and avo ids asking for information that might be considered sensitive or offensive. In sensitive or offensive item s are prohibited b y the National Assessment of Educational Progress Authorization Act , and the Governing Board reviews all items for bias and sensitivi ty . Throughout the item development process, NCES staff works with consultants, contractors, and internal reviewers to identify and eliminate potential bias in the item s. 

### 12. **Estimation of respondent reporting burden ( 2014** **– 2016**  **)** Because there are such fluctuations in the burden numbers over the three years of this collection , we will continue our practice of showing the estimate of each ye ar ’ s burden rather than averaging the total over the three ye ars. The burden numbers and respondents will be changed accordingl y for each ye ar, based on the final list of assessments and special studies to be assessed. Exhibit 1 provides the burden information per respondent group, b y grade and b y ye ar , for the 2014

–2016 assessments . Exhibit 2 summarizes the burden acro ss the three years. A description of the respondents or study is provided below, as supporting information for Exhibit 1 :  Students - Students in fourth , eighth , and twelfth grades complete assessment booklets or forms that contain 50 or 60 minutes of cogni tive blocks, followed by noncognitive block(s ) which require a total of 15 minutes to complete. The core noncognitive items are answered b y all students across subject areas and are related to demographic information. In addition, students answer subject - specific noncognitive items . Additional student burden accounts for time to read directions, distribute test booklets (for paper- and -pencil assessments), and log on to the computer and view a tutorial (for computer- delivered assessments). This addi tional burden is estimated at 10 minutes for paper - and -pencil and 15 minutes for computer- delivered assessments. Therefore, the total burden for students is 25 minutes for paper- and -pencil and 30 minutes for computer delivered assessments.  Teachers - The t eachers of fourth - and eighth - grade students (and twelfth - grade economics students) participating in NAEP are asked to complete questionnaires about their teaching background, education, training, and classroom organization. Fourth - grade teacher burden is estimated to be 30 minutes because fourth - grade teachers often have multiple subject - specificNAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 17

sections to complete. Eighth - and twelfth - grade teacher burden is 20 minutes because they onl y have one subject - specific section .  Principals/Administrators - The school administrators in the sampled schools are asked to complete a questionnaire . The core item s are designed to measure school characteristics and policies that research has shown are highl y correlated with student achievement. A section with subject - sp ecific item s concentrates on curriculum and instructional services issues. The burden for school administrators is estimated at 30 minutes.  SD and ELL worksheets are completed by school personnel concerning students identified as SD or ELL . These worksheets will be used to determine the appropriate accommodations for students.  Submission of Samples - Survey sample information is collected from schools in the form of lists of potential students who may participate in NAEP. This sample information can be gathered manually or electronically

at the school, district, or state level. If done at the school or district level, some burden will be incurred b y school personnel. It is estimated that it will take one hour for school personnel to complete the submissi on process. Based on recent comparable experience, 64 percent , 48.7 percent , and 70.4 percent of the schools or districts will complete the sample submission process for 2014, 2015, and 2016, respectivel y. Note that the estimated percent is dependent upon the nature of the sample (i.e., national or state) and the assessment program (i.e., inclusion of LTT).  Pre - Assessment Visit - The pre - assessment visit is the opportunity for the NCES contractor field staff to meet with the school personnel to review procedures and logistics for the upcoming assessment. It is estimated that it will take two hour s for school personnel to complete the p re a ssessment v isit.  HSTS - The NAEP HSTS periodically surveys the curricula being followed in our nation ’ s high schools and the course - taking patterns of high school students through a collection of transcripts. This data collection requires three hours per school from a sample of approximately 800 schools.  National Indian Education Study (NIES) - NIES is designed to describe the condition of education for American Indian and Alaska Native (AI/AN) students in the United States. Additional questionnaires designed for NIES are given to student s (estimated at 15 minutes ), t eachers (20 minutes) , and school administrators (30 minutes).NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 18 

### EXHIBIT 1 **Estimated Burden for NAEP 2014 –** **2016 Assessments ,**  **By Year, By Grade Level** (Note: all explanatory notes and footnotes are displayed following the 2016 table) **2014 Subjects**  **# of Students Avg. time per**  **respondent**

**3 Student**  **Burden (in hours)**  **# of**

**Teachers Avg. time per** 

**respondent Teacher** 

**Burden (in hours)**  **# of**

**Schools Avg. time per** 

**respondent School** 

**Burden (in hours)**  **SD/ELL**

**(# of school personnel)**  **Avg. time per**

**respondent SD/ELL** 

**Burden 4**  **(in hours)** Operational (US history, civics, geography); Pilot (science P&P, science ICT, science HOTs) 21,000 25 minutes for P&P; 30 minutes for CDA 8,875 2,180 30 minutes 1,090 545 30 minutes 273 545 10 minutes 805

Special Studies 1 5,000 25 minutes for P&P; 30 minutes for CDA 2,292 512 30 minutes 256 128 30 minutes 64 128 10 minutes 192

Pre-assessment visit 673 2 hours 1,346 Sample submission

2 431 1 hour 431 **4th Grade Totals**  **26,000 N/A**  **11,167 2,692**  **N/A 1,346**  **673 N/A**  **2,114 673**  **N/A 997**  Operational (US history, civics, geography, TEL); Pilot (science P&P, science ICT, science HOTs) 47,000 25 minutes for P&P; 30 minutes for CDA 21,375 7,710 20 minutes 2,570 1,285 30 minutes 643 1,285 10 minutes 1,410

Special Studies 1 5,000 25 minutes for P&P; 30 minutes for CDA 2,292 833 20 minutes 278 139 30 minutes 69 139 10 minutes 150

Pre-assessment visit 1,424 2 hours 2,848 Sample submission

2 911 1 hour 911 **8th Grade Totals**  **52,000 N/A**  **23,667 8,543**  **N/A 2,848**  **1,424 N/A**  **4,471 1,424**  **N/A 1,560**  Operational (US history, civics, geography); Pilot (reading, math, science P&P, science ICT, science HOTs) 34,000 25 minutes for P&P; 30 minutes for CDA 14,292 531 30 minutes 266 531 10 minutes 793

Special Studies 1 5,000 25 minutes for P&P; 30 minutes for CDA 2,292 78 30 minutes 39 78 10 minutes 117

Pre-assessment visit 609 2 hours 1,218 Sample submission

2 390 1 hour 390 **12th Grade Totals**  **39,000 N/A**  **16,584 N/A**  **N/A N/A**  **609 N/A**  **1,913 609**  **N/A 910**  **Total 117,000**  **N/A 51,418**  **11,235 N/A**  **4,194 2,706**  **N/A 8,498**  **2,706 N/A**  **3,467**

**Total number of respondents 133,647**  **Total Burden (in hours) 67,577**  **4th Grade**

**8th Grade**

**12th Grade**NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 19 **2015 Subjects**  **# of Students Avg. time per**  **respondent**

**3 Student**  **Burden (in hours)**  **# of**

**Teachers Avg. time per** 

**respondent Teacher** 

**Burden (in hours)**  **# of**

**Schools Avg. time per** 

**respondent School** 

**Burden (in hours)**  **SD/ELL**

**(# of school personnel)**  **Avg. time per**

**respondent SD/ELL** 

**Burden 4**  **(in hours)** Operational (math, reading, science P&P, science ICT, science HOTs, Puerto Rico math); Pilot (reading, math, writing) 506,000 25 minutes for P&P; 30 minutes for CDA 212,083 41,640 30 minutes 20,820 10,410 30 minutes 5,205 10,410 10 minutes 19,397

Special Studies 1 10,000 25 minutes for P&P; 30 minutes for CDA 4,583 832 30 minutes 416 208 30 minutes 104 208 10 minutes 383

NIES 10,000 15 minutes 2,500 8,000 20 minutes 2,667 2,000 30 minutes 1,000 Pre-assessment visit 10,618 2 hours 21,236 Sample submission

2 5,171 1 hour 5,171 **4th Grade Totals**  516,000 **N/A** 219,166 42,472 **N/A** 23,903 10,618 **N/A** 32,716 10,618 **N/A** 19,780 Operational (math, reading, science P&P, science ICT, science HOT, Puerto Rico math); Pilot (reading, math, writing) 498,500 25 minutes for P&P; 30 minutes for CDA 208,333 47,100 20 minutes 15,700 7,850 30 minutes 3,925 7,850 10 minutes 14,955

Special Studies 1 10,000 25 minutes for P&P; 30 minutes for CDA 4,583 954 20 minutes 318 159 30 minutes 80 159 10 minutes 300

NIES 10,000 15 minutes 2,500 8,000 20 minutes 2,667 2,000 30 minutes 1,000 Pre-assessment visit 8,009 2 hours 16,018 Sample submission

2 3,900 1 hour 3,900 **8th Grade Totals**  508,500 **N/A** 215,416 48,054 **N/A** 18,685 8,009 **N/A** 24,923 8,009 **N/A** 15,255 Operational (math, reading, science P&P, science ICT, science HOTs); Pilot (reading, math, writing) 140,700 25 minutes for P&P; 30 minutes for CDA 59,333 2,325 30 minutes 1,163 2,325 10 minutes 3,283

Special Studies 1 10,000 25 minutes for P&P; 30 minutes for CDA 4,583 167 30 minutes 84 167 10 minutes 233

HSTS 800 3 hours 2,400 Pre-assessment visit 2,492 2 hours 4,984 Sample submission

2 1,214 1 hour 1,214 **12th Grade Totals**  150,700 **N/A** 63,916 **N/A N/A**  **N/A** 2,492 **N/A** 9,845 2,492 **N/A** 3,516 **Total 1,175,200**  **N/A 498,498**  **90,526 N/A**  **42,588 21,119**  **N/A 67,484**  **21,119 N/A**  **38,551**

**Total number of respondents Total Burden (in hours)**  **647,121 1,307,964**  **4th Grade**

**8th Grade**

**12th Grade**NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 20 **201 6**  **Subjects # of Students**  **Avg. time per respondent** 

**3 Student**  **Burden (in hours)**  **# of**

**Teachers Avg. time per** 

**respondent Teacher** 

**Burden (in hours)**  **# of**

**Schools Avg. time per** 

**respondent School** 

**Burden (in hours)**  **SD/ELL**

**(# of school personnel)**  **Avg. time per**

**respondent SD/ELL** 

**Burden 4**  **(in hours)** Age 9: Operational and Pilot (reading and math) 14,700 25 minutes 6,125 420 420 10 minutes 564 Age 13: Operational and Pilot (reading and math) 14,700 25 minutes 6,125 460 460 10 minutes 441 Age 17: Operational and Pilot (reading and math) 14,700 25 minutes 6,125 530 530 10 minutes 343 Pre-assessment visit 1,410 2 hours 2,820 Sample submission 2 993 1 hour 993 **LTT totals 44,100**  **N/A 18,375**  **N/A N/A**  **N/A 1,410**  **N/A 3,813**  **1,410 N/A**  **1,348** Pilot (US history, geography, civics) 1,500 25 minutes 625 140 30 minutes 70 35 30 minutes 18 35 10 minutes 58 Special Studies 1 5,000 25 minutes for P&P; 30 minutes for CDA 2,292 464 30 minutes 232 116 30 minutes 58 116 10 minutes 192 Pre-assessment visit 151 2 hours 302 Sample submission 2 106 1 hour 106 **4th Grade Totals**  **6,500 N/A**  **2,917 604**  **N/A 302**  **151 N/A**  **484 151**  **N/A 250**  Operational (arts); Pilot (US history, geography, civics) 12,500 25 minutes 5,208 2,136 20 minutes 712 356 30 minutes 178 356 10 minutes 375 Special Studies 1 5,000 25 minutes for P&P; 30 minutes for CDA 2,292 858 20 minutes 286 143 30 minutes 72 143 10 minutes 150 Pre-assessment visit 499 2 hours 998 Sample submission 2 351 1 hour 351 **8th Grade Totals**  **17,500 N/A**  **7,500 2,994**  **N/A 998**  **499 N/A**  **1,599 499**  **N/A 525**  Pilot (US history, geography, civics, economics) 6,000 25 minutes 2,500 80 30 minutes 40 80 10 minutes 140 Special Studies 1 5,000 25 minutes for P&P; 30 minutes for CDA 2,292 67 30 minutes 34 67 10 minutes 117 Pre-assessment visit 147 2 hours 294 Sample submission 2 103 1 hour 103 **12th Grade Totals**  **11,000 N/A**  **4,792 N/A**  **N/A N/A**  **147 N/A**  **471 147**  **N/A 257**  **Total 79,100**  **N/A 33,584**  **3,598 N/A**  **1,300 2,207**  **N/A 6,367**  **2,207 N/A**  **2,380**

**Total number of respondents 87,112**  **Total Burden (in hours) 43,631**  **LTT 5** 

**4th Grade**

**8th Grade**

**12th Grade**NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 21 Notes for all tables in Exhibit 1 Cells with cross - hashes are sub - sets or repeats of previous cells. Therefore, they are shown for row calculation purposes only and are not included in the column totals. Light gray s haded cells are intentionally left blank because no burden is associated with that cell. 1 - Because the specific special studies are not known at this time, a placeholder is included for burden calculation purposes. 2 - Based on most recent comparable data, it is estimated that 64% of district or school personnel will perform sample selection activities in 2014, 48.7% in 2015, and 70.4% in 2016. 3 - P&P refers to paper - and -pencil assessments and CDA refers to computer - delivered assessm ents. All special study placeholders assume that half will be P&P and half will be CDA. 4 - SD and ELL burden is calculated as the number of hours that school personnel spend to complete the SD/ELL worksheets for students identified as SD and/or ELL (estimated at 10 minutes per student). The overall SD/ELL burden is a factor of the number of students identified as SD/ELL. Based on most recent data, it is estimated that 23 percent of grade 4/age 9 students, 18 percent of grade 8/age 13, and 14 percent of gra de 12/age 17 students will be identified as SD/ELL. 5 - Teacher and school questionnaires are not administered in association with the LTT assessments. 

### EXHIBIT **2 Total**  **Estimated Burden for NAEP 2014 –** **2016 Assessments Total number of respondents**  **Total number of responses Total Burden (in hours)**  **2014** 133,647 138,085 67,577 **2015** 1,307,964 1,380,168 647,121 **2016** 87,112 90,872 43,631 **Total 1,528,723**  **1,609,126 758,329**  

### 13. **Cost to respondents**  There are no direct costs to respondents. 

### 14. **Estimates of cost to the federal government** The total cost to the federal government for the administrations of the 2014

–2016 activities is estimated to be approximately

$ 48.15 million . The cost es timate is broken down as follows:  $ 13.25 million for the printing, packaging, and distribution phases of the administrations;  $ 27.1 million for the cost for the field supervisors and data collectors to go into schools to administer the 2014

–2016 assessment s , including travel expenses and testing equipment costs ; and  $ 7.8 million for web operations and maintenance costs related to the support of the computer - delivered assessments.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 22 

### 15. **Reasons for changes in burden (from last System Clearance submittal)** The nature of NAEP is that burden alternates from a relativel y low burden in national - level administration ye ars (i.e., even ye ars) to a substantial burden increase in state - level administration years that include one or more assessments that support national , state -by - state , and certain urban districts reporting (i.e., odd ye ars) . In state/district assessment years, NAEP samples approximate ly 1,000,000 students, while in national - only

assessment years, approximate ly 100,000 students . In 2015 , NAEP will conduc t state/district assessments, and in 2014 and 2016, national - level assessments . The previous S ys tem Clearance included burden for two state/district assessments (2011 and 2013) and onl y one national - level assessment (2012) , therefore the overall number of respondents and responses is lower in this clearance request than in the previous one. However, because we have adjusted the estimated respondent time for students and the pre - assessment visit , the overall time burden estimate is slightl y higher for this versus the previous System Clearance . Previousl y, only the time spent responding to the noncognitive questionnaires was included in the total burden calculation for the students , but now we also include the time for instructions and booklet distribution , which increases the per - student burden estimate b y an additional 10 minutes for paper - and -pencil assessments and by 15 minutes for computer- delivered assessments. Finally , recent reports from the field staff have indicated that the pre - assessment visits require two hours, rather than the one hour previously estimated. Therefore, we have adjusted the burden estimate for pre assessment visit to two hours. 

### 16. **Time sch edule for data collection**  The time schedule for the data collection for the 2014

–2016 assessments is shown below. **2014** Main NAEP January

–March 2014 **2015** Main NAEP January

–March 2015 **2016** Long - Term Trend Main NAEP October 2015

–May 2016 January

–March 2016 

### 17. **Approval for not displaying**  **OMB approval expiration date**  No exception is requested. 

### 18. **Exceptions to Certification Statement** No exception is requested.