# N

ATIONAL 

# C

ENTER FOR 

# E

DUCATION 

# STATISTICS 

# N ATIONAL 

# A SSESSMENT OF 

# E DUCATIONAL 

# P

ROGRESS 

# SUPPORTING STATEMENT PART A Request for System Clearance for NAEP Assessments for 2014 - 2016 OMB# 1850 - 0790 v.36

November 1 , 2012**Table of Contents**

A. JUSTIFICATION ................................ ................................ ................................ ................................ ............... 1 1a. Circumstances making the collection of information necessary ................................ ................................ ......... 1 1b. Overvie w of

NAEP assessments ................................ ................................ ................................ ........................ 2 1c. Overvie w of

2014

–2016 NAEP assessments ................................ ................................ ................................ ..... 4 1d. Rationale for OMB System Clearance ................................ ................................ ................................ ............... 7 2. How , by

whom, and for what purpose the data will be used ................................ ................................ .............. 8 3. Use of techniques to reduce burden ................................ ................................ ................................ .................... 9 4. Efforts to identify duplication ................................ ................................ ................................ ........................... 1 0 5. Burden on small businesses or other small entities ................................ ................................ .......................... 1 1 6. Consequences of collecting information less frequently ................................ ................................ .................. 1 1 7. Consistency with 5 CFR 1320.5 ................................ ................................ ................................ ....................... 1 1 8. Consultations outside the agency ................................ ................................ ................................ ..................... 1 1 9. Payments or gifts to respondents ................................ ................................ ................................ ...................... 1 3 10. Assurance of confidentiality ................................ ................................ ................................ ............................. 1 3 11. Sensitive questions ................................ ................................ ................................ ................................ ........... 1 6 12. Estimation of respondent reporting burden (2014

–2016) ................................ ................................ ................. 1 6 13. Cost to respondents ................................ ................................ ................................ ................................ ........... 2 1 14. Estimates of cost to the federal government ................................ ................................ ................................ ..... 2 1 15. Reasons for changes in burden (from last System Clearance submittal) ................................ .......................... 2 2 16. Time schedule for data c ollection ................................ ................................ ................................ ..................... 2 2 17. Approval for not displaying OMB approval expiration date ................................ ................................ ............ 2 2 18. Exceptions to Certification Statement ................................ ................................ ................................ .............. 2 2 B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS ................................ ....... 1 Appendix A Statute Authorizing NAEP ................................ ................................ ................................ .................. A - 1 Appendix B External Consultants NAEP Design and Analysis Committee ................................ ................................ ................................ ........ B - 1 NAEP Validity Studies Panel ................................ ................................ ................................ ......................... B - 2 NAEP Quality Assurance Technical Panel ................................ ................................ ................................ ..... B - 3 NAEP Socio - Economic Status Panel ................................ ................................ ................................ ............. B - 4 NAEP National Indian Education Study Technical Revie w Panel ................................ ................................ . B - 5 NAEP Civics Standing Committee ................................ ................................ ................................ ................. B - 6 NAEP Economics Standing Committee ................................ ................................ ................................ ........ B - 7 NAEP Geography Standing Committee ................................ ................................ ................................ ........ B - 8 NAEP Mathematics Standing Committee ................................ ................................ ................................ ..... B - 9 NAEP Reading Standing Committee ................................ ................................ ................................ ........... B - 10 NAEP Science Standing Committee ................................ ................................ ................................ ........... B - 11 NAEP Technology and Engineering Literacy Standing Committee ................................ ........................... B - 12 NAEP U.S. History Standing Committee ................................ ................................ ................................ .... B - 13 NAEP Writing Standing Committee ................................ ................................ ................................ ............ B - 14 Appendix C Example of Sample Design Document (2013 Assessment) ................................ ................................ . C - 1NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 1 

### A. **JUSTIFICA**

**TION 1a.**  **Circumstances making the collection of information necessary**  The National Assessment of Educational Progress (NAEP) is a federally authorized survey

of student achievement at grades 4, 8, and 12 1 in various subject areas, such as mathematics, reading, writing, science, U.S. history , civics, geography, economics, technology and engineering literacy (TEL) , and the arts. In the current legislation that reauthorized NAEP , the National Assessment of Educational Progress Authorization Act ( Public Law 107 - 279 Titl e III, section 303 ) , Congress mandate s the collection of national education survey data through a national assessment program. The National Assessment of Educational Progress Authorization Act also requires the assessment to collect data on specified student groups and characteristics , including information organized by race/ethnicity , gender, socio - economic status, disability, and limited English proficiency . It requires fair and accurate presentation of achievement data and permits the collection of background , noncognitive, or descriptive information that is related to academic achievement and aids in fair reporting of results. The intent of the law is to provide representative sample data on student achievement for the nation, the states, and subpopulati ons of students and to monitor progress over time. NAEP is administered by the National Center for Education Statistics (NCES) in the Institute for Education Sciences of the U.S. Department of Education. The National Assessment Governing Board (henceforth referred to as the Governing Board) sets policy for NAEP and determines the content framework for each assessment. As a result of the National Assessment of Educational Progress Authorization Act , the Governing Board has final authority on the appropriateness of all cognitive and noncognitive assessment items. The NAEP assessments are conducted b y an alliance of organizations under contract with the U.S. Department of Education . 2 The se assessments contain different kinds of item s — “cognitive” assessment item s, which measure student knowledge of an academic subject , and “noncognitive” or “background” item s, which gather factual information such as demographic variables, as well as construct - related information, such as courses taken . A cop y of the current statute is included in a ppendix A.

1 As described in section 1b, main NAEP assesses students in grades 4, 8, and 12, and long - term trend NAEP assesses students at ages 9, 13, and 17. 2 The current contract expires on March 26, 2013. A new contract will be awarded prior to that date.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 2 

### 1b. **Overview of NAEP assessments**  The following provides a broad overview of NAEP assessments. Please note that the Governing Board determines NAEP policy and assessment schedule , 3 and future Governing Board decisions may result in changes to some aspects of an assessment (e.g., which subjects are assessed in which years). However, the overall methodology

and assessment process will remain consistent. NAEP consists of two assessment programs: the NAEP long - term trend (LTT) assessment and the main NAEP assessment. The LTT assessments are designed to give information on the changes in academic performance and have been administered for over 40 years . They are administered nationall y every four ye ars (but are not r eported at state or district level) and report student performance at ages 9, 13, and 17 in mathematics and reading. The main NAEP assessment s report current achievement levels and short - term trends in student achievement at grades 4, 8, and 12 for the nation and, for certain assessments, states and select urban districts. These assessments follow subject - area frameworks developed by the Governing Board and use the latest advances in assessment methodology . The subject - area frameworks evolve to match instructional practices. NCES is responsible for developing the cognitive items an d for selecting the final set of items. In addition to reporting overall results of student performance and achievement, NAEP also reports student performance results for various subgroups of students and on various educational factors. Guidance for what i s asked in the questionnaires is set by the Governing Board. NCES is responsible for developing the noncognitive items and for selecting the final set of items . T h e items are designed to (a) provide the information for disaggregating data according to cate gories specified in the legislation, (b) provide contextual information that is subject - specific (e.g. reading, mathematics) and has an impact and known relationship to achievement, and (c) provide policy relevant information specified by the Governing Board. Q uestionnaires are generally administered to students at grades 4, 8, and 12; to teachers at grades 4 and 8 ( and at grade 12 for economics ); and to school administrators at grades 4, 8, and 12. Students with disabilities (SD) and English language learn ers (ELL) worksheets are completed by teachers or

3 The Go verning Board assessment schedule can be found at http://www.nagb.org/naep/assessment - schedule.htm .NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 3

administrators of students identified as SD or as ELL , in order to determine if individual students can meaningfully

participate and, if so, which accommodation ( s ) they should receive . Assessment ty

pes lis ted in this document are described as follows: Operational An assessment whose results will be used for NAEP reporting

purposes. Pilot An assessment that contains a pretest of items and possible assessment

conditions to obtain information regarding clarity, difficulty levels, timing, and feasibility of items and conditions. Special Study A study to investigate topics such as content issues, delivery options,

accommodations, linking to other surveys, or reporting variables. All three ty

pes of activities may

simultaneousl y be in the field during any given data collection effort. Each is described in more detail below: 1) **O perational assessments**  **:** “ Operational ” NAEP administrations, unlike pilot administrations, collect data to publicl y report on the education al achievement of students. The NAEP results are reported in the Nation’s Report Card , which is used by policy makers, state and local educators, principals, teachers, and parents to inform educational policy decis ions. 2) **Pilot testing :**  Pilot testing of cognitive and noncognitive items is carried out in all subject areas. In addition to ensuring that items measure what is intended, the data collected from pilot tests serve as the basis for selecting the most effective items and data collection procedures for the subsequent operational assessments. Pilot testing is a cost - effective means for revising and selecting items prior to an operational data collection because, while fewer numbers of students participate, the items are administered to a nationally representative sample of students and data are gathered about performance that crosses the spectrum of student achievement. It ems that do not work well can be dropped or modified before the operational administration. Fo r example, when pilot data indicate very low response percentages in some response categories, the number of response categories provided can be reduced.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 4 Prior to pilot testing, many new items are pre - tested with small groups of sample participants (cleared under the NCES pretesting generic clearance agreement; OMB #1850 - 0803 ) . All noncognitive items undergo one - on - one cognitive interviews, which is useful for identifying questionnaire and procedural problems before larger scale pilot testing is undertaken. Select cognitive items also undergo pre -pilot testing, such as item tryouts or cognitive interviews, in order to test out new item ty pes or formats or challenging content. 3) **Special studies :**  Special studies are an opportunity for NAEP to investigate particular aspects without impacting the reporting of NAEP results. Previous special studies have focused on linking NAEP to other assessments or linking across NAEP frameworks, investigating alternative sets of items for particular samples, evaluating specific accommodations, investigating administration modes (such as computer- delivered assessment alternatives), and providing targeted data on specific student populations . In addition to the overarching goal of NAEP to provide data about student achievement at the national, state, and district levels, NAEP also provides speciall y targeted data on an as needed basis. At times, this may onl y mean that a special anal ys is of the existing data is necessary . At other times the targeting may include the additio n of a short add - on questionnaire targeted at specified groups. For example, in the past, additional student, teacher, and school questionnaires were developed and administered as part of the National Indian Education Study

(NIES) that NCES conducted on behalf of the Office of Indian Education. Through such targeted questionnaires, important information about the achievement of a specific group is gathered at minimal additional burden. These typ es of special studies are intentionally

kept to a minimum and are designed to avoid jeopardizing the main purpose of the program. 

### 1c. **Overview of 2014** **– 2016**  **NAEP assessments**  The basic methodology

and procedures described under this sy stem clearance will be employed for the 2014, 2015, and 2016 NAEP assessments. Subsequent clearance packages submitted for the specific assessment years will describe the scope of the assessment, including additional studies, school and student samples, estimated burden, and items requiring clearance. The 2014 data collection consists of the following:NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 5

 Operational national assessments for U.S. history , civics, and geography at grades 4, 8, and 12 ;  Operational national assessment for TEL 4 at grade 8 ;  Pilot assessments for 201 5 science , including interactive computer tasks ( ICT s) 4 and hands - on tasks ( HOTs ) , at grades 4, 8, and 12 ; and  Pilot assessments for 201 5 reading and mathematics at grade 12 . The 2015 data collection consists of the following:  Operational n ational and state assessment s in reading, math ematics , and science at grades 4 and 8 ;  Operational national , plus 15 states , assessments in reading and mathematics at grade 12;  Operational n ational assessments in science ICT s 4 and HOT s at grades 4, 8 and 12 ;  Operational Puerto Rico mathematics assessment at grades 4 and 8; and  Pilot assessments for 2017 reading, mathematics , and writing4 at grades 4, 8 , and 12 . The 2016 data collection consis ts of the following:  Operational national arts assessment at grade 8;  Operational national LTT assessments in reading and mathematics for ages 9, 13, and 17;  Pilot assessments for 2020 LTT reading and mathematics for ages 9, 13, and 17;  Pilot assessments for 201 8 U.S. history , geography , and civics for grades 4, 8, and 12 ; and  Pilot assessment for 2018 economics at grade 12 . Special Studies Special studies are conducted in accordance with the assessment development, research, or additional reporting needs of NAEP. For example , over recent assessment cycles the following special studies were conducted:  NAEP

–Trends in International Mathematics and Science Study **(** TIMSS ) Li nking Study ;  Reading Accessible Blocks Study ;

4 TEL, Science ICT, and w riting will be administered by computer. All other assessments are c urrently planned as a paper- and pencil administration.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 6

 Knowledge and Skills Appropriate (KaSA) Study (Mathematics) ;  NAEP

–Lexile © Study (Reading) ;  NAEP

–Program for International Student Assessment ( PI SA) Linking Study (Mathematics) ; and  NAEP

–High School Longitudinal Study (HSLS) Study . Over the course of the 2014

–2016 NAEP assessments, special studie s will be conducted. The specifics of future studies will be included in the subsequent OMB submittals. Though an exhaustive list of special studies cannot be provided given the real - time nature of these projects, two special studies are currently being co nsidered :  High School Transcript Study (HSTS)

– The NAEP HSTS, planned for 2015, periodicall y surveys the curricula being followed in our nation’s high schools and the course - taking patterns of high school students through a collection of transcripts. The HSTS information is provided by a sample of the public and private schools and the sample of students is representative of the U.S. population of graduating seniors . The transcript study includes onl y those students whose transcripts indicate that they graduated the year that the study was conducted. Most of the students sampled in the transcript study are in schools that participated in NAEP. The data collected from those students that participated in NAEP make it possible to link course - taking patterns to academic performance, as measured b y NAEP.  National Indian Education Survey (NIES)

– NIES is planned as part of the 2015 assessments in reading and mathematics. The national sample includes students from both public and nonpublic schools that have both large and small American Indian/Alaska Native (AI/AN) student populations. The administration of the NAEP assessment will be followed with the administration of a questionnaire specifically designed for the NIES study . Questionnaire data will be linked to NAEP performance data. In addition, NAEP is in the process of transi tioning assessments from paper- and -pencil to computer delivered assessments . Therefore , the NAEP program will most likely undertake some special studies to explore aspects associated with th e transition to computer- delivered assessments, such as item ty pes, hardware, s ys tem performance , and comparability across assessment modes.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 7 

### 1d. **Rationale for OMB System Clearance** NCES is requesting s ys tem clearance for the NAEP assessments to be administered in the 2014

–2016 timeframe, similar to the system clearance approval that was granted for the 2011

–2013 and 2008

–2010 NAEP administrations (OMB 1850 - 0790). The primary reason for the s ys tem clearance request is that it enables NAEP to meet its large and complex assessment reporting schedules and deliverables through a more efficient clearance process. Since the pas sage of the No Child L eft Behind Act in 2001 (P.L . 107 - 110), the numbers of assessments, volumes of participants, and complexity of the development and production processes have all increased dramaticall y. Because of the National Assessment of Educational Progress Authorization Act requirements, every

state participates in the reading and mathematics assessments at grades 4 and 8 and most states participate in the state - level writing and science assessments. These state - level assessments require that sample s be selected to permit reporting of results for each state as well as for the nation, resulting in a much larger sample size than would otherwise be necessary . In addition, because of the complex matrix sampling plan used by NAEP, upwards of 1,000 distinc t assessment booklets or forms may need to be printed or assembled for one assessment year alone. Given the cyclical nature of the NAEP assessments and their complexity and scope, there is a relatively short and fixed time period in which production can o ccur. Prior to the finalization of the content, many activities must be conducted, including quality reviews, anal ys is of any prior data, assembl y of forms, and approval b y the Governing Board and OMB. To meet these demanding timelines, NCES is requesting a waiver of the 60 - day

federal register notice for clearances of the NAEP assessments and studies described in this system clearance package. Parts A and B of this sy stem clearance package contain supporting information for the assessments to be given in 2014

–2016 . This submittal does not contain any volume II (specific questionnaire) material. Specific yearl y questionnaires will be submitted as they are developed over the course of the s ys tem clearance period. Clearance of three ye ars duration (beginning April 1 , 2013 ) is requested for the 2014

– 2016 assessments. At the end of this sy stem clearance period (i.e., prior to March 31 , 2016 ), NCES will submit a new request for sy stem clearance to cover future NAEP administrations.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 8 

### 2. **How , by w**  **hom, and for w hat p**  **urpose the data will be used** Under this request for s ystem clearance, NCES is asking for approval of the planned assessments, including the various questionnaires (e.g., student, teacher, and school) and data collection efforts that will be part of the NAEP 2014

–2016 national, state, and district assessments, as well as any special studies that are planned to be conducted in the 2014

–2016 timeframe. Results will be reported on the 2014 operational assessments in U.S. history , civics, geography, and TEL ; the 2015 operational assessments in reading , mathematics , and science , including ICT s and HOT s ; and the 2016 operational arts assessment and long - term trend assessment s in reading and mathematics. The schedule for subjects to be assessed has been establi shed by the Governing Board; however, it is subject to modification. Therefore, NCES is requesting some leeway with regard to the specific subject assessments that will be administered, while holding the methodology and general procedures constant. NAEP u ses the results from the pilot tests and some special studies to inform future assessments and procedures. Item s are evaluated for their effectiveness and appropriateness to be used in an operational assessment. Results from the operational assessments and some special studies are released to the public. The NAEP results are reported in the Nation’s Report Card, which is used by policy makers, state and local educators, principals, teachers, and parents to help inf orm educational policy

decisions . The NAEP Report Cards provide national results, trends for different student groups, results on scale scores and achievement levels, and sample item s. In reports with state or urban district results, there are sections that provide overview information on the performance of these jurisdictions. NAEP does not provide scores for individual students or schools. In addition to the Report Card , more information is available online (http://nationsreportcard.gov/) and in one -page summary

reports , called snapshots , for each participating state or urban district. Additional data tools are available online for those interested in:  anal yz ing NAEP data and creating tables and graphics (http://nces.ed.gov/nationsreportcard/naepdata/);  comparing state performance b y various demographic groups (http://nces.ed.gov/n ationsreportcard/statecomparisons/);  comparing districts (http://nces.ed.gov/nationsreportcard/districts/) ;NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 9

 seeing the knowledge and skills demonstrated by students performing at dif ferent scale scores (http://nces.ed.gov/nationsreportcard/itemmaps/);  seeing NAEP performance results and student demographics for each state (http://nces.ed.gov/nationsreportcard/states/); and  searching, sorting, and providing data for sample NAEP item s (http://nces.ed.gov/nationsreportcard/itmrlsx/). In addition to contributing to the reporting tools mentioned above, data from the questionnaires are used as part of the marginal estimation procedures that produce the student achievement results. In addition, questionnaire data is used to perform quality control checks on school - reported data and in special reports, such as the Black

– White Achievement Gap report. Finally , there are numerous opportunities for secondary data anal ys is because of NAEP ’ s large scale, the regularity of its administrations, and its stringent quality control processes for data collection and anal ys is. NAEP data are used by researchers and educators who have diverse interests and varying levels of anal ytical experience. 

### 3. **Use of techniques to reduce burden** For all operational and pilot assessments, NAEP will continue to take advantage of proven, modern measurement techniques, which greatl y enhance the power and value of the NAEP data collected. Through the use of a partial balanced incomplete block (BIB) spiraling variant of matrix sampling, a variety of anal ys es are feasible because the data are not booklet bound. Covariances are computed among all item s in a subject area, so that  composites of item s can be appraised empiricall y for coherence and construct validity ;  the dimensional structure of each subject area can be determined anal ytically as reflected in student performance consistencies;  item response theory

(IRT) scaling can be applied to unidimensional sets of items or tasks, regardless of which booklet they appear in;  IRT scales can be developed having common meaning across exercises, population subgroups, age levels, and time periods;  powerful trend anal ys es can be undertaken b y m eans of these common scales;NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 10

 performance scales can be correlated with background, attitudinal, and program variables to address a rich variety

of educational and policy issues; and  public - use electronic files can be made much more useful because secondary anal ys es are also not booklet - bound. To minimize the burden for participating scho ols and students, the following procedures will be used:  No student takes the complete assessment and, given that NAEP results are reported for groups of students, using a BIB spiral allows the program to administer a lengthy overall assessment without overburdening any one student.  Trained administrators will conduct the assessments at all grades.  Assessment administrations will be limited, whenever possible, to approximately 90 minutes to facilitate school scheduling (note, computer- delivered assessments may be scheduled for up to 120 minutes). The various questionnaires (student, teacher, and school) are constructed to minimize respondent burden using the information gathered from pilot testing. Based on pilot test data, items that do not yield useful information or that produce redundant information can be eliminated. Also, to reduce respondent burden, NAEP may

employ some or all of the following techniques:  maintaining items across administrations rather than piloting new ones;  rotating some non - requir ed items across assessments;  spiraling items across respondents to provide a broader range of content coverage while minimizing individual respondent burden; and  reducing the number of new item s piloted for existing frameworks. Another technique for improving the response process consists of pre - testing items using cognitive interviews, item tryouts, and usability studies on a smaller number of students to maximize the success of items. These techniques allow items to be refined before they go to larg er groups of students. 

### 4. **Efforts to identify duplication** The proposed assessments, including the questionnaires, do not exist in the same format or combination in the U.S. Department of Education or elsewhere. The noncognitive data gathered b y NAEP comprise theNAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 11

only comprehensive cross - sectional survey performed regularl y on a large - scale basis that can be related to extensive achievement data in the United States . No other federall y funded studies have been designed to collect data for the purpose of regularl y assessing trends in educational progress and comparing these trends across states . None of the major non - federal studies of educational achievement were designed to measure changes in national achievement. In short, no existing data source in the pu blic or private sector duplicates NAEP. 

### 5. **Burden on small businesses or other small entities** The school samples for NAEP contain small - , medium - , and large - size schools, including private schools . Schools are included in the sample proportional to their representation in the population , or as necessary to meet reporting goals . It is necessary to include small and private schools so that the students attending such schools are represented in the data collection and in the reports . 

### 6. **Consequences of collec ting information less frequently**  Under the National Assessment of Educational Progress Authorization Act , Congress has mandated the on - going collection of NAEP data. Failure to collect the 2014

–2016 assessment data on the current schedule would affect the quality and schedule of the NAEP assessments, and would result in assessments that would not fulfill the mandate of the legislation. 

### 7. **Consistency with 5 CFR 1320.5** No special circumstances are involved. This data collection observes all requirements of 5 CFR 1320.5. 

### 8. **Consultations outside the agency** In addition to the contractor s responsible for the development and administration of the NAEP assessments, the program involves many consultants and is also reviewed by specialists serving on various technical review panels . These consultants and special reviewers bring expertise concerning students of different ages, ethnic backgrounds, geographic regions, learning abilities, and socioeconomic levels ; the specific subject areas being assessed ; the anal ys is methodologies employed; and large - scale assessment design and practices . Contractor staff and consultants have reviewed all item s for bias and sensitivity

issues, grade appropriateness, and appropri ateness of content across states .NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 12

In particular , subject area standing committees play a central role in the development of NAEP assessment instruments and have been essential in helping create assessment content that is appropriate for the targeted populations, and that meets the expectations outlined in the Governing Board frameworks. One of the most important functions of the committee s is to contribute to the validation of the assessments. Through detailed reviews of items, scoring guides, tasks, constructed- respo ns e item training sets for scorers , and other materials, the committees help establish that the assessments are accurate, accessible, fair, relevant, and grade - level appropriate and that each item measures the knowledge and skills it was design ed to measure. When appropriate, members of subject area standing committees will also review the questionnaires with regards to appropriateness with existing curricular and instructional practices. The following examples of outside personnel , based on current membership, are provided in a ppendix B:  NAEP Design and Anal ysis Committee  NAEP Validity

Studies Panel  NAEP Quality

Assurance Technical Panel  NAEP Socio - E conomic Status Panel  NAEP National Indian Education Study Technical Review Panel  NAEP Civics Standing Committee  NAEP Economics Standing Committee  NAEP Geography Standing Committee  NAEP Mathematics Standing Committee  NAEP Reading Standing Committee  NAEP Sci ence Standing Committee  NAEP Technology

and Engineering Literacy Standing Committee  NAEP U.S. History

Standing Committee  NAEP Writing Standing Committee As has been the practice for the past few ye ars, OMB representatives will be invited to attend the technical review panel meetings that are most informative for OMB purposes.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 13 

### 9. **Payments or gifts**  **to respondents**  In general, there will be no gifts or pay ments to respondents, although students do get to keep the NAEP pencils or earbuds used in the paper - and -pencil and computer- delivered assessment s, respectively . On occasion, NAEP will leave educational materials behind at schools for their use ( e . g ., science kits from the science HOTs assessments). Schools participating in the High School Transcript Study are paid the established fee for providing student transcripts. Given that the study pays schools the prevailing rate to perform a standard service, estimates of school - level burden for that function are not included in this volume. Special studies (i.e., cognitive lab studies and background item studies) sometimes include remuneration for respondents and will be explicated in the specific - ye ar clearance package submittals. Some schools also offer recognition parties with pizza or other perks for stu dents who participate ; however , these are not reimbursed by NCES or the contractor. If any incentives are proposed as part of a future special study , they would be justified as part of that future clearance package. In addition, as appropriate, the amounts would be consistent with amounts approved in other studies with similar conditions. 

### 10. **Assurance of confidentiality**  NAEP has policies and procedures that ensure privacy , security , and confidentiality , in compliance with the legislation (Confidential Information Protection provisions of Title V, Subtitle A, Public Law 107 - 347 and the National Assessment of Educational Progress Authorization Act). Specificall y for the NAEP project, this ensures that privacy , security , and confidentiality policies and procedures are in compliance with the Privacy

Act of 1974 and its amendments, NCES Confidentiality Procedures, and the Department of Education ADP Security manual. The National Assessment of Educational Progress Authorization Act requires the confidentiality of personall y identifiable information [ 20 U.S.C. §9622 (c) (3) ] : (A) IN GENERAL. -- The Commissioner for Education Stati stics shall ensure that all personall y identifiable information about students, their academic achievement, and their families, and that information with respect to individual schools, remains confidential, in accordance with section 552a of title 5 , Unite d States Code . (B) PROHIBIT ION. -- The Assessment Board, the Commissioner for Education Statistics, and any contractor or subcontractor shall not maintain any s ys tem of records containing a student ’ s name, birth information, Social Security number, or parents ’ name or names, or any other personall y identifiable information.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 14

Each contractor develops a Data Security Plan and NCES ensures that all current contractor policies and procedures are in compliance with all NAEP security and confidentiality

requirement s. In addition, all NAEP contractor staff with access to confidential NAEP information are required to sign an affidavit of nondisclosure that affirms, under severe penalty for unlawful action, that they will protect NAEP information from non - authorized ac cess or disclosure. The affidavits are in keeping with the NCES Standard for Maintaining Confidentiality (Standard 4 - 2). All contractors must also comply with directive OM: 5 - 101, which requires that all staff with access to data protected b y the Privacy A ct and/or access to U.S. Department of Education sy stems and who will work on the contract for 30 day s or more go through the security screening procedures. An important privacy

and confidentiality

issue is the protection of the identity of assessed students, their teachers, and their schools. To assure this protection, NAEP has established security procedures, described below, that closely control access to potentially identifying information. Students’ names are submitted to the Sampling and Data Collection contractor for selecting the student sample. This list also includes the month/year of birth, race/ethnicity , gender, and a status code for students with disabilities , English language learners , and participation in the National School Lunch Program. The student sample is selected and the data for selected students are submitted to the Materials Preparation, Distribution, Processing and Scoring (MDPS) contractor, who includes the data in the P ackaging and D istribution sy stem for the production of the Administration Schedule and student labels, which are then forwarded to field staff and used to manage and facilitate the assessment. These data are also added to the School Control System (SCS) used by field staff to p rint materials used by the schools. Student information is deleted from the Packaging and Distribution sy stem after the assessment begins. All labels are removed from the assessment materials and destro yed at the schools, upon completion of the assessment. The section of the Administration Schedule with names is removed and placed in the school storage envelope . The school storage envelope contains all of the forms and materials with student names. It is kept at the school until the end of the school year and then destroyed b y s chool personnel. In addition to student information, teacher and principal names are collected and recorded on a roster, which is used to keep track of the distribution and collection of NAEP teacher and school questionnaires. The roster is kept at the school until questionnaires are collected. At that time, questionnaires and the portion of the roster without names is returned for processing. The portion of the roster with names is kept at the school, in the school storage envelope , which is destroyed at the end of the school year.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 15

The completed Administration Schedules and teacher and school questionnaire rosters (without student, teacher, or principal names) are returned to the MDPS contractor for processing. Furthermore, to ensure the confidentiality of respondents, NAEP staff will use the following precautions:  Data files will not identify individual respondents.  No personall y identifiable information, either b y s chools or respondents, will be gathered or released b y third parties. No permanent files of names or other direct identifiers of respondents will be maintained.  Student participation is voluntary .  NAEP data are perturbed. Data perturbation is a statistical data editing technique implemented to ensure privacy

for student and school respondents to NAEP’s assessment questionnaires for assessments in which data are reported or attainable via restricted - use licensing arrangements with NCES . The process is coordinated in strict confidence with the IES Disclosure Review Board (DRB) , with details of the process shared onl y with the DRB and a minimal number of contractor staff. After the components of NAEP are completed in a school, neither student- nor teacher- reported data are retrievable b y personal identifiers. We emphasize that confidentiality is assured for individual schools and for individual students, teachers, and principals. The following text is printed on all student, teacher, and school questionnaires: The information you provide will be used for statistical purpose s only. In accordance with the Confidential Information Protection provisions of Title V, Subtitle A, Public Law 107 - 347 and other applicable Federal laws, your responses will be kept confidential and will not be disclosed in identifiable form to anyone ot her than employees or agents. By law, every NCES employee as well as every agent, such as contractors and NAEP coordinators, has taken an oath and is subject to a jail term of up to 5 years, a fine of up to $250,000, or both if he or she willfully disclose s ANY identifiable information about you. For the HSTS component of NAEP, student transcripts are collected from schools for sampled students, and school staff members complete a School Information Form that provides general information about class periods , credits, graduation requirements, and other aspects of school policy . To maintain the privacy of student and school identities, students’ names are removed from the transcripts and questionnaires at the school and given a unique identification number, which is used to match the transcript records to the NAEP questionnaire and performance information, on an individual basis. NCES ensures that the data collected from schools and students are used for statistical purposes onl y.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 16 

### 11. **Sensitive questions** The National Assessment of Educational Progress emphasizes voluntary respondent participation, assures confidentiality

of individual responses, and avoids asking for information that might be considered sensitive or offensive. In sensitive or offensive item s are prohibited b y the National Assessment of Educational Progress Authorization Act , and the Governing Board reviews all items for bias and sensitivity . Throughout the item development process, NCES staff works with consultants, contractors, and internal reviewers to identify and eliminate potential bias in the item s. 

### 12. **Estimation of respondent reporting burden ( 2014** **– 2016**  **)** Because there are such fluctuations in the burden numbers over the three years of this collection , we will continue our practice o f showing the estimate of each ye ar ’ s burden rather than averaging the total over the three ye ars. The burden numbers and respondents will be changed accordingl y for each ye ar, based on the final list of assessments and special studies to be assessed. Exhibit 1 provides the burden information per respondent group, b y grade and b y ye ar , for the 2014

–2016 assessments . Exhibit 2 summarizes the burden across the three years. A description of the respondents or study is provided below, as supporting information for Exhibit 1 :  Students - Students in fourth , eighth , and twelfth grades complete assessment booklets or forms that contain 50 or 60 minutes of cognitive blocks, followe d by noncognitive block(s ) which require a total of 15 minutes to complete. The core noncognitive items are answered b y all students across subject areas and are related to demographic information. In addition, students answer subject - specific noncognitive items . Additional student burden accounts for time to read directions, distribute test booklets (for paper- and -pencil assessments), and log on to the computer and view a tutorial (for computer- delivered assessments). This additional burden is estimated at 10 minutes for paper - and -pencil and 15 minutes for computer- delivered assessments. Therefore, the total burden for students is 25 minutes for paper- and -pencil and 30 minutes for computer delivered assessments.  Teachers - The teachers of fourth - and eighth - grade students (and twelfth - grade economics students) participating in NAEP are asked to complete questionnaires about their teaching background, education, training, and classroom organization. Fourth - grade teacher burden is estimated to be 30 minutes be cause fourth - grade teachers often have multiple subject - specificNAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 17

sections to complete. Eighth - and twelfth - grade teacher burden is 20 minutes because they onl y have one subject - specific section .  Principals/Administrators - The school administrators in the sampled schools are asked to complete a questionnaire . The core item s are designed to measure school characteristics and policies that research has shown are highl y correlated with student achievement. A section with sub ject - specific item s concentrates on curriculum and instructional services issues. The burden for school administrators is estimated at 30 minutes.  SD and ELL worksheets are completed by school personnel concerning students identified as SD or ELL . These w orksheets will be used to determine the appropriate accommodations for students.  Submission of Samples - Survey sample information is collected from schools in the form of lists of potential students who may participate in NAEP. This sample information can be gathered manually or electronically

at the school, district, or state level. If done at the school or district level, some burden will be incurred b y school personnel. It is estimated that it will take one hour for school personnel to complete the subm ission process. Based on recent comparable experience, 64 percent , 48.7 percent , and 70.4 percent of the schools or districts will complete the sample submission process for 2014, 2015, and 2016, respectivel y. Note that the estimated percent is dependent u

pon the nature of the sample (i.e., national or state) and the assessment program (i.e., inclusion of LTT).  Pre - Assessment Visit - The pre - assessment visit is the opportunity for the NCES contractor field staff to meet with the school personnel to review procedures and logistics for the upcoming assessment. It is estimated that it will take two hour s for school personnel to complete the p re a ssessment v isit.  HSTS - The NAEP HSTS periodically survey s the curricula being followed in our nation ’ s high schools and the course - taking patterns of high school students through a collection of transcripts. This data collection requires three hours per school from a sample of approximately 800 schools.  National Indian Education Study (NIES) - NIES is designed to describe the condition of education for American Indian and Alaska Native (AI/AN) students in the United States. Additional questionnaires designed for NIES are given to student s (estimated at 15 minutes ), t eachers (20 minutes) , and school administrato rs (30 minutes).NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 18 

### EXHIBIT 1 **Estimated Burden for NAEP 2014 –** **2016 Assessments ,**  **By Year, By Grade Level** (Note: all explanatory notes and footnotes are displayed following the 2016 table) **2014 Subjects**  **# of Students Avg. time per**  **respondent**

**3 Student**  **Burden (in hours)**  **# of**

**Teachers Avg. time per** 

**respondent Teacher** 

**Burden (in hours)**  **# of**

**Schools Avg. time per** 

**respondent School** 

**Burden (in hours)**  **SD/ELL**

**(# of school personnel)**  **Avg. time per**

**respondent SD/ELL** 

**Burden 4**  **(in hours)** Operational (US history, civics, geography); Pilot (science P&P, science ICT, science HOTs) 21,000 25 minutes for P&P; 30 minutes for CDA 8,875 2,180 30 minutes 1,090 545 30 minutes 273 545 10 minutes 805

Special Studies 1 5,000 25 minutes for P&P; 30 minutes for CDA 2,292 512 30 minutes 256 128 30 minutes 64 128 10 minutes 192
Pre-assessment visit 673 2 hours 1,346 Sample submission

2 431 1 hour 431 **4th Grade Totals**  **26,000 N/A**  **11,167 2,692**  **N/A 1,346**  **673 N/A**  **2,114 673**  **N/A 997**  Operational (US history, civics, geography, TEL); Pilot (science P&P, science ICT, science HOTs) 47,000 25 minutes for P&P; 30 minutes for CDA 21,375 7,710 20 minutes 2,570 1,285 30 minutes 643 1,285 10 minutes 1,410

Special Studies 1 5,000 25 minutes for P&P; 30 minutes for CDA 2,292 833 20 minutes 278 139 30 minutes 69 139 10 minutes 150
Pre-assessment visit 1,424 2 hours 2,848 Sample submission

2 911 1 hour 911 **8th Grade Totals**  **52,000 N/A**  **23,667 8,543**  **N/A 2,848**  **1,424 N/A**  **4,471 1,424**  **N/A 1,560**  Operational (US history, civics, geography); Pilot (reading, math, science P&P, science ICT, science HOTs) 34,000 25 minutes for P&P; 30 minutes for CDA 14,292 531 30 minutes 266 531 10 minutes 793

Special Studies 1 5,000 25 minutes for P&P; 30 minutes for CDA 2,292 78 30 minutes 39 78 10 minutes 117
Pre-assessment visit 609 2 hours 1,218 Sample submission

2 390 1 hour 390 **12th Grade Totals**  **39,000 N/A**  **16,584 N/A**  **N/A N/A**  **609 N/A**  **1,913 609**  **N/A 910**  **Total 117,000**  **N/A 51,418**  **11,235 N/A**  **4,194 2,706**  **N/A 8,498**  **2,706 N/A**  **3,467**

**Total number of respondents 133,647**  **Total Burden (in hours) 67,577**  **4th Grade**

**8th Grade**

**12th Grade**NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 19 **2015 Subjects**  **# of Students Avg. time per**  **respondent**

**3 Student**  **Burden (in hours)**  **# of**

**Teachers Avg. time per** 

**respondent Teacher** 

**Burden (in hours)**  **# of**

**Schools Avg. time per** 

**respondent School** 

**Burden (in hours)**  **SD/ELL**

**(# of school personnel)**  **Avg. time per**

**respondent SD/ELL** 

**Burden 4**  **(in hours)** Operational (math, reading, science P&P, science ICT, science HOTs, Puerto Rico math); Pilot (reading, math, writing) 506,000 25 minutes for P&P; 30 minutes for CDA 212,083 41,640 30 minutes 20,820 10,410 30 minutes 5,205 10,410 10 minutes 19,397

Special Studies 1 10,000 25 minutes for P&P; 30 minutes for CDA 4,583 832 30 minutes 416 208 30 minutes 104 208 10 minutes 383
NIES 10,000 15 minutes 2,500 8,000 20 minutes 2,667 2,000 30 minutes 1,000 Pre-assessment visit 10,618 2 hours 21,236 Sample submission

2 5,171 1 hour 5,171 **4th Grade Totals**  516,000 **N/A** 219,166 42,472 **N/A** 23,903 10,618 **N/A** 32,716 10,618 **N/A** 19,780 Operational (math, reading, science P&P, science ICT, science HOT, Puerto Rico math); Pilot (reading, math, writing) 498,500 25 minutes for P&P; 30 minutes for CDA 208,333 47,100 20 minutes 15,700 7,850 30 minutes 3,925 7,850 10 minutes 14,955

Special Studies 1 10,000 25 minutes for P&P; 30 minutes for CDA 4,583 954 20 minutes 318 159 30 minutes 80 159 10 minutes 300
NIES 10,000 15 minutes 2,500 8,000 20 minutes 2,667 2,000 30 minutes 1,000 Pre-assessment visit 8,009 2 hours 16,018 Sample submission

2 3,900 1 hour 3,900 **8th Grade Totals**  508,500 **N/A** 215,416 48,054 **N/A** 18,685 8,009 **N/A** 24,923 8,009 **N/A** 15,255 Operational (math, reading, science P&P, science ICT, science HOTs); Pilot (reading, math, writing) 140,700 25 minutes for P&P; 30 minutes for CDA 59,333 2,325 30 minutes 1,163 2,325 10 minutes 3,283

Special Studies 1 10,000 25 minutes for P&P; 30 minutes for CDA 4,583 167 30 minutes 84 167 10 minutes 233
HSTS 800 3 hours 2,400 Pre-assessment visit 2,492 2 hours 4,984 Sample submission

2 1,214 1 hour 1,214 **12th Grade Totals**  150,700 **N/A** 63,916 **N/A N/A**  **N/A** 2,492 **N/A** 9,845 2,492 **N/A** 3,516 **Total 1,175,200**  **N/A 498,498**  **90,526 N/A**  **42,588 21,119**  **N/A 67,484**  **21,119 N/A**  **38,551**

**Total number of respondents Total Burden (in hours)**  **647,121 1,307,964**  **4th Grade**

**8th Grade**

**12th Grade**NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 20 **201 6**  **Subjects # of Students**  **Avg. time per respondent** 

**3 Student**  **Burden (in hours)**  **# of**

**Teachers Avg. time per** 

**respondent Teacher** 

**Burden (in hours)**  **# of**

**Schools Avg. time per** 

**respondent School** 

**Burden (in hours)**  **SD/ELL**

**(# of school personnel)**  **Avg. time per**

**respondent SD/ELL** 

**Burden 4**  **(in hours)** Age 9: Operational and Pilot (reading and math) 14,700 25 minutes 6,125 420 420 10 minutes 564 Age 13: Operational and Pilot (reading and math) 14,700 25 minutes 6,125 460 460 10 minutes 441 Age 17: Operational and Pilot (reading and math) 14,700 25 minutes 6,125 530 530 10 minutes 343 Pre-assessment visit 1,410 2 hours 2,820 Sample submission 2 993 1 hour 993 **LTT totals 44,100**  **N/A 18,375**  **N/A N/A**  **N/A 1,410**  **N/A 3,813**  **1,410 N/A**  **1,348** Pilot (US history, geography, civics) 1,500 25 minutes 625 140 30 minutes 70 35 30 minutes 18 35 10 minutes 58 Special Studies 1 5,000 25 minutes for P&P; 30 minutes for CDA 2,292 464 30 minutes 232 116 30 minutes 58 116 10 minutes 192 Pre-assessment visit 151 2 hours 302 Sample submission 2 106 1 hour 106 **4th Grade Totals**  **6,500 N/A**  **2,917 604**  **N/A 302**  **151 N/A**  **484 151**  **N/A 250**  Operational (arts); Pilot (US history, geography, civics) 12,500 25 minutes 5,208 2,136 20 minutes 712 356 30 minutes 178 356 10 minutes 375 Special Studies 1 5,000 25 minutes for P&P; 30 minutes for CDA 2,292 858 20 minutes 286 143 30 minutes 72 143 10 minutes 150 Pre-assessment visit 499 2 hours 998 Sample submission 2 351 1 hour 351 **8th Grade Totals**  **17,500 N/A**  **7,500 2,994**  **N/A 998**  **499 N/A**  **1,599 499**  **N/A 525**  Pilot (US history, geography, civics, economics) 6,000 25 minutes 2,500 80 30 minutes 40 80 10 minutes 140 Special Studies 1 5,000 25 minutes for P&P; 30 minutes for CDA 2,292 67 30 minutes 34 67 10 minutes 117 Pre-assessment visit 147 2 hours 294 Sample submission 2 103 1 hour 103 **12th Grade Totals**  **11,000 N/A**  **4,792 N/A**  **N/A N/A**  **147 N/A**  **471 147**  **N/A 257**  **Total 79,100**  **N/A 33,584**  **3,598 N/A**  **1,300 2,207**  **N/A 6,367**  **2,207 N/A**  **2,380**

**Total number of respondents 87,112**  **Total Burden (in hours) 43,631**  **LTT 5** 

**4th Grade**

**8th Grade**

**12th Grade**NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 21 Notes for all tables in Exhibit 1 Cells with cross - hashes are sub - sets or repeats of previous cells. Therefore, they are shown for row calculation purposes only and are not included in the column totals. Light gray s haded cells ar e intentionally left blank because no burden is associated with that cell. 1 - Because the specific special studies are not known at this time, a placeholder is included for burden calculation purposes. 2 - Based on most recent comparable data, it is estimated that 64% of district or school personnel will perform sample selection activities in 2014, 48.7% in 2015, and 70.4% in 2016. 3 - P&P refers to paper - and -pencil assessments and CDA refers to computer - delivered assessments. All special study placeholder s assume that half will be P&P and half will be CDA. 4 - SD and ELL burden is calculated as the number of hours that school personnel spend to complete the SD/ELL worksheets for students identified as SD and/or ELL (estimated at 10 minutes per student). Th e overall SD/ELL burden is a factor of the number of students identified as SD/ELL. Based on most recent data, it is estimated that 23 percent of grade 4/age 9 students, 18 percent of grade 8/age 13, and 14 percent of grade 12/age 17 students will be identified as SD/ELL. 5 - Teacher and school questionnaires are not administered in association with the LTT assessments. 

### EXHIBIT **2 Total**  **Estimated Burden for NAEP 2014 –** **2016 Assessments Total number of respondents**  **Total number of responses Total Burden (in hours)**  **2014** 133,647 138,085 67,577 **2015** 1,307,964 1,380,168 647,121 **2016** 87,112 90,872 43,631 **Total 1,528,723**  **1,609,126 758,329**  

### 13. **Cost to respondents** There are no direct costs to respondents. 

### 14. **Estimates of cost to the federal government** The total cost to the federal government for the administrations of the 2014

–2016 activities is estimated to be approximately

$ 48.15 million . The cost estimate is broken down as follows:  $ 13.25 million for the printing, packaging, and distribution phases of the administrations;  $ 27.1 million for the cost for the field supervisors and data collectors to go into schools to administer the 2014

–2016 assessment s , including travel expenses and testing equipment costs ; and  $ 7.8 million for web operations and maintenance costs related to the support of the computer - delivered assessments.NAEP 2014 - 2016 System Clearance: Supporting Statement Part A page - 22 

### 15. **Reasons for changes in burden (from last System Clearance submittal)** The nature of NAEP is that burden alternates from a relativel y low burden in national - level administration ye ars (i.e., even ye ars) to a substantial burden increase in state - level administration years that include one or more assessments that support national, state - by - state , an d certain urban districts reporting (i.e., odd ye ars) . In state/district assessment years, NAEP samples approximate ly 1,000,000 students, while in national - only

assessment years, approximate ly 100,000 students . In 2015 , NAEP will conduct state/district ass essments, and in 2014 and 2016, national - level assessments . The previous S ys tem Clearance included burden for two state/district assessments (2011 and 2013) and onl y one national - level assessment (2012) , therefore the overall number of respondents and responses is lower in this clearance request than in the previous one. However, because we have adjusted the estimated respondent time for students and the pre - assessment visit , the overall time burden estimate is slightl y higher for this versus the previ ous Sy stem Clearance . Previousl y, only the time spent responding to the noncognitive questionnaires was included in the total burden calculation for the students , but now we also include the time for instructions and booklet distribution , which increases the per - student burden estimate b y an additional 10 minutes for paper - and -pencil assessments and by 15 minutes for computer- delivered assessments. Finally , recent reports from the field staff have indicated that the pre - assessment visits require two hours, rather than the one hour previously estimated. Therefore, we have adjusted the burden estimate for pre assessment visit to two hours. 

### 16. **Time schedule for data collection** The time schedule for the data collection for the 2014

–2016 assessments is shown bel ow. **2014** Main NAEP January

–March 2014 **2015** Main NAEP January

–March 2015 **2016** Long - Term Trend Main NAEP October 2015

–May 2016 January

–March 2016 

### 17. **Approval for not displaying**  **OMB approval expiration date**  No exception is requested. 

### 18. **Exceptions to Certification Statement**  No exception is requested.