## OECD

**PROGRAM FOR INTERNAT IONA** **L STUDENT ASSESSMENT (PISA 2015**  **) FIELD TEST AND RECRU**  **ITMENT FOR FIELD**

**TEST AND MAIN STUDY REQUEST FOR OMB CLEA** **RANCE OMB# 1850**  **- 0755**  **v .**  **14** 

## SUPPORTING STATEMENT PART A

**Submitted by:**  **National Center for Education Statistics**  **U.S. Department of Education Institute of Education Sciences**  **Washington, DC June**  **2013**i 

## TA BLE OF CONTENTS PREFACE ................................ ................................ ................................ ................................ ...... ii A. JUSTIFICATION ................................ ................................ ................................ ................... 1 A.1 Importance of Information ................................ ................................ ............................... 1 A.2 Purposes and Uses of Data ................................ ................................ ............................... 2 A.3 Improved Information Technology (Reduction of Burden) ................................ ........ 4 A.4 Efforts to Identify Duplication ................................ ................................ ......................... 5 A.5 Minimizing Burden for Small Institutions ................................ ................................ ....... 6 A.6 Frequency of Data Collection ................................ ................................ ........................... 6 A.7 Special Circumstances ................................ ................................ ................................ ........ 6 A.8 Consultations Outside NCES ................................ ................................ ........................... 6 A.9 Payments or Gifts to Respondents ................................ ................................ ................... 7 A.10 Assurance of Confidentiality ................................ ................................ ............................. 8 A. 11 Sensitive Questions ................................ ................................ ................................ ............. 9 A. 12 Estimates of Burden ................................ ................................ ................................ .......... 9 A.13 Total Annual Cost Burden ................................ ................................ ............................... 11 A.14 Annualized Cost to Federal Government ................................ ................................ ..... 12 A.15 Program Changes or Adjustments ................................ ................................ .................. 12 A.16 Plans for Tabulation and Publication ................................ ................................ ............. 12 A.17 Display OMB Expiration Date ................................ ................................ ....................... 13 A.18 Exceptions to Certification Statement ................................ ................................ ........... 13 B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS B.1 Respondent Universe B.2 Statistical Methodology B.3 Maximizing Response Rates B.4 Purpose of Field test and Data Uses B.5 INDIVIDUALS CONSULTE D ON STUDY DESIGN APPENDIX A: RECRUITMENT MATERIALS APPENDIX B: PARENTAL CONSENT MATERIALS APPENDIX C: INSTRUME NTS APPENDIX D: NON
- RESPONSE BIAS ANALYS IS PLANii **PREFACE** The Program for International Student Assessments (PISA) is an international assessment of 15
- year - olds which focuses on assessing students’ science, mathematics, and reading literacy. PISA was first administe red in 2000 and is conducted every three years. The sixth cycle of PISA, PISA 201 5 , is being administered at a time when interest is increasing, both worldwide and in the United States, in how well schools are preparing students to meet the challenges of the future, and how the students perform compared with their peers in other education systems of the world. The participation in the PISA study among education systems 1 has significantly increased since the initial survey in 2000: 43 education systems in 2000, 41 in 2003, 57 in 2006, 6 5 in 2009, 6 5 in 2012, and approximately 65 are expected to participate in 201 5 . The United States has participated in all of the previous cycles, and will participate in 201 5 in order to track trends and to compare the p erformance of U.S. students with that in other education systems . PISA 201 5 is sponsored by the Organization for Economic Cooperation and Development (OECD). In the United States, PISA 201 5 is conducted by the National Center for Education Statistics (NCE S) of the Institute of Education Sciences, U.S. Department of Education. PISA is a collaboration among the participating countries, the OECD, and a group of international organizations each under contract to the OECD (hereafter referred to as the PISA Inte rnational Consortium ), including the Educational Testing Service (ETS), Westat, the German Institute for Educational Research (D IPF ) , Pearson, and led by ETS . In each administration of PISA, one of the subject areas ( science, mathematics, or reading literacy) is the major domain and has the broadest content coverage, while the other two subjects are the minor domains. Other areas may also be assessed, such as , in the case of PISA 2015, collaborative problem solving and financial literacy . PISA emphasizes f unctional skills that students have acquired as they near the end of mandatory schooling (aged 15 years). Moreover, PISA assesses students’ knowledge and skills gained both in and out of school environments. The focus on the “yield” of education in and out of school makes it different from other international assessments such as the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS), which are closely tied to school curriculum frameworks and assess younger and grade
- based populations. PISA 2015 represents an important shift for PISA. While previous cycles have included optional computer
based components, and in 2012 the United States administered computer
- based assessments in reading, mathematics, and problem solving, the primary means of assessing students and collecting questionnaire data has been through paper
- based instruments. In PISA 2015, the entire assessment and the questionnaires will be administered on computer. Moving to c omputer will enable PISA to include more innovative assessment items not possible with paper
- based instruments, will allow for greater efficiency in scoring and data processing, will allow PISA to collect better and more information about what students know and can do , and measure how students interact with others and solve problems. While it is possible for countries to continue using paper
- based instruments, and some countries are choosing to do so, those instruments will not include new items. The United States will administer PISA 2015 on computer . PISA 201 5 will focus on science literacy as the major domain. Reading and mathematics literacy will also be assessed as minor domains. In addition, there will be a collaborative problem solving assessment and an assess ment of financial literacy . PISA 201 5 represents the second cycle with the major domain in science literacy (PISA 200 6 was the first). This is the first time PISA is assessing collaborative probl em solving . The science, mathematics, and reading literacy assessments and the collaborative problem solving assessment are

1 Some PISA participants are subnational jurisdictions (e.g., Hong Kong, China).iii

core components of PISA 201 5 and all education systems administering PISA 2015 on computers are required to participate. Financial literacy is optional for countries. In addition to the cognitive assessments described above, PISA 201 5 will include questionnaires administered to assessed students , school principals , and teacher s . The school and teacher questionnaires will be delivered online. The school and student questionnaire ar e core components of PISA and as such are required for countries, while the teacher questionnaire is optional for countries. To prepare for the main study in 201 5 , PISA countries will conduct a field test in the spring of 201 4 , to evaluate assessment and questionnaire items. The field test will also be used to evaluate the quality of new items developed to address the PISA 2015 science literacy, collaborative problem solving, and financial literacy frameworks, as well as new items in the student, schoo l, and teacher questionnaires, developed per the PISA 2015 background questionnaire framework. Another important element of this field test is a mode effects study to support PISA’s transition from a paper
- and
- pencil to a fully computer
- based assessment in 2015. For the mode effects study, some students will take paper
- based versions of PISA “trend items” (items that have been administered in past administrations of PISA that are used to measure trends over time) and some students will take computer
- based versions of those same items. The mode effect study will include trend items in science, mathematics, reading, and financial literacy. The field test will also be used, in the United States, to help determine whether to administer the optional financial literacy assessment and the optional teacher questionnaire in the main study. In Part B plans for determining whether to field the financial literacy assessment and the teacher questionnaire in the main study are detailed. Finally, the field test will be used to test school recruitment, data collection, and data management procedures in preparation for the main study. The U.S. PISA field test data collection will occur from April
- May 201 4 . The United States plans to recruit 54 public schools and 6 private schools for the field test , for a total of 60 national schools , with the expectation that the 54 public schools will be large enough to yield 36 assessed students from each school for a total student sample of approximately 2 , 0 00 students . F urthermore, w e anticipate that in 39 of the 54 public schools , 12 students will be sampled for the paper
- and
- pencil field test assessment (n= 4 6 8) , and 24 for the computer - based assess ment (n=1 , 296). In the remaining 15 public schools and 6 private schools, all students will be assessed via computer. The financial literacy assessment will only be administered in the 39 schools that are administering both paper
- and
- pencil and computer
- based assessments. Students will be subsampled from the original student sample and will return to take the financial literacy assessment in a second session. From the paper
- and
- pencil assessments, 6 students will be sampled and in the computer
- based assessment 9 students will be sampled to take financial literacy . In addition to the field test , NCES plans to conduct focus groups with principals to examine the challenges of recruitment and ways to increase participation and knowledge of PISA in the main study . A separate OMB request will be submitted for these activities. The U.S. PISA main study w ill be conducted from September through November 201 5 . The main study will involve a nationally
- representative sample of 5,600 students in the target population from 1 65 schools. Each student will be administered a 2
- hour computer - bas ed assessment that will include some combination of science, mathematics, reading, and collaborative problem solving items and a 30
- minute student questionnaire . In addition, if financial literacy will be administered in the main study, a subsample of 8 students will return for an additional 1
- hour assessment in financial literacy. The school principal of each sampled school will complete a 45
- minute online questionnaire. The United States also may opt to administer a 30 minute teacher questionnaire , which would be administered to up to 2 5 teacher s who are eligible to teach the modal grade (grade 10) within a school and would be delivered online .iv

I n order to meet the international data collection sc hedule for the s pring 201 4 field test, questionnaires must be finalized by September 2013 and recruiting activities begun by October 201 3. This submission requests approval for : 1. recruiting for the 2014 field test and 201 5 main study; 2. conducting the field test data collection ; and 3. a description of the overarching plan for all of the phases of the data collection , including the 201 5 main study . 2 The international schedule calls for field test data collection to take place in April
- May 201 4 , recruiting for the main study beginning in the f all of 201 4 (at least 12 months in advance of the data collection), and main study data collection in the f

all of 201 5 . Field test recruitment materials, including letters to state and district officials and school principals, text for a PISA field test brochure, “Frequently Asked Questions , ” and a “Summary of Activities” are included in Appendix A. Parental consent letters and related materials for the field test are included in Appendix B. Main study materials will be based on these , but will reflect the main study design and components to be administered. It is important to note that because PISA is a collaborative international study, the U.S. administration of PISA operates under some constraints, particularly around the schedule and the availability of instruments, which are negotiated internationally. For example, at the time that this package is submitted, the final versions of the field test questionnaires are not yet available. Instead, in Appendix C , NCES has included the international versions of the field test questionnaire items , with proposed adaptations to these items for use in the United States . Once the final adaptations are approved by the international contractor, by the end of May 2013, the final versions of the questionnaires will replace the draft questionnaires in this submission. Further, in order to begin recruiting schools for the main study by September 201 4 , in May 201 4 we will submit a change
- request memo to OMB that will : provide the final main study recruitment materials and parental consent letters; document any changes to the design and procedures for the main study; and detail the resulting respondent burden estimates for the main study data collection. Lastly, in late fall 2014 we will submit a clearance package, with a 30
- day notice published in the federal register, which will include the final main study instruments for data collectio n in the fall of 201 5 . The main study questionnaires will be a subset of the field test instruments .

2 The materials that will be used in the 2015 main study will be based upon the field test materials included in this submission. Additionally, this submission is designed to adequately justify the need for and overall practical utility of the full study and to present the overarching plan for all of the phases of th e data collection, providing as much detail about the measures to be used as is available at the time of this submission. As part of this submission, NCES is publishing a notice in the Federal Register allowing first a 60
- and then a 30
- day public comment period. For the final proposal for the full study, after the field test, NCES will publish a notice in the Federal Register allowing an additional 30
- day public comment period on the final details of 2015 main study.1 **A. JUSTIFICATION**  **A.1 Importance of Information**  As part of a continuing cycle of international education studies, the United States, through the National Ce nter for Education Statistics (NCES), is participating in several international assessments and surveys. The Program for International Student Assessment (PISA), sponsored by the Organization for Economic Cooperation and Development (OECD), is one of these studies. In light of the growing concerns related to international economic competitiveness, the changing face of our workplace, and the expanding international marketplace in which we trade, knowing how our students and adults compare with their peers around the world has become an even more prominent issue than ever before. Nationwide, interest in understanding what other nations are doing to further the educational achievement of their populations has increased beyond simple comparisons. Data at critic al points during the education career of our students will help inform policymakers in their efforts to guide and restructure the American education system. These critical points may occur during primary, secondary, or tertiary education, as well as extending into adult education and training programs. Consequently, generating comparative data about students in school, at the end of schooling, and about adults in workplace and in community has become an important focus for NCES. PISA 201 5 is part of the larger international program that NCES has actively participated in through collaboration with, and representation at, the OECD, the Asia
- Pacific Economic Cooperation (APEC), and the International Association for the Evaluation of Educational Achievement (IEA ). Collaboration with Statistics Canada, Eurostat, and ministries of education throughout the world helps to round out the portfolio of data NCES compiles. Through this active participation, NCES has sought to strengthen the quality, consistency, and timeliness of international data. To continue this effort, the United States must follow through with well
- organized and executed data gathering activities within our national boundaries. These efforts will allow NCES to build a data network that can provide th e information necessary for informed decision
- making on the part of national, state, and local policymakers. PISA measures students' knowledge, skills, and competencies primarily in three subject areas

– science, reading, and mathematics literacy. The overall strategy is to collect in
- depth information on student capabilities in one of these three domains every 3 years so that detailed information on each becomes available every nine years. During each 3
- year survey cycle, the major focus is on one content domain, with a minor focus on the other two content domains. The major focus for the data collection in 201 5 is on science literacy, with a minor focus on mathematics and reading. The 201 5 data collection will be the second time the focus has been on scien ce literacy, thus allowing the first in
- depth comparison of performance in science . The target population for this project will be a nationally representative sample of 15
- year - old students. PISA 201 5 will assess students with computer - based assessments in science , mathematics, reading, collaborative problem solving, and financial literacy . Moving to computer will enable PISA to include more innovative assessment items not possible with paper
- based instruments, will allow for greater efficiency in scoring and data processing, and will allow PISA to collect better and more information about what students know and can do. Over the last few decades, the world has become accustomed to hearing about Gross Domestic Products, Consumer Price Indices, unemployment rates, and other similar terms in news reports comparing national economies. The use of these economic indicators allows for discussion and debate of complex economic activities with well
- respected measures of that activity. Edu cation policymakers and the general public have a similar need to discuss what is going on in the field of education with indicators that are based on valid and2

reliable data and other information. Outcome data from PISA allow U.S. policymakers to gauge U. S. performance in relation to other countries, as well as monitor progress over time in comparison to these countries. The results of the PISA assessments, published every 3 years along with related indicators, will allow national policymaker s to compare the performance of their education systems with those of other countries. Further, the results will provide a basis for better assessment and monitoring of the effectiveness of education systems at the national level. Without these kinds of data, U.S. policymakers will be limited in their ability to gain insight into the educational performance and practices of other nations as they compare to the United States, and would have lost an investment made in previous cycles in measuring trends. The Educational Te sting Service (ETS) , under contract to the OECD, is responsible for the international components of this project , along with several other international organizations . Westat, the data collection contractor for the United States will work directly with ETS and the PISA U.S. National Project Managers from NCES. **A.2 Purposes and Uses of Data**  Governments and the general public want solid evidence of education outcomes. In the late 1990s, the OECD launched an extensive program for producing policy
- oriented and internationally comparable indicators of student achievement on a regular basis and in a timely manner. PISA is at the heart of this program. How well are schools preparing students to meet the challenges of the future? Parents, students, the public, and those who run education systems need to know whether children are acquiring the necessary skills and knowledge, whether they are prepared to become tomorrow's workers, to continue learning throughout life, to analyze, to reason, and to communicate ideas eff ectively. The results of OECD’s PISA, published every 3 years along with related indicators, allow national policymaker s to compare the performance of their education systems with those of other countries. Further, the results provide a basis for better as sessment and monitoring of the effectiveness of education systems at national levels. Through PISA, OECD produces three types of indicators:  Basic indicators that provide a baseline profile of the knowledge, skills, and competencies of students;  Contextual indicators that show how such skills relate to important demographic, social, economic, and education variables; and  Trend indicators that emerge from the ongoing, cyclical nature of the data collection. **PISA 201 5**  **Components** The primary focus for the asse ssment and questionnaires for PISA 201 5 will be on science literacy. The PISA science framework define s science literacy as an individual’s:
• s cientific knowledge and use of that knowledge to identify questions, acquire new knowledge, explain scientific phenomena and draw evidence
- based conclusions about science
- related issues ; • understanding of the characteristic features of science as a form of human knowledge and enquiry; • a wareness of how science and technology shape our material, intelle ctual and cultural environments; and • w illingness to engage in science
- related issues, and with the ideas of science, as a reflective citizen. As in all administrations of PISA, reading and mathematics literacy also will be assessed, although they will be “minor domains” in 201 5 . Questionnaires will be administered to students, school principals and teachers .3

As part of the transition to a fully computerized assessment, a mode effects study will be required as part of the field test for countries participating in the comp uter - based assessment. This will investigate the invariance between item parameters across the two modes of administration. To do this, 18 paper
- based forms in science, mathematics , and reading have been identified (6 forms in each domain) plus an addition al 2 forms for financial literacy . The tasks in these blocks have been adapted and transformed into an iden tical set of 20 computer - based forms . **Assessment instrumen**

**ts Field Test**  **:** There are a total of 60 forms in the field test for science, mathematics, reading, and collaborative problem solving which will be administered in a 2 hour session . There are additional 4 forms for financial literacy that will be administered to a subsample of students in a second, one - hour session. Wi thin each assessment session e ach sampled student will complete one form. The forms are designed to collect data on new items and to conduct the mode effects study to support the transition from paper
- based to computer
based format.
• 24 computer
- based form s contain new items assessing science or science and collaborative problem solving
• 20 paper - based forms where 18 contain a combination of reading, mathematics, and/or science , and 2 that contain financial literacy “trend items” from past cycles of PISA , and
• 20 computer - based forms where 18 contain a combination of reading, mathematics, and/or science , and 2 that contain financial literacy “trend items” from past cycles of PISA that have been “reauthored” for the computer
- based environment. The 20 paper - based and 20 computer - based forms will be used for the mode effects study. The financial literacy assessment will be evaluated after the field test and during main study school recruitment to determine whether it should be administered in the main s tudy (see further details in Supporting Statement Part B) . **Main Study :**  There will be a total of 66 computer
- based forms administered in a 2 hour session; each student will take one form. All forms contain items assessing science and one or two additional subjects . Most students will take forms containing science and reading literacy or science and mathematics literacy. The United States plans to conduct financial literacy in the main study, pending a review of the field test experience . A subsample of 8 students will be selected to take financial literacy. **Questionnaires** The background questionnaires for PISA 2015 have been developed to address a background questionnaire framework developed internationally . In addition, the questionnaires include items that have been included in multiple cycles of PISA allowing the investigation of patterns and trends over time. Countries adapt the questions to fit their national context and the questionnaires are reviewed and verified to ensure they remain comparable across countries. **School questionnaire.** A representative from each participating school will be asked to provide information on basic demographics of the school population and more in
- depth information on one or more specific issues (generally related to the co ntent of the assessment in the major domain, science). Basic information to be collected includes data on school location; measures of socio
- economic context of the school, including location, school resources, facilities, and community resources; school s ize; staffing patterns; instructional practices; and school organization. The in
- depth information is designed to address a very limited selection of issues that are of particular interest and that focus primarily on the major content domain, science. For both the field test and main study, i t is anticipated that the school questionnaire will take approximately 60 minutes in the field test and 45 minutes in the main study . It will be available to respondents online.4

**Teacher questionnaire.** The United States will administer a teacher questionnaire as part of the field test . Based on the field test results a decision will be made to pursue the teacher questionnaire for the main study. The teacher questionnaire will be offered online and is estimated to take approximately 45 minutes to complete in the field test and 30 minutes in the main study . Within a school, a total of up to 25 teachers who are eligible to teach the modal grade (grade 10) will be selected. Up to ten teachers will be s cience teachers (teachers who are eligible to teach grade 10 in a science subject) and up to 15 teachers will be non - science teachers (teachers who are eligible to teach grade 10, but not in science) . The teachers and students are not linked, that is, the teachers are not necessarily teachers of the sampled students. The sampling selection for teacher and students are independent of one another. The results of the field test will be used to decide whether to carry the teacher questionnaire to the main study . We will evaluate not only the item data, but the reaction and response to the questionnaire s by teachers and schools to make sure the instrument does not negatively impact school response rates. Criteria for evaluating the administration of the teacher questionnaire are described in Part B. **Student questionnaire.** Participating students will be asked to provide basic demographic data and information pertaining to the major assessment domain, science . Basic information to be collected includes demographics (e.g., age, gender, language, race, ethnicity); socio
- economic background of the student (e.g., parental education, economic background); student's education career; and educational resources and their use at home and at school. Domain
- specific information will include instructional experiences and time spent in school, as perceived by the students, and student attitudes. It is anticipated that the student questionnaire will take approximately 30 minutes to complete. In the field test there will be multiple forms of the questionnaire in order to try out different items and item formats . The main study may or may not use multiple forms. The field test versions of the PISA questionnaires are include d in Appendix C. For the field test , both paper
based and computer - based versions of the questionnaire will be administered. However, students selected to take the paper
- based assessment will receive one page of core trend questionnaire items (5 minutes). Students taking CBA will receive a full version of the stu dent quest ionnaire with 5 minutes of core items and 25 minutes of additi onal items for a total of 30 minutes. In addition, students sampled for financial literacy will receive 5 minutes of background questions related to financial literacy (these are the s ame items that were used in 2012) . For the main study, only a computer - based version of the student questionnaire will be administered. **A.3 Improved Information Technology (Reduction of Burden)**  The PISA 201 5 design and procedures are prescribed internation ally. Data collection will consist of computer - based responses for science, mathematics, reading, collaborative problem solving, and financial literacy . Responses to the computer
- based assessments will be captured electronically. During the field test , 42 students within each school will be sampled. I n 39 of the schools up to 12 students will be assessed separately using paper
- based instruments in order to conduct the mode effects study. The remaining 3 0 students will take the computer
- based assessment. In the United States, the computer
- based assessments will be implemented using laptops carried into school s by the data collection staff. In PISA 201 5 , the school questionnaire will be available for the first time to school administrators on
- line as the main mode of administration . The teacher questionnaire will also be administered on
- line . We are examining the feasibility of providing a paper
- based back
- up of the school and teacher questionnaire s . Also , the student questionnaire s will be delivered via computer after the completi on of the computer
- based assess ment . Data from the questionnaire will be stored and transmitted in a simil ar way as the assessment data.5

**A.4 Efforts to Identify Duplication**  A number of international comparative studies already exist to measure achievement in science, mathematics , and reading, including the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS). The Program for the International Assessment of Adult C ompetencies (PIAAC), administered in 2012 , measure s the reading literacy, numeracy, and problem
- solving skills of adults. In addition, the United States has been conducting its own national surveys of student achievement for more than 4 0 years through the National Assessment of Educational Progress (NAEP) program. PISA differs from these studies in several important ways: **Content.** PISA is designed to measure “literacy” broadly, while other studies, such as TIMSS and NAEP, have a strong link to curriculum frameworks and seek to measure students’ mastery of specific knowledge, skills, and concepts. The content of PISA is drawn from broad content areas, such as understanding, using, and reflecting on written information for reading, in contrast to more specific curriculum
- based content such as decoding and literal comprehension. Moreover, PISA differs from other assessments in the tasks that students are asked to do. PISA focuses on assessing students’ knowledge and skills in science, reading, and mathematics li teracy in the context of everyday situations. That is, PISA emphasizes the application of knowledge to everyday situations by asking students to perform tasks that involve interpretation of real
-world materials as much as possible. A study based on expert panels’ reviews of mathematics and science items from PISA, TIMSS, and NAEP reports that PISA items require multi
- step reasoning more often than either TIMSS or NAEP. 3 The study also shows that PISA mathematics and science literacy items often involve the interpretation of charts and graphs or other “real world” material. These tasks reflect the underlying assumption of PISA: as 15
- year - olds begin to make the transition to adult life, they need to know not only how to read, or know particular mathematical f ormulas or scientific concepts, but also how to apply this knowledge and these skills in the many different situations they will encounter in their lives. The computer
based assessments to be included in 2012 add additional “real world” tasks, given the predominance of technology in the lives of young adults. **Age -**  **based sample.** The goal of PISA is to represent outcomes of learning rather than outcomes of schooling. By placing the emphasis on age, PISA intends to show not only what 15
- year - olds have learned in school, but outside of school, as well as over the years, not just in a particular grade. In contrast, NAEP, TIMSS, and PIRLS are all grade
- based samples: NAEP assesses students in grade 4, 8, and 12; TIMSS assesses students in grades 4 and 8; and PIRLS assesses students in grade 4. PISA thus seeks to show the overall yield of an education system and the cumulative effects of all learning experience. Focusing on students at age 15 provides an opportunity to measure broad learning outcomes while all stude nts are still required to be in school across the many participating nations. Finally, because years of education vary among countries, choosing an age
- based sample makes comparisons across countries somewhat easier than a grade
based sample. **Information collected**  . The kind of information PISA collects also reflects a policy purpose slightly different from the other assessments. PISA collects only background information related to general school context and student demographics. This differs from other international studies such as TIMSS, which collects background information related to how teachers in different countries approach the task of teaching and how the approved curriculum is implemented in the classroom. The results of PISA will certainly inform education policy and spur further investigation into differences within and between countries, but PISA is not intended
3 Neidorf, T.S., Binkley, M., Gattis, K ., and Nohara, D. (2006). Comparing Mathematics Content in the National Assessment of Educational Progress (NAEP), Trends in International Mathematics and Science Study (TIMSS), and Program for International Student Assessment (PISA) 2003 Assessments (NCES 2006 - 029). U.S. Department of Education. Washington, DC: National Center for Education Statistics.6

to provide direct information about improving instructional practice in the classroom. The purpose of PISA is to generate useful indicators to benchmark performance and inform policy. Alternate sources for these data do not exist. This study represents the U.S. participation in an international study involving approximately 65 countries and jurisdictions in the PISA 201 5 field test in spring of 2014 and the main study in fall of 2015 . The United States must collect the same information at the same time as the other nations for purposes of making international comparisons. No other study in the United States will be using the instruments dev eloped by the international sponsoring organization, and thus no alternative sources of comparable data are available. In order to participate in the international study, the United States must agree to administer the same core instruments that will be administered in the other countries. Because the items measuring academic achievement have been developed with intensive international coordination, any changes to the PISA 201 5 instruments would also require international coordination. **A.5 Minimizing Burden**  **for Small Entities** No small entities are part of this sample. The school sample for PISA will contain small
- , medium
- , and large
size schools from a wide range of school types, including private schools, and burden will be minimized wherever possible for all institutions participating in the data collection. For example, the selection of schools to be assessed in the PISA 2015 field test (spring 201 4 ) will avoid overlap with the selection of schools for NAEP or TIMSS, which will also be in the field in the spring of 201 4 . Schools included in the field test will have a low likelihood of being included in the main study. Student burden will be reduced through the use of multiple forms of the student background questionnaire. In the field test this will allow P ISA to test out new background items or differing versions of items without adding to administration time. In addition, contractor staff will assume as much of the organization and test administration as possible within each school. Contractor staff will undertake all test administration and these staff will also assist with parental notification, sampling, and other tasks as much as possible within each school. **A.6 Frequency of Data Collection**  This request to OMB is for the PISA 2015 field test in s pring of 2014 and main study in f all of 2015 . PISA is conducted on a 3
- year cycle as prescribed by the international sponsoring organization, and adherence to this schedule is necessary to establish consistency in survey operations among the many participa ting countries. **A.7 Special Circumstances**  No special circumstances exist in the data collection plan for PISA 201 5 that would necessitate unique or unusual manners of data collection. None of the special circumstances identified in the Instructions for Supporting Statement applies to the PISA 201 5 study. **A.8 Consultations Outside NCES**  Consultations outside NCES have been extensive and will continue throughout the life of the project. The nature of the study requires this, because international studies typi cally are developed as a cooperative enterprise involving all participating countries. PISA 2015 is being developed and operated, under the auspices of the OECD, by a consortium of organizations. Key persons from these organizations who are involved in the design, development and operation of PISA 2015 are listed below. **Organization for Economic Cooperation and Development** Andreas Schleicher , Indicators and Analysis Division7

2, rue André Pascal , 75775 Paris Cedex16 , FRANCE Tel: +33 (1) 4524 9366 , Fax: +33 (1) 4524 9098 **Educational Testing Service** Irwin Kirsch , Project Director , ETS Corporate Headquarters 660 Rosedale Road , Princeton, NJ 08541 USA Tel : 1
- 609
- 921
- 9000 , Fax : 1
- 609
- 734
- 5410 **Westat** Keith Rust, Director of Sampling 16 0 0 Research Boulevard , Rockville, Maryland 20850
- 3129 USA Tel: 301 251 8278 , Fax: 301 294 2034 **A.9 Payments or Gifts to Respondents**  Currently, the minimum response rate targets required by OECD are 85 percent of original schools and 80 percent of students, while the NCES minimum response rate target is 85 percent at the student level. These high response rates are difficult to achieve in school
- based studies. The United States failed to reach the school response rate targets for the study in all previous PISA administrations (200 0, 2003, 2006, 2009 , and 2012) and had to adjust incentives upwards in the middle of the recruitment and data collection period in order to meet minimum response rate requirements. Gaining sufficient student cooperation is also challenging. While we met th e NCES requirement in PISA 2006 by 6 percent and PISA 2009 by barely 2 percent , unweighted results from 2012 suggest that we improved our rate to 89 percent . The added monetary incentives used in 2012 , particularly for school coordinators, had an impact on the increased student response rates in 2012 . School Coordinators indicated that the incentives were meaningful to them and the end result was an increase in the student response rate from 86 percent in 2009 to 89 percent. NCES is using a multi
- pronged ap proach to address the challenge of gaining school and student cooperation and learn as much as possible during the field test about how to achieve acceptable participation rates. First, our PISA contractor will review the most recent PISA 2012 experience to understand where possible improvements can be made in materials and communication with schools. S taff with experience working on the National Assessment of Educational Progress (NAEP) , PISA , other international assessments, other large
- scale data collections, and with expertise in effective approaches to school recruitment will provide input so that strategies can be identif ied for achieving high response rates and serve as an ongoing source of ideas and feedback. Second, in September
- October 201 3 we will conduct focus groups with principals of refusing schools in PISA 2012 to gain insights into the barriers to participating in PISA and desirable approaches to gaining cooperation . We will also continue the use of effective incentives. The proposed amounts are described below and are based on the amounts used in PISA 2012 . **Schools.** Schools participating in PISA will receive $200. In order to meet the minimum school response rates mandated by the PISA international governing board, and in order to compensate the school for the increased disruption and burden resulting from the addition of a second session, we believe it is necessary to offer schools this incentive to encourage participation. **School coordinators.** The school coordinator will be offered $ 2 00 . The role of the school coordinator is critical for the success of the study. The coordinator is expected to coordinate logistics with the data collection contractor; supply a list of eligibl e students and teachers for sampling to the data collection contractor; communicate with teachers, students, and parents about the study to encourage participation; assist the test administrator in ensuring that the sampled students attend the testing sess ions; and assist the test administrator in arranging for make
- up sessions as needed..8

**Teachers:** The field test will implement a teacher questionnaire delivered as an on
- line instrument. Up to 25 teachers will be selected from each school. As in the Teaching and Learning International Survey (TALIS), s elected teachers will be offered $ 2 0 for complet ing the questionnaire. **Students** . As described in section A.2 and in Part B, in some field test schools, some students will take the paper - based assessment and some will take the computer
- based assessment (CBA) . In the remainder of the field test schools, all sampled students will take only the CBA . The former model reflects the international need to conduct a mode effect study of paper
- based assessment versu s computer - based asses s ment . The latter (CBA only schools) are included in the field test to obtain a sufficient sample size for item statistics and because this will be the main study mode of administration for all schools. A s in the previous PISA, all participating s tudents will be offered $25. Those students that participate in the financial literacy assessment will receive an additional $15 ; this is consistent with the incentive in PISA 2012 for students that returned for a second 1
- hour assessment sess ion . Additionally, students participating in the assessment during non
- school hours (after school or on a Saturday), which is an accommodation offered in the main study when it is not possible to find a suitable time within school hours, and one that is e xercised rarely, only as a last resort, will be offered $35. The increased incentive over and above the “normal” incentive compensates students for travel time and other activities (work, sports) that a student may miss to participate in the assessment out of hours ( i n PISA 2006 , 2009 and 2012 , we received approval to offer up to $50 for after
- school and up to $75 for Saturday assessments in the main study). Incentives for students will only be provided with the explicit permission of the school principal. All student incentives will be offered directly to the student. P arents will be informed of the amount of the payment the students will receive in the consent form/letter in advance of the assessment. The payments will likely take the form of a personal ch eck . This was the method used in the field test and main study in 2012 and it worked well on the whole. **A.10 Assurance of Confidentiality**  Procedures for handling confidential aspects of the study that will be used in PISA 201 5 will mirror those used in pas t administrations of PISA. E xpertise in data security and confidentiality wa s a significant criterion in the selection of the PISA 2015 contractor. The plan for maintaining confidentiality includes signed confidentiality agreements and notarized nondisclos ure affidavits obtained from all personnel who will have access to individual identifiers. Also included in the plan is personnel training regarding the meaning of confidentiality, particularly as it relates to handling requests for information and providing assurance to respondents about the protection of their responses; controlled and protected access to computer files under the control of a single data base manager; built
- in safeguards concerning status monitoring and receipt control systems; and a secured and operator
- manned in
- house computing facility. Letters and other materials will be sent to parents and school administrators describing the voluntary nature of this survey. The material sent will include a brochure that describes the study and convey s the extent to which respondents and their responses will be kept confidential (copies of letters to be used in the field test and the brochure text are included in Appendix A). The following statement will appear on the front cover of the questionnaires (the phrase “ gather the data needed , and complete and review the information collection ” will not be included on the student questionnaire): U.S. participation in this study is sponsored by the National Center for Education Statistics (NCES), U.S. Department of Education. Your responses are protected by federal statute (20 U.S.C., § 9573). Your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law .9

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this voluntary information collection is 1850
- 0755. The time required to complete this information collection is estimated to average XX minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving the form, please write to: U.S. Department of Education, Washington, D.C. 20202
- 4537. If you have comments or concerns regarding the status of your indiv idual submission of this form, write directly to: Program for International Student Assessment (PISA), National Center for Education Statistics, U.S. Department of Education, 1990 K Street, N.W., Washington, D.C. 20006.

OMB No. 1850
- 0755, Approval Expires xx/xx/ 2016 . Data files, accompanying software, and documentation will be delivered to NCES at the end of the project. No school or individual names or addresses will be included on these files or documentation. NCES unders tands the legal and ethical need to protect the privacy of the PISA respondents and has extensive experience in developing data files for release that meet the government’s requirements to protect individually identifiable data from disclosure. The contrac tor will conduct a thorough disclosure analysis of the PISA 201 5 data when preparing the data files for use by researchers , in compliance with 20 U.S.C., § 9573 . S chools with high disclosure risk will be identified and , to ensure that individuals may not b e identified from the data files, a variety of masking strategies will be used , including swapping data and omitting key identification variables (i.e., school name and address) from both the public
- and restricted
- use files (though the restricted
- use file will include an NCES school ID that can be linked to other NCES databases to identify a school); omitting key identification variables such as state or ZIP Code from the public
- use file; and collapsing or developing categories for continuous variables to retain information for analytic purposes while preserving confidentiality in public
- use files. **A. 11 Sensitive Questions**  PISA 201 5 does not include questions usually considered to be of a highly sensitive nature, such as items concerning religion, substance abuse, or sexual activity. The international version of the questionnaire includes items enquiring about students’ height and weight, with the intention to use the answers to construct measure of Body Mass Index (BMI). The United States will not administer these items. Several items in the background questionnaires may be considered sensitive by some of the respondents, such as the socioeconomic context of the school, parents’ education and occupation, family possessions, and students’ belongings. Research indicates that the constructs these items represent are strongly correlated to academic achievement, and they have been used in the five previous cycles of PISA (2000, 2003, 2006, 2009 , and 2012). Therefore, the items are essential for the anticipated analyses and to retain consistency in planned comparis ons with the international data . **A. 12 Estimates of Burden**  This package requests approval for the field test recruitment and data collection, and for the main study recruitment activities , which are to begin early in the fall of 2014 , after the completion of the field test. Burden estimates are shown in Table A.2. The time required for students to respond to the assessment (cognitive items) portion of the study, and associated directions, are shown in gray font and are not included in the totals because they are not subject to the PRA. Student, administrator, and teacher questi onnaires are included in the requested burden totals. Recruitment and pre
- assessment activities include the time involved10

in a school deciding to participate, completing teacher and student listing forms, distributing parent consent materials, and arranging assessment space. Burden estimates for the main study (calculated based on the scenario of the United States participating in the core and international optional teacher questionnaire) are also provided for information purposes in table A .2 . The cost/ burden to respondents for the PISA 201 5 field test is calculated for the estimated time required of students and school staff (school administrator and school coordinators) to complete recruitment, pre
assessment, and assessment activities (see Table A. 2 ) in 60 national schools . In addition, Puerto Rico has expr essed interest in participating in PISA 2015 . Puerto Rico expects to fund its own participation and to administer paper
- based instruments rather than the computer
- based instruments. Because Puerto Rico ha s not participated in PISA before (except in 2012 when PR administered PISA instruments as a research study) and is not administering the same instruments as in the national sample , Puerto Rico must participate in the field test and will asse ss students in 25 schools . The estimated burden for 25 Puerto Rico schools has been included in the burden estimates in case Puerto Rico do es participate . In Puerto Rico, the school questionnaire will be administered as a paper version , the teacher questio nnaire will not b e administered , and financial literacy will not be assessed . The main study burden estimates are shown below the field test estimates. In 2012, three states provided their own funding to participate in PISA. There is growing interest in PI SA at the state level and more states may choose to participate in 2015 . In addition, the President’s proposed 2014 budget includ e s funding for some states to collect state
- level PISA data. The main study burden estimates reflect burden for the inclusion o f up to 10 states and Puerto Rico (but do not include estimates for assessing financial literacy in any of the state sample s o r Puerto Rico) . The main study burden estimates will be updated following the field test as final design decisions are made. For the field test , the total student response burden of 1,3 73 hours from 3,053 students is bas ed on (a) a five minute core - only questionnaire for 468 students taking the paper
- based assessment in the national field test sample; (b) a 30
- minute background questionnaire for 1,692 students taking the computer
- based assessment in the national field test sample; (c) an additional 5 minute student questionnaire for a subsample of 497 students taking the national financial literacy assessment; and (d ) 893 student s taking a 30
- minute questionnaire in Puerto Rico . At an estimated $ 7.25 per hour (the 2009 Federal minimum wage) cost to students, the dollar cost of the field test study for students is estimated at $ 9, 954 . The total school response burden of 1,4 53 hours for schools in the field test is based on the following:  In 60 schools in the national sample , a 60
- minute school questionnaire for 60 school administrators ; a 45
- minute teacher questionnaire for 1,200 teachers ; 90 minutes for 60 school administrators during the field test recruitment process; and an average of 4 hours for 60 school coordinators to coordinate logistics with the data collection contractor; supply a list of eligible students and teachers for sampling to the data collection contractor; communicate with teachers, students, and parents about the study to encourage participation; assist the test administrator in ensuring the sampled students attend the testing sessions; and assist the test administrator in arranging make
- up sessions as nee ded.  In 25 schools in Puerto Rico, a 60
- minute school questionnaire for 25 school administrators; 90 minutes for 25 school administrators during the field test recruitment process; and an average of 4 hours for 25 Puerto Rican school coordinators performing the same duties described above for school coordinators in the national sample .11

**Table A -**  **2 .**  **Burde n**  **estimates for PISA 201 5**  **field test and main study**  Sam ple Expected

res ponse rate Number of respondents Num ber of responses Per respondent (mi nut es) To tal burden (hours) Student US C omputer-based D irections (com puter-based assessm ent) 1 ,991 0.85 1 ,692 1 ,692 1 0 2

82 Com puter-based assessm ent 1 ,991 0.85 1 ,692 1 ,692 1 20 3,384 Financial L

iteracy com puter-based assessm ent 3 51 0.85 2 98 2 98 6 0 2

98 Student Questionnaire ( fu ll questionnaire) 1 ,991 0.85 1 ,692 1 ,692 3 0 8

46 Financial L

iteracy background item s CBA 3 51 0.85 2 98 2 98 5 2 5 US m ode effe ct study D irections 5 51 0.85 4 68 4 68 1 0 7 8 Paper-and-pencil test booklet 5 51 0.85 4 68 4 68 1 20 9 37 Financial L

iteracy paper-based assessm ent 2 34 0.85 1 99 1 99 6 0 1

99 Student Questionnaire ( paper-based core only) 5 51 0.85 4 68 4 68 5 3 9 Financial L

iteracy background item s (paper-based) 2 34 0.85 1 99 1 99 5 1 7 Puerto Rico D irections 1 ,050 0.85 8 93 8 93 1 0 1

49 Paper-and-pencil test booklet 1 ,050 0.85 8 93 8 93 1 20 1

,785 questionnaire) 1 ,050 0.85 8 93 8 93 3 0 4

46 Total St udent

Burden F

ield Trial 3 , 0 53 3,5 50 1 , 3 73 School Adm inistrator Q uestionnaire (U.S. online) 6 0 1

.00 6 0 6 0 6 0 6 0 Q uestionnaire (Puerto Rico paper-based) 2 5 1

.00 2 5 2 5 6 0 2 5 T eachers Q uestionnaire 1 ,500 0.80 1 ,200 1 ,200 4 5 9

00 School Adm inistrator 8 5 1

.00 8 5 8 5 9 0 1

28 School C

oordinator 8 5 1

.00 8 5 8 5 2

40 3 40 Total Sc hool Burden F ield Trial 1 , 4 55 1,4 55 1 , 4 53 Total Burden F

ield Trial 4, 5 08 5,0 05 2 , 825 National sam ple Student D irections 6,930 0.85 5 ,891 5 ,891 1 0 9

82 Com puter-based assessm ent 6,930 0.85 5 ,891 5 ,891 1 20 1

1,781 Financial L

iteracy com puter-based assessm ent 1 ,2 80 0.85 1 ,088 1 ,088 6 0 1

,088 Student Questionnaire ( com puter-based) 6,930 0.85 5 ,891 5 ,891 3 0 2,945 Financial L

iteracy background item s 1 ,2 80 0.85 1 ,088 1 ,088 5 9 1 Total St udent

Burden Main St

udy 5 , 8 91 6 ,9 79 3 5 3 , 0 36 School Adm inistrator Q uestionnaire (on-line) 1 65 1 .00 1 65 1 65 4 5 1

24 T eacher Q uestionnaire (on-line) 3,8 82 0.85 3,300 3,300 3 0 1

,650 Recruitm ent and Pre-Assessment Activity School Adm inistrator 1 65 1 .00 1 65 1 65 9 0 2

48 School C

oordinator 1 65 1 .00 1 65 1 65 2 40 6 60 Student 1 0 states com put er-based assessm ent Directions 21, 000 0.90 1 8,900 1 8, 900 1 0 3,150 Com puter-based assessm ent 21, 000 0.90 1 8,900 1 8, 900 1 20 37,800 Student Questionnaire ( computer-based) 21, 000 0.90 1 8,900 1 8, 900 3 0 9,450 Puerto Rico (Paper-based test booklet) Directions - paper and pencil test 2,1 00 0.90 1 ,890 1 ,890 1 0 315 Paper-and-pencil test booklet 2,1 00 0.90 1 ,890 1 ,890 1 20 3,780 Student Questionnaire ( paper-based) 2,1 00 0.90 1 ,890 1 ,890 3 0 9

45 School Adm inistrator Q uestionnaire (US

: O n-line) 5 00 1 .00 5 00 5 00 4 5 375 Q uestionnaire (Puerto Rico paper-based) 5 0 1

.00 5 0 5 0 4 5 3 8 T eacher Q uestionnaire (on-line; states only) 1 2, 500 0.85 1 0,625 1 0, 625 3 0 5

,313 Recruitm ent and Pre-Assessment Activity School Adm inistrator 5 50 1 .00 5 50 5 50 9 0 8

25 School C

oordinator 5 50 1 .00 5 50 5 50 2 40 2,200 Total Burden Main St udy 42, 751 4 3, 839 2 4, 825 5

, 9 38 6 ,4 35 6 , 7 58 Field T rial— Based on core + international options

Main Study —Based on core + international options

State benchmarking (up to 10 st ates & Puerto Rico) Recruitm ent and Pre-Assessment Activity (includes Puerto Rico)

NO TE: OM B Clearance Reques te d: To ta l Burden includes tota l burden as so ci at ed wit h conducting th e PISA 2 015 Fi eld Te st

a nd th e recruitm ent

a nd preas se ssme nt activ it ies fo r th e PISA

2 015 Ma in Stu dy. Th e PISA 2 015 Ma in Stu dy

b urden is c ons erv

at iv ely high becaus e the PISA 2 015 Ma in Stu dy

m ay i nclude up to 1 0 benchm arking st ates a nd Puerto Rico, howev er the burden is held cons is te nt with nat ional sample sc hools becaus e of p otential v

ariabilit y b et ween stat

es . To ta l student

b urden does not i nclude ti me fo r cognit iv e as se ssme nt

a nd its as so ci ated ins tr uctions . Pu erto Rico has expres se d interes t i n participating in PI SA 2 015 and would be ex pected to conduct a field trial. Th e es ti ma te s fo r Puert o Rico s chools a nd st udents hav e been added to th e paper-bas ed burden es ti mates fo r th e field trial and th e ma in study. The Teacher Ques ti onnaire will not b e adm inis tered in Puerto Rico. Total Burden Request ed in this Submission12

For the main study, recruitment and pre
- assessment burden of 3,025 hours is based on 715 school administrators at 90 minutes per school and an average of 4 hours for 715 school coordinators to coordinate the assessment in the school. This includes 165 scho ol administrators and 165 school coordinators participating in national PISA. In addition, the burden includes 500 school administrators and 500 school coordinators in state samples, accounting for the possible participation of up to ten states, plus 50 sc hool administrators and 50 school coordinators in Puerto Rico. Burden cost is calculated by combining the field test burden and the burden for the main study recruitment and pre
- assessment activities for the national, state , and Puerto Rico samples . At an estimated $50.00 per hour cost to administrators (1 , 2 86 hours), and an estimated $35.00 per hour cost for school coordinators and teachers (3,200 hours and 900 hours respectively), the dollar cost of the field test for schools is $ 207,800 ($ 64,300 for school administrators, $112,000 for school coordinators, and $31,500 for teachers). **A.13 Total Annual Cost Burden**  Other than the burden associated with completing the PISA questionnaires and assessments (estimated above in Section A.12), the field test and main study impose no additional cost to respondents. **A.14 Annualized Cost to Federal Government**  The cost to the Federal Government for conducting the PISA 2015 field test is estimated to be $ 2 , 483,434 over a 2
- year period. The total cost to the Fe deral Government for conducting the PISA 201 5 main study is estimated to be of $ 4, 278,915 over a 4
- year period. This is based on the national data collection contract, valued at $ 6, 762,349 over five years , from January 201 3 to January 201 8 , and includes cos ts for the national sample . These figures include all direct and indirect costs of the project . **A.15 Program Changes or Adjustments**  There is an overall reduction in burden , because the last approval was for the full scale PISA 20 12 collection, while this clearance request is only for field test and recruitme nt activities for PISA 2015 . With regards to content, there are some changes to PISA 201 5 from the previous rounds of data collection. The main change is that the assessment will focus on science litera cy during this cycle. The result is that the bulk of the items will be science items and that math e matics and reading will be the secondary components. Also, the collaborative problem solving assessment is new. The teacher questionnaire is also new for 201 5. The move to fully computer - based assessments also represent s a s ignificant change . Additionally, there are minor changes in wording to some of the questionnaire items, and questions that focused on student attitudes toward mathematics or reading now foc us on attitudes toward science . E ach student will still take a single 30
- minute questionnaire . **A.16 Plans for Tabulation and Publication**  The PISA2015 field test is designed to provide a statistical review of the performance of items on the assessments and questionnaires in preparation for the main data collection. The international contractor, ETS , will provide the international instruments to be used in the fiel d test and will report to the participating countries on the results of the field test . Based on the field test results, ETS , with input and agreement from the participating countries, will make final revisions in the survey instruments, materials, and doc uments in preparation for the main study. For the main study in 201 5 , an analysis of the U.S. and international data will be undertaken to provide an understanding of the U.S. national results in relation to the international results. Based on proposed analyses13

of the international data set by ETS , and the need for NCES to report results from the perspective of an American constituency, a plan is being prepared for the statistical analysis of the U.S. national data set as compared to the international data set. Analysis of data will include examinations of the science, reading, and mathematics literacy and collaborative problem
- solving of U.S. students in relation to their international counterparts; and the relationships between student performance and stud ent and school background variables. All reports and publications will be coordinated with the release of information from the international organizing body. Planned publications and reports for the PISA 201 5 main study include the following: **General Audi ence Report.**  This report will present information on the status of science, mathematics, and reading education among students in the United States in comparison to their international peers, written for a non - specialist, general U.S. audience. This report will present the results of analyses in a clear and non
technical way, conveying how U.S. students compare to their international peers, and what factors, if any, may be associated with the U . S. results. **Survey Operations/Technical Report.** This document will detail the procedures used in the main study (e.g., sampling, recruitment, data collection, scoring, weighting, and imputation) and describe any problems encountered and the contractor’s response to them. The primary purpose of the main study survey ope rations/technical report is to document the steps undertaken by the United States in conducting and completing the study. This report will include an analysis of non
- response bias , which will assess the presence and extent of bias due to nonresponse. Selected characteristics of respondent students and schools will be compared with those of non
- respondent schools and students to provide information about whether and how they differ from respondents along dimensions for which we have data for the nonresp onding units, as required by NCES standards. Electronic versions of each publication are made available on the NCES website. Schedules for tabulation and publication of PISA 201 5 results in the United States are dependent upon receiving data files from the international sponsoring organization. With this in mind, the expected data collection dates and a tentative reporting schedule are as follows: April
- December 201 3 Prepare OMB clearance documents, data collection manuals, forms, assessment materials, questionnaires for field test October 201 3
- February 201 4 Contact and gain cooperation of states, districts, and schools for field test March
– May 201 4 Select student samples and collect field test data July 201 4 Deliver raw data to international sponsoring organization August
– September 201 4 November 2014 Receive Field test Report from international sponsors , S ubmit OMB package September 201 4
–September 201 5 Prepare for the main study phase/ recruit schools June/July 201 5 Summer conference for sampled schools September 201 5
–

November 201 5 Collect main study data March
- April 201 6 Receive final data files from international sponsors August
- December 201 6 Produce General Audience Report and Survey Operations/Technical Report for the United States **A.17 Display OMB Expiration Date**  The OMB expiration date will be displayed on all data collection materials. **A.18 Exceptions to Certification Statement**  No exceptions are requested to the "Certification for Paperwork Reduction Act Submissions" of OMB Form 83
- I.