PROGRAM FOR INTERNATIONAL STUDENT ASSESSMENT

(PISA 2015) MAIN STUDY DATA COLLECTION

REQUEST FOR OMB CLEARANCE

OMB# 1850-0755 v.16

SUPPORTING STATEMENT PART A

Submitted by:

National Center for Education Statistics

U.S. Department of Education

Institute of Education Sciences

Washington, DC

January 2015
TABLE OF CONTENTS

PREFACE...................................................................................................................................... 2

A. JUSTIFICATION ................................................................................................................... 3

A.1 Importance of Information ............................................................................................... 3

A.2 Purposes and Uses of Data ............................................................................................... 5

A.3 Improved Information Technology (Reduction of Burden)........................................ 7

A.4 Efforts to Identify Duplication......................................................................................... 7

A.5 Minimizing Burden for Small Institutions....................................................................... 8

A.6 Frequency of Data Collection ........................................................................................... 8

A.7 Special Circumstances........................................................................................................ 8

A.8 Consultations Outside NCES ........................................................................................... 9

A.9 Payments or Gifts to Respondents................................................................................... 9

A.10 Assurance of Confidentiality ...........................................................................................11

A. 11 Sensitive Questions...........................................................................................................12

A. 12Estimates of Burden ........................................................................................................12

A.13 Total Annual Cost Burden...............................................................................................13

A.14 Annualized Cost to Federal Government .....................................................................13

A.15 Program Changes or Adjustments..................................................................................14

A.16 Plans for Tabulation and Publication.............................................................................14

A.17 Display OMB Expiration Date .......................................................................................15

A.18 Exceptions to Certification Statement...........................................................................15

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1 Respondent Universe

B.2 Statistical Methodology

B.3 Maximizing Response Rates

B.4 Purpose of Field test and Data Uses

B.5 Individuals Consulted on Study Design

Appendix A: Recruitment Materials (previously approved, OMB# 1850-0755 v.15)

Appendix B: Parental Consent Materials (previously approved, OMB# 1850-0755 v.15)

Appendix C: Data Collection Instruments (final versions, including Student Locator Questions)

Appendix D: Non-Response Bias Analysis Plan
2

PREFACE

The Program for International Student Assessments (PISA) is an international assessment of 15-year-olds 

which focuses on assessing students’ science, mathematics, and reading literacy. PISA was first administered 

in 2000 and is conducted every three years. The sixth cycle of PISA, PISA 2015, is being administered at a 

time when interest is increasing, both worldwide and in the United States, in how well schools are preparing 

students to meet the challenges of the future, and how the students perform compared with their peers in 

other education systems of the world. The participation in the PISA study among education systems1has 

significantly increased since the initial survey in 2000: 43 education systems in 2000, 41 in 2003, 57 in 2006, 65

in 2009, 65 in 2012, and approximately 70 are expected to participate in 2015. The United States has 

participated in all of the previous cycles, and will participate in 2015 in order to track trends and to compare 

the performance of U.S. students with that in other education systems.

PISA 2015 is sponsored by the Organization for Economic Cooperation and Development (OECD). In the 

United States, PISA 2015 is conducted by the National Center for Education Statistics (NCES) of the 

Institute of Education Sciences, U.S. Department of Education. PISA is a collaboration among the 

participating countries, the OECD, and a group of international organizations each under contract to the 

OECD (hereafter referred to as the PISA International Consortium), including the Educational Testing 

Service (ETS), Westat, the German Institute for Educational Research (DIPF), Pearson, and led by ETS.

In each administration of PISA, one of the subject areas (science, mathematics, or reading literacy) is the 

major domain and has the broadest content coverage, while the other two subjects are the minor domains. 

Other areas may also be assessed, such as, in the case of PISA 2015, collaborative problem solving and 

financial literacy. PISA emphasizes functional skills that students have acquired as they near the end of 

mandatory schooling (aged 15 years). Moreover, PISA assesses students’ knowledge and skills gained both in 

and out of school environments. The focus on the “yield” of education in and out of school makes it 

different from other international assessments such as the Trends in International Mathematics and Science 

Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS), which are closely tied to 

school curriculum frameworks and assess younger, grade-based populations.

PISA 2015 represents an important shift for PISA. While previous cycles have included optional computerbased components, and in 2012 the United States administered computer-based assessments in reading, 

mathematics, and problem solving, the primary means of assessing students and collecting questionnaire data 

has been through paper-based instruments. In PISA 2015, the entire assessment and the questionnaires will 

be administered on computer. Moving to computer enables PISA to include more innovative assessment 

items not possible with paper-based instruments, allows for greater efficiency in scoring and data processing, 

allows PISA to collect better information about what students know and can do, and measures how students 

interact with others and solve problems. While it is possible for countries to continue using paper-based 

instruments--and some countries are choosing to do so--those instruments will not include new items. The 

United States will administer PISA 2015 on computer.

PISA 2015 will focus on science literacy as the major domain. Reading and mathematics literacy will also be 

assessed as minor domains. In addition, there will be a collaborative problem solving assessment and an 

assessment of financial literacy. PISA 2015 represents the second cycle with the major domain being science 

literacy (PISA 2006 was the first). This is the first time PISA is assessing collaborative problem solving. The 

science, mathematics, and reading literacy assessments and the collaborative problem solving assessment are 

core components of PISA 2015 and all education systems administering PISA 2015 on computers are 

required to participate. Financial literacy is optional for countries. The United States will administer financial 

 

1

Some PISA participants are subnational jurisdictions (e.g., Hong Kong, China). 
3

literacy in 2015.

In addition to the cognitive assessments described above, PISA 2015 will include questionnaires administered 

to assessed students, school principals, and, for the first time, teachers. The school and teacher questionnaires 

will be delivered online. The school and student questionnaire are core components of PISA and as such are 

required for countries, while the teacher questionnaire is optional for countries. The United States will 

administer the teacher questionnaire in 2015.

The U.S. PISA main study data collection will occur during October-November 2015. It will involve a 

nationally representative sample of approximately 5,891 students in the target population from approximately 

165 schools. To accommodate the financial literacy assessment, students will be subsampled from the original 

student sample and will be asked to return for a second session. In addition, we anticipate the participation of

up to two states using their own funding in PISA 2015 (under discussion). To obtain state-level scores, 50 

schools will be sampled in the two US states in addition to the schools already selected in the national sample. 

Puerto Rico, using its own funding, will also participate in PISA 2015 but will administer paper instruments, 

which precludes the inclusion of collaborative problem solving, and will not administer the financial literacy 

assessment. As in the US states, 50 schools will be sampled in Puerto Rico for PISA 2015.

Each student will be administered a 2-hour computer-based assessment that will include some combination of 

science, mathematics, reading, and collaborative problem solving items as well as a 30-minute student 

questionnaire. In addition, a subsample of 11 students in each participating school will be asked to return for 

an additional 1-hour assessment in financial literacy. The school principal of each sampled school will be 

asked to complete a 45-minute online questionnaire and up to 25 teachers in each school will be asked to 

complete a 30-minute online teacher questionnaire.

NCES may conduct a methodological study to validate PISA by relating student performance on PISA 2015 

to other cognitive, education, and employment outcomes, as is being done with the students who participated 

in PISA 2012. To support the validation study, the PISA 2015 students will be asked to supply contact 

information so NCES can contact them in the future. After the assessment, schools will also be asked if they 

are able to supply student contact information. The follow-up study, including any follow-up contact with 

students, would be carried out under a separate OMB clearance request. In this current request we are seeking 

approval only to gather student contact information as was done in PISA 2012.

The materials to be used in PISA 2015 main study are based upon those approved for the field test data 

collections (OMB# 1850-0755 v.14), which adequately justified the need for and overall practical utility of the 

full study as proposed, along with an overarching plan for the phases of the data collection over 3 years, and 

which provided as much detail on the measures to be used as was available at the time of the submission. 

OMB approved the initial phase of this collection in September 2013 and now NCES published a notice in 

the Federal Register allowing a 30-day public comment period on the details of the subsequent study 

components described in this submission. In order to meet the international data collection schedule for the 

fall of 2015 main study, questionnaires must be finalized by February 2015. This submission requests 

approval for conducting the 2015 main study data collection and collecting student contact information for a 

potential future follow-up study.

A. JUSTIFICATION

A.1 Importance of Information

As part of a continuing cycle of international comparative education studies, the United States, through the 

National Center for Education Statistics (NCES), participates in several international assessments and 
4

surveys. The Program for International Student Assessment (PISA), sponsored by the Organization for 

Economic Cooperation and Development (OECD), is one of these studies.

In light of the growing concerns related to international economic competitiveness, the changing face of our 

workplace, and the expanding international marketplace in which we trade, knowing how U.S. students and 

adults compare with their peers around the world in critical subject areas has become an even more 

prominent issue than ever before. Nationwide, interest in understanding what other nations are doing to 

further the educational achievement of their populations has increased beyond simple comparisons.

Data at critical points during the education career of our students helps inform policymakers in their efforts 

to guide and restructure the American education system. These critical points may occur during primary, 

secondary, or tertiary education, as well as extending into adult education and training programs.

Consequently, generating comparative data about students in school, at the end of schooling, and about adults 

in the workplace and in communities has become an important focus for NCES.

PISA 2015 is part of the larger international program that NCES has actively participated in through 

collaboration with, and representation at, the OECD, the Asia-Pacific Economic Cooperation (APEC), and 

the International Association for the Evaluation of Educational Achievement (IEA). Collaboration with 

Statistics Canada, Eurostat, and ministries of education throughout the world helps to round out the portfolio 

of data NCES compiles.

Through this active participation, NCES strengthens the quality, consistency, and timeliness of international 

data. To continue this effort, the United States must follow through with well-organized and executed data 

gathering activities within our national boundaries. These efforts allow NCES to maintain and build upon a 

data network that provides the information necessary for informed decision-making on the part of national, 

state, and local policymakers.

PISA measures students' knowledge, skills, and competencies primarily in three subject areas – science, 

reading, and mathematics literacy. The overall strategy is to collect in-depth information on student 

capabilities in one of these three domains every 3 years so that detailed information on each becomes 

available every nine years. During each 3-year survey cycle, the major focus is on one content domain, with a 

minor focus on the other two content domains. The major focus for the sixth data collection cycle in 2015 is 

on science literacy, with a minor focus on mathematics and reading literacy. The 2015 data collection will be 

the second time the focus has been on science literacy, thus allowing the first in-depth comparison of 

performance in science. The target population for this project will be a nationally representative sample of 15-

year-old students. PISA 2015 will assess students with computer-based assessments in science, mathematics, 

reading, collaborative problem solving, and financial literacy. Moving to computer enables PISA to include 

innovative assessment items not possible with paper-based instruments, allows for greater efficiency in 

scoring and data processing, and allows PISA to collect more information about what students know and can 

do.

Over the last few decades, the world has become accustomed to hearing about Gross Domestic Products, 

Consumer Price Indices, unemployment rates, and other similar terms in news reports comparing national 

economies. The use of these economic indicators allows for discussion and debate of complex economic 

activities with well-respected measures of that activity. Education policymakers and the general public have a 

similar need to discuss what is going on in the field of education with indicators that are based on valid and 

reliable data. Outcome data from PISA allow U.S. policymakers to gauge U.S. student performance in relation 

to their peers in other countries, and to monitor progress over time in comparison to these countries. In 

addition to providing policymakers and educators with comparative performance data, PISA results provide a 

basis for better assessment and monitoring of the effectiveness of education systems at the national level.

Without these kinds of data, U.S. policymakers are limited in their ability to gain insight into the educational 
5

performance and practices of other nations and result in the loss of investments made in previous cycles in 

measuring trends.

In the United States, PISA 2015 is conducted by NCES, with Westat serving as the data collection contractor. 

The Educational Testing Service (ETS), under contract to the OECD, is the primary entity responsible for 

the international components of this project along with several other international organizations.

A.2 Purposes and Uses of Data

Governments and the general public want solid evidence of education outcomes. In the late 1990s, the 

OECD launched an extensive program for producing policy-oriented and internationally comparable 

indicators of student achievement on a regular, timely basis. PISA is at the heart of this program. PISA seeks 

to help parents, students, the public, and those who run education systems understand whether children are 

acquiring necessary skills and knowledge, whether they are prepared to become tomorrow's workers who will 

be required to continue learning throughout their lifetimes to analyze, to reason, and to communicate ideas 

effectively.

Through PISA, the OECD produces three types of indicators:

 Basic indicators that provide a baseline profile of the knowledge, skills, and competencies of students;

 Contextual indicators that show how such skills relate to important demographic, social, economic, and 

education variables; and

 Trend indicators that emerge from the ongoing, cyclical nature of the data collection.

PISA 2015 Components

The primary focus for the assessment and questionnaires for PISA 2015 is on science literacy. The PISA 

science framework defines science literacy as an individual’s:

• scientific knowledge and use of that knowledge to identify questions, acquire new knowledge, explain 

scientific phenomena and draw evidence-based conclusions about science-related issues;

• understanding of the characteristic features of science as a form of human knowledge and enquiry;

• awareness of how science and technology shape our material, intellectual, and cultural environments; 

and

• willingness to engage in science-related issues, and with the ideas of science, as a reflective citizen.

As in all administrations of PISA, reading and mathematics literacy also will be assessed, although they will be 

minor domains in 2015. Questionnaires will be administered to students, school principals, and teachers. In 

addition to the main PISA assessment subjects, two additional components will be measured in 2015: 

collaborative problem solving and financial literacy.

Assessment instruments

Main Study: There will be a total of 66 computer-based assessment forms administered in a 2 hour session; 

each student will take one form. All forms contain items assessing science and one or two additional subjects. 

Most students will take forms containing science and reading literacy or science and mathematics literacy. No 

student is asked to complete items related to all subjects being assessed. In addition to the main assessment, 

the United States will conduct financial literacy by selecting a subsample of 11 students from the main sample 

of students who will be asked to complete a separate financial literacy assessment. The international PISA 

consortium increased the sample size for financial literacy from the field test sample size of 8 students to 11 

students in the main study. This was instituted to counteract the 20 percent or more attrition rate of students 

that was observed internationally in the field test.
6

Questionnaires

The background questionnaires for PISA 2015 have been developed to address a questionnaire framework 

developed internationally. The school and student questionnaires contain items that have been included in 

previous cycles of PISA allowing for the investigation of patterns and trends over time. Countries adapt the 

questions to fit their national context and the questionnaires are reviewed and verified to ensure they remain 

comparable across countries. The teacher questionnaire is a new component for PISA 2015 and is based, in 

part, on items used in other studies, notably the Teaching and Learning International Survey (TALIS) which 

was fielded in 2013 (OMB# 1850-0888 v.3).

School questionnaire. The principal (or his or her designee) from each participating school will be asked to 

provide information on basic demographics of the school population and more in-depth information on one 

or more specific issues (generally related to the content of the assessment in the major domain – science).

Basic information to be collected includes data on school location; measures of socio-economic context of 

the school, including location, school resources, facilities, and community resources; school size; staffing 

patterns; instructional practices; and school organization. The school questionnaire will take 45 minutes on 

average to complete, and will be available to respondents online.

Teacher questionnaire. The teacher questionnaire, new for PISA in 2015, will be offered online and will

take approximately 30 minutes on average to complete. There are two versions of the teacher questionnaire:

for science teachers and non-science teachers. The questionnaire asks about teachers’ background 

information, their education and professional development, and school and teaching practices within a 

school. A total of up to 25 teachers who are eligible to teach the modal grade (grade 10) will be selected to 

complete the questionnaire in each participating school. The PISA sampling plan requires that up to 10 

teachers be science teachers (teachers who are eligible to teach grade 10 in a science subject) and up to 15 

teachers be non-science teachers (teachers who are eligible to teach grade 10, but not in science). The teachers 

and students are not linked, that is, the teachers are not necessarily teachers of the sampled students. The 

sampling selection for teacher and students are independent of one another.

Student questionnaire. Participating students will be asked to provide basic information both demographic 

and pertaining to the major assessment domain – science. The information that will be collected includes

demographics (e.g., age, gender, language, race, and ethnicity); socio-economic background of the student 

(e.g., parental education, economic background); student's education career; and educational resources and 

their use at home and at school. Domain-specific information will include instructional experiences and time 

spent in school, as perceived by the students, and student attitudes. The student questionnaire will take 

approximately 30 minutes on average to complete after the assessment.

The final, main study versions of the PISA questionnaires are included in Appendix C. In addition, students 

sampled for financial literacy will receive 5 minutes of background questions related to financial literacy (these 

are the same items that were used in 2012). For the main study, only a computer-based version of the student 

questionnaire will be administered.

Collecting Contact Information. NCES may conduct a methodological study to validate PISA by relating 

student performance on PISA 2015 to other cognitive, education, and employment outcomes, as was done 

with the students who participated in PISA 2012. To support the validation study we will collect contact 

information for students that participate in PISA 2015. The validation study, including any follow-up contact 

with students, would be carried out under a separate OMB clearance request. Appendix C includes items that 

will be used to ask students for their email, home address, and phone numbers, along with those of a relative 

or a close friend. These ‘locator’ items are the same as those used to collect contact information from the 

PISA 2012 students (OMB# 1850-0755 v.13) with one additional question asking for the email of a relative or 

another contact.
7

Parent consent and informational materials (Appendix A and B) indicate the possibility of a future follow-up 

study and contact: “NCES may contact your teenager after the PISA assessment for a follow-up study to 

improve PISA for the future. In order to do so NCES will ask your teenager for contact information.” The 

letter to schools and the “FAQs” (Appendix A) also state: “NCES may follow up with participating students 

as part of a study to improve PISA for the future.”

A.3 Improved Information Technology (Reduction of Burden)

The PISA 2015 design and procedures are prescribed internationally. Data collection will consist of

computer-based responses for science, mathematics, reading, collaborative problem solving, and financial 

literacy. Responses to the computer-based assessments will be captured electronically. In the United States, 

the computer-based assessments will be implemented using laptops carried into schools by the data collection 

staff.

In PISA 2015, the school questionnaire will be available for the first time to school administrators online as 

the main mode of administration. The teacher questionnaire will also be administered online. We are 

examining the feasibility of providing a paper-based back-up of the school and teacher questionnaires. Also, 

the student questionnaires will be delivered via computer after the completion of the computer-based 

assessment. Data from the questionnaire will be stored and transmitted in a similar way as the assessment 

data, following NCES standards for maintaining respondent privacy and confidentiality.

A.4 Efforts to Identify Duplication

A number of international comparative education studies already exist to measure achievement in science, 

mathematics, and reading, including the Trends in International Mathematics and Science Study (TIMSS) and 

the Progress in International Reading Literacy Study (PIRLS). The Program for the International Assessment 

of Adult Competencies (PIAAC), administered in 2012, measures the reading literacy, numeracy, and 

problem-solving skills of adults. In addition, the United States has been conducting its own national surveys 

of student achievement for more than 40 years through the National Assessment of Educational Progress 

(NAEP) program. PISA differs from these studies in several important ways:

Content. PISA is designed to measure “literacy” broadly, while other studies, such as TIMSS and NAEP, 

have a strong link to curriculum frameworks and seek to measure students’ mastery of specific knowledge, 

skills, and concepts that are taught in school. The content of PISA is drawn from broad content areas, such as 

understanding, using, and reflecting on written information for reading, in contrast to more specific 

curriculum-based content constructs such as decoding and literal comprehension. Moreover, PISA differs 

from other assessments in the tasks that students are asked to do. PISA focuses on assessing students’ 

knowledge and skills in science, reading, and mathematics literacy in the context of everyday situations. That 

is, PISA emphasizes the application of knowledge to everyday situations by asking students to perform tasks 

that involve interpretation of real-world materials as much as possible. A study based on expert panels’ 

reviews of mathematics and science items from PISA, TIMSS, and NAEP reported that PISA items require 

multi-step reasoning more often than either TIMSS or NAEP.2 The study also showed that PISA 

mathematics and science literacy items often involve the interpretation of charts and graphs or other “real 

world” material. These tasks reflect the underlying assumption of PISA: as 15-year-olds begin to make the 

transition to adult life, they need to know not only how to read, or know particular mathematical formulas or 

 

2

Neidorf, T.S., Binkley, M., Gattis, K., and Nohara, D. (2006). Comparing Mathematics Content in the National Assessment of Educational 

Progress (NAEP), Trends in International Mathematics and Science Study (TIMSS), and Program for International Student Assessment 

(PISA) 2003 Assessments (NCES 2006-029). U.S. Department of Education. Washington, DC: National Center for Education Statistics.
8

scientific concepts, but also how to apply this knowledge and these skills in the many different situations they 

will encounter in their lives. The computer-based assessment in 2015 adds additional “real world” tasks, given 

the predominance of technology in the lives of young adults today.

Age-based sample. The goal of PISA is to represent outcomes of learning rather than outcomes of 

schooling. By placing the emphasis on age, PISA intends to show not only what 15-year-olds have learned in

school, but also outside of school and over the years, not just in a particular grade. In contrast, NAEP, 

TIMSS, and PIRLS are all grade-based samples: NAEP assesses students in grade 4, 8, and 12; TIMSS 

assesses students in grades 4 and 8; and PIRLS assesses students in grade 4. PISA thus seeks to show the 

overall yield of an education system and the cumulative effects of all learning experiences. Focusing on 

students at age 15 provides an opportunity to measure broad learning outcomes while all students are still 

required to be in school across the many participating nations. Finally, because years of education vary among 

countries, choosing an age-based sample makes comparisons across countries somewhat easier than a gradebased sample.

Information collected. The kind of information PISA collects also reflects a policy purpose slightly different 

from the other assessments. For example, PISA collects background information related to general school 

context and student demographics with which to interpret student performance broadly. This differs from 

other international studies such as TIMSS, which collect background information more directly related to 

students’ classroom experiences.

Approximately 70 countries and jurisdictions will participate in PISA 2015. The data collection instruments 

included in this submission were developed with intensive international coordination, including input from 

U.S. representatives. In order to participate in the international study, the United States must agree to 

administer the same core instruments that will be administered in the other countries. No other study in the 

United States will be using the instruments developed by the international sponsoring organization, and thus 

no alternative sources of comparable data are available.

A.5 Minimizing Burden for Small Entities

No small entities are part of this sample. The school sample for PISA will contain small-, medium-, and largesized schools from a wide range of school types, including private schools, and burden will be minimized 

wherever possible for all institutions participating in the data collection. Contractor staff will assume as much 

of the organization and test administration as possible within each school, including carrying in all necessary 

computer equipment to conduct the assessment and undertaking all test administration. Contractor staff will 

also assist with parental notification, sampling, and other tasks as much as possible within each school.

A.6 Frequency of Data Collection

This request is for the PISA 2015 main study data collection in fall of 2015. PISA is conducted on a 3-year 

cycle as prescribed by the international sponsoring organization, and adherence to this schedule is necessary 

to establish consistency in survey operations among the many participating countries.

A.7 Special Circumstances

None of the special circumstances identified in the Instructions for Supporting Statement applies to the PISA 

2015 study.
9

A.8 Consultations Outside NCES

Consultations outside NCES have been extensive and will continue throughout the life of the project. The 

nature of the study requires this, because international studies typically are developed as a cooperative 

enterprise involving all participating countries. PISA 2015 is developed and operated by an international 

consortium of organizations under the auspices of the OECD. Key persons from these organizations who 

were involved in the design, development, and operation of PISA 2015 outside of NCES are listed below.

Organization for Economic Cooperation and Development

Andreas Schleicher, Indicators and Analysis Division

2, rue André Pascal, 75775 Paris Cedex16, FRANCE, Tel: +33 (1) 4524 9366, Fax: +33 (1) 4524 9098

Educational Testing Service

Irwin Kirsch, Project Director, ETS Corporate Headquarters

660 Rosedale Road, Princeton, NJ 08541 USA, Tel: 1-609-921-9000, Fax: 1-609-734-5410

Westat

Keith Rust, Director of Sampling

1600 Research Boulevard, Rockville, Maryland 20850-3129 USA, Tel: 301 251 8278, Fax: 301 294 2034

A.9 Payments or Gifts to Respondents

Currently, the minimum response rate targets required by OECD are 65 percent of original schools and 80 

percent of sampled students. The NCES minimum response rate target is 85 percent at the student level. 

Historically, these high response rates have been difficult to achieve in school-based studies. The United 

States failed to reach the school response rate targets in all previous PISA administrations (2000, 2003, 2006, 

2009, and 2012) and, due to school attrition over the course of the study, had to adjust incentives upwards in 

the middle of the recruitment and data collection period in order to meet the minimum school response rate 

requirement of 65 percent. Gaining sufficient student cooperation has also been challenging. The NCES 

student response rate requirements were met in PISA 2006 by 6 percentage points, in PISA 2009 by 2 

percentage points, and in PISA 2012 by 4 percentage points. The proposed incentive amounts described 

below are the same as those offered in PISA 2012.

Schools. Schools taking part in PISA will be offered $200 for their participation. In order to meet the 

minimum school response rates mandated by the PISA international governing board, and in order to thank

the schools for agreeing to the disruption and burden associated with taking part in the study, we believe it is 

necessary to offer schools the incentive to encourage their participation.

Although the field test was successful, we anticipate difficulty in reaching the required school response rates

in the main study, as has been the case in all past administrations of PISA. Although the field test had more 

schools participate than required and the disposition of the schools was positive toward PISA, the 

recruitment of these schools did not follow the model that will be used in the main study because we were 

not required to build a response rate for original schools. In the field test, we had to obtain participation from 

an adequate number of schools to get the required number of student responses for the test items. Therefore, 

we pooled our schools (both originally sampled schools and their replacement schools), approached the 

original schools first, but moved to the replacement schools quickly with very little, if any, conversion effort. 

This will not be the case in the main study where we must pursue the original schools until we obtain a 

satisfactory rate of participation. The historical experience for the United States is that obtaining a sufficient 

response rate of 65 percent of originally sampled, eligible schools is difficult and has required additional 

efforts. Moreover, in the field test we did not go to schools in states where we have traditionally had difficulty 

gaining school cooperation. Finally, we are learning from our recruitment efforts for other NCES studies that 
10

the current climate regarding voluntary assessments is even more challenging this year than in the past years 

and we are concerned that we may face even greater challenges than in past administrations of PISA.

Given these anticipated difficulties in securing school participation, we propose a second-tier of incentive, 

also approved and used in the PISA 2012 main study, which will allow us to offer schools a higher amount of 

$800 if necessary (this higher amount was used in the incentive experiment conducted as part of the PISA 

2012 field test). This second tier will not be initiated until near the end of the current academic year, in June

2015, after we have approached all original schools and had an opportunity to try different conversion efforts,

such as addressing the specific concerns of refusing schools and making personal visits to schools to discuss 

the study face to face. If, at that time, we have not reached a participation rate of at least 68 percent of 

original schools, we will implement the higher incentive rate (to meet the 65 percent of original schools

minimum requirement we need to recruit at least 68 percent of original schools factoring a 3 percent attrition 

of schools over the summer months before data collection begins in fall 2015). We will approach refusing 

schools with the second-tier incentive only if necessary and at the point of our last chance to convert them.

School coordinators. The school coordinator will be offered $200. The role of the school coordinator is 

critical for the success of the study. The coordinator is expected to coordinate logistics with the data 

collection contractor; supply a list of eligible students and teachers for sampling to the data collection 

contractor; communicate with teachers, students, and parents about the study to encourage participation; 

assist the test administrator in ensuring that the sampled students attend the testing sessions; and assist the 

test administrator in arranging for make-up sessions if needed..

Teachers: As in the Teaching and Learning International Survey (TALIS) conducted in 2013, selected 

teachers will be offered $20 for completing the online questionnaire. Up to 25 teachers will be selected from 

each school.

Students. The student burden in PISA 2015 will be the same as in PISA 2012, and as such, all participating 

students will be offered $25. Those students that also participate in the financial literacy assessment will 

receive an additional $15, which is consistent with the incentive offered in PISA 2012 for students that 

returned for a second 1-hour assessment session.

Additionally, students participating in the assessment during non-school hours (after school or on a Saturday), 

which is an accommodation offered in the main study when it is not possible to find a suitable time within 

normal school hours, and one that is exercised rarely, will be offered $35. The increased incentive over and 

above the base incentive reflects the addition of travel time and potentially missing other activities (work, 

sports, etc.) for students participating in the assessment out of hours (in PISA 2006, 2009, and 2012, we 

received approval to offer up to $50 for after-school and up to $75 for Saturday assessments in the main 

study).

All student incentives will be offered directly to the student, and only with the explicit permission of the 

school principal. Parents will be informed of the amount of the payment the students will receive in the 

consent form/letter in advance of the assessment. The payments will likely take the form of a personal check. 

This was the method used in the field test and main study in 2012 and it worked well on the whole.

NCES is requesting the same student incentive amounts in PISA 2015 as were offered to students in 2012, 

because the 2015 main study has the same session structure and added burden to accommodate a subsample 

of students returning for the financial literacy assessment as did PISA 2012. NCES again needs to achieve an 

acceptable student response rate while requiring additional time from students asked to return for the second 

session, which is required by the international guidelines for the financial literacy assessment. Providing an 

equal incentive amount for all students, despite a significant increase in burden for those selected for financial 
11

literacy, would be difficult to explain to schools and students and could hurt attendance in the second session.

NCES will administer two-sessions in all schools during the main study as prescribed by the international 

protocols. As in 2012, recruiting more schools and students in the national sample to avoid double sessions 

for students would be far more expensive than the second session incentive, and would not follow the 

international assessment design. The incentives described above will also be offered to schools, teachers, 

school coordinators, and students who participate as part of the state-level PISA samples. Schools, teachers, 

school coordinators, and students in Puerto Rico will not be offered incentives, per the decision of the Puerto 

Rico Department of Education.

A.10 Assurance of Confidentiality

Procedures for handling confidential aspects of the study that are used in PISA 2015 mirror those used in 

past administrations of PISA. Expertise in data security and confidentiality was a significant criterion in the 

selection of the PISA 2015 contractor. The plan for maintaining confidentiality includes signed confidentiality 

agreements and notarized nondisclosure affidavits obtained from all personnel who will have access to 

personal individual identifiers (PII). Also included in the plan is personnel training regarding the meaning of 

confidentiality, particularly as it relates to handling requests for information and providing assurance to 

respondents about the protection of their responses; controlled and protected access to computer files under 

the control of a single data base manager; built-in safeguards concerning status monitoring and receipt control 

systems; and a secured and operator-manned in-house computing facility.

To assure the confidentiality of student contact information collected to support the future validation study, 

student contact information will be collected, transmitted, and stored separately from responses to the test 

booklets and questionnaires. Contact information and responses to the 2015 instruments will be linked only 

by ID codes. Students’ names and contact information will never be made publically available.

Letters and other materials sent to parents and school administrators describe the voluntary nature of this 

survey. These materials include a brochure that describes the study and conveys the extent to which 

respondents and their responses are kept confidential (see Appendix A). The following statement will appear 

on the opening screen of the online questionnaires (the phrase “gather the data needed, and complete and 

review the information collection” will not be included for the student questionnaire as it does not apply to 

students):

U.S. participation in this study is sponsored by the National Center for Education Statistics (NCES), U.S. 

Department of Education. Your responses are protected by federal statute (20 U.S.C., § 9573). Your answers may 

be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose 

except as required by law.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of 

information unless such collection displays a valid OMB control number. The valid OMB control number for this 

voluntary information collection is 1850-0755. The time required to complete this information collection is 

estimated to average XX minutes per response, including the time to review instructions, search existing data 

resources, gather the data needed, and complete and review the information collection. If you have any suggestions 

for improving the form, or comments or concerns regarding the accuracy of the time estimate(s) or the status of 

your individual submission of this form, please write to: Program for International Student Assessment (PISA), 

National Center for Education Statistics, U.S. Department of Education, 1990 K Street, N.W., Room 8122, 

Washington, D.C. 20006.

OMB No. 1850-0755, Approval Expires xx/xx/20xx.

Data files, accompanying software, and documentation will be delivered to NCES at the end of the project.

No school or individual names or addresses will be included on these files or documentation.
12

NCES understands the legal and ethical need to protect the privacy of the PISA respondents and has 

extensive experience in developing data files for release that meet the government’s requirements to protect 

individually identifiable data from disclosure. The contractor will conduct a thorough disclosure analysis of 

the PISA 2015 data when preparing the data files for use by researchers, in compliance with 20 U.S.C., § 9573

and NCES statistical standards. Schools with high disclosure risk will be identified and, to ensure that 

individuals may not be identified from the data files, a variety of masking strategies will be used, including

swapping data and omitting key identification variables (i.e., school name and address) from both the publicand restricted-use files (though the restricted-use file will include an NCES school ID that can be linked to 

other NCES databases to identify a school); omitting key identification variables such as state or ZIP Code 

from the public-use file; and collapsing or developing categories for continuous variables to retain 

information for analytic purposes while preserving confidentiality in public-use files. Per NCES standards, the 

PISA 2015 data will be vetted by the IES Disclosure Review Board (DRB) prior to being approved for 

release.

A. 11 Sensitive Questions

PISA 2015 does not include questions usually considered to be of a highly sensitive nature, such as items 

concerning religion, substance abuse, or sexual activity. Several items in the background questionnaires may 

be considered sensitive by some of the respondents, such as the socioeconomic context of the school, 

parents’ education and occupation, family possessions, and students’ belongings. Research indicates that the 

constructs these items represent (i.e., socioeconomic status) are strongly correlated to academic achievement, 

and they have been used in the five previous cycles of PISA (2000, 2003, 2006, 2009, and 2012). Therefore, 

the items are deemed essential for the anticipated analyses and to retain consistency in planned comparisons 

with the international data.

A. 12 Estimates of Burden

This package requests approval for the PISA main study data collection, which is scheduled for OctoberNovember 2015. Burden estimates are shown in Table A. The time required for students to respond to the 

assessment (cognitive items) portion of the study, and associated directions, are shown in gray font and are 

not included in the totals because they are not subject to the PRA. Student, school (administrator), and 

teacher questionnaires are included in the requested burden totals. Recruitment and pre-assessment activities 

include the time involved in a school deciding to participate, completing teacher and student listing forms, 

distributing parent consent materials, and arranging assessment space.

The cost/burden to respondents for the PISA 2015 is calculated for the estimated time required of students 

and school staff (school administrator and school coordinators) to complete recruitment, pre-assessment, and 

assessment activities in 165 national schools. Burden and cost estimates in Table A also include estimates for 

Puerto Rico and two U.S. states.

3

In Puerto Rico, a paper-and-pencil version of the school questionnaire will 

be administered, while the financial literacy assessment and teacher questionnaire will not be administered.

For the 2015 main study data collection, the student response burden is based on a 30-minute background 

questionnaire for students taking the computer-based assessment and an additional 5 minute questionnaire 

for a subsample of students taking the national financial literacy assessment. The school response burden is 

based on a 45-minute questionnaire for school administrators and a 30-minute teacher questionnaire. Also, 

burden estimate for the already approved and begun school recruitment is being carried over and is included 

in this request because recruitment will continue through September 2015.

 

3 Two U.S. states have expressed interest in participating in PISA 2015. Burden estimates are included for two states in the event that both states decide to 

participate.
13

Table A. Burden estimates for PISA 2015 main study

Burden cost is calculated by combining the burden for the main study for the national, up to two U.S. states, 

and Puerto Rico samples. At an estimated $7.25 per hour (the 2009 Federal minimum wage) cost to students 

(5,990 hours), the total cost estimate for student burden is $43,426. At an estimated $50.00 per hour cost to 

administrators (1,969 hours), and an estimated $35.00 per hour cost for teachers (2,713 hours respectively), 

the dollar cost of the main study for all schools is $193,376 ($98,438 for school administrators, and $94,938 

for teachers).

A.13 Total Annual Cost Burden

Other than the burden associated with completing the PISA questionnaires and assessments (estimated above 

in Section A.12), the main study data collection imposes no additional cost to respondents.

A.14 Annualized Cost to Federal Government

The total cost to the Federal Government for conducting the PISA 2015 main study is estimated to be 

$4,278,915 over a 4-year period. This is based on the national data collection contract valued at $6,762,349

over five years, from January 2013 to January 2018, and includes costs for the national sample. These figures 

include all direct and indirect costs of the project.
14

A.15 Program Changes or Adjustments

There is an overall increase in burden because the last approval was for the PISA 2015 field test and 

recruitment activities, while this clearance request is for the main study data collection, requiring a larger 

number of schools, students, school administrators, and teachers in the sample than was needed for the field 

test. Also, this request carries over the already approved burden associated with PISA 2015 main study 

recruitment, because the recruitment activities are scheduled to continue through September 2015.

Regarding the content of PISA, there are some changes to PISA 2015 from the previous rounds of data 

collection. The main changes are that (1) the assessment will focus on science literacy during this cycle, (2)

collaborative problem solving is being assessed, (3) a teacher questionnaire is being administered for the first 

time, and (4) the assessment and questionnaires have been moved to a computer-based platform.

Additionally, there are minor changes in wording to some of the questionnaire items, and questions that 

focused on student attitudes toward mathematics or reading now focus on attitudes toward science.

A.16 Plans for Tabulation and Publication

For the main study in 2015, an analysis of the U.S. and international data will be undertaken to provide an 

understanding of the U.S. national results in relation to the international results. Based on proposed analyses 

of the international data set by ETS, and the need for NCES to report results from the perspective of an 

American constituency, a plan is being prepared for the statistical analysis of the U.S. national data set as 

compared to the international data set. Analysis of data will include examinations of the science, reading, and 

mathematics literacy; collaborative problem-solving; and financial literacy of U.S. students in relation to their 

international counterparts; and of the relationships between student performance and student and school 

background variables. The planned NCES report on PISA 2015 will be modeled on prior PISA reports, 

notably PISA 2012.

All reports and publications will be coordinated with the release of information from the international 

organizing body. Planned publications and reports for the PISA 2015 main study include the following:

General Audience Report. This report will present information on the level of science, mathematics, and 

reading achievement among students in the United States in comparison to their international peers, written 

for a non-specialist, general U.S. audience. This report will present the results of analyses in a clear and nontechnical way, conveying how U.S. students compare to their international peers, and what factors, if any, may 

be associated with the U.S. results. This report will be modeled on prior PISA reports, notably PISA 2012.

Survey Operations/Technical Report. This document will detail the procedures used in the main study 

(e.g., sampling, recruitment, data collection, scoring, weighting, and imputation) and describe any problems 

encountered and the contractor’s response to them. The primary purpose of the main study survey 

operations/technical report is to document the steps undertaken by the United States in conducting and 

completing the study. This report will include an analysis of non-response bias, which will assess the presence 

and extent of bias due to nonresponse. Selected characteristics of respondent students and schools will be 

compared with those of non-respondent schools and students to provide information about whether and how 

they differ from respondents along dimensions for which we have data for the nonresponding units, as 

required by NCES standards.

Electronic versions of each publication are made available on the NCES website. Schedules for tabulation and 

publication of PISA 2015 results in the United States are dependent upon receiving data files from the 

international sponsoring organization. With this in mind, the expected data collection dates and a tentative 

reporting schedule are as follows:
15

November 2014–September 2015 Prepare for the main study phase/ recruit schools

June 2015 Summer meeting for sampled schools

October 2015–November 2015 Collect main study data

August-October 2016 Receive final data files from international sponsors

August - December 2016 Produce General Audience Report and Survey Operations/Technical Report for 

the United States

A.17 Display OMB Expiration Date

The OMB expiration date will be displayed on all data collection materials.

A.18 Exceptions to Certification Statement

No exceptions are requested to the "Certification for Paperwork Reduction Act Submissions" of OMB Form 

83-I.