1

PROGRAM FOR INTERNATIONAL STUDENT ASSESSMENT 

(PISA) VALIDATION STUDY

REQUEST FOR OMB CLEARANCE

OMB# 1850-NEW v.1

SUPPORTING STATEMENT PART A

Submitted by:

National Center for Education Statistics

U.S. Department of Education

Institute of Education Sciences

Washington, DC

July 1, 2013

Revised August 9, 2013
2

TABLE OF CONTENTS

PREFACE................................................................................................................ 3

A. JUSTIFICATION ............................................................................................... 4

A.1 Importance of Information ............................................................................................... 4

A.2 Purposes and Uses of Data ............................................................................................... 7

A.3 Use of Information Technology ....................................................................................... 8

A.4 Efforts to Identify Duplication......................................................................................... 8

A.5 Minimizing Burden for Small Entities............................................................................. 8

A.6 Frequency of Data Collection ........................................................................................... 8

A.7 Special Circumstances of Data Collection....................................................................... 8

A.8 Consultations Outside NCES ........................................................................................... 8

A.9 Provision of Payments or Gifts to Respondents............................................................ 9

A.10 Assurance of Confidentiality ............................................................................................. 9

A.11 Sensitive Questions...........................................................................................................10

A.12 Estimates of Response Burden.......................................................................................10

A.13 Estimates of Cost to Respondents.................................................................................11

A.14 Cost to Federal Government ..........................................................................................11

A.15 Program Changes or Adjustments..................................................................................11

A.16 Publication Plans and Timetable.....................................................................................11

A.17 Display OMB Expiration Date .......................................................................................11

A.18 Exceptions to Certification Statement...........................................................................11

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL 

METHODS

B.1 Potential Respondent Universe and Sampling

B.2 Methods for Maximizing Response Rates

B.3 Tests of Procedures and Methods

B.4 Individuals Consulted on Study Design

Appendix A: Respondent Materials: Address Updates and Study Information

Appendix B: Draft Education and Skills Online Questionnaire Items
3

PREFACE

The Program for International Student Assessments (PISA) is an international assessment of 15-year-old

students sponsored by the Organization for Economic Cooperation and Development (OECD). In the 

United States, it is conducted by the National Center for Education Statistics (NCES), of the Institute of 

Education Sciences, U.S. Department of Education. PISA was first administered in 2000 and is conducted 

every three years. As part of PISA’s most recent administration, in 2012, PISA’s fifth cycle, a nationallyrepresentative sample of approximately 5,200 15-year-old students took the PISA assessments in the United 

States (OMB# 1850-0755).

PISA provides information about student performance in mathematics, science, and reading literacy, as well 

as problem-solving, in the United States and more than 65 other participating countries and education 

systems. PISA is designed to assess how well students nearing the end of compulsory schooling are prepared 

for the challenges of further education and the workforce. To do so, PISA measures students’ literacy

(knowledge and skills to solve problems in real-life contexts) and collects information about students’ home 

background, attitudes, and experiences in and out of school through a questionnaire completed by students. 

Furthermore, school principals of the participating students complete a questionnaire about school policies 

and practices, providing additional context for the student measures.

NCES proposes to conduct a follow-up study with students who participated in PISA 20121to learn how 

performance on PISA relates to subsequent outcomes and skills of young adults. The follow-up study—

referred to in materials to potential respondents as the PISA Young Adult Follow-Up Study, and in this 

clearance request in short as the PISA Validation Study — will provide information about how students’ skills 

and experiences at age 15, collected through PISA, relate to subsequent literacy, numeracy, and problemsolving skills, as well as educational attainment, education and work experiences, skills used in daily life, and 

aspects of health and well-being. In fall 2015, when these students will be 18 years of age, they will be asked 

to take the web-based version of the OECD’s Program for the International Assessment of Adult 

Competencies (PIAAC) assessment and background questionnaire—the Education and Skills Online 

Assessment (ESO).

This submission is for initial communication with the PISA 2012 sample respondents who provided their 

contact information during their participation in PISA 2012. As part of the initial communication, they will be 

asked to update their contact information (referred to as the address update) and will receive information 

about the upcoming PISA Validation Study. This approval request (currently in draft form) describes the 

study rationale and design and includes draft materials to support the initial contact and address updates (see 

Appendix A). The draft ESO background questionnaire items are included in Appendix B. In 2014, another

request will be submitted for approval of intensive recruitment of sample respondents and data collection for 

both the field trial and the main study. At that time, the final version of the ESO questionnaire will be 

provided.

 

1 The sample of ~5,200 students took the PISA mathematics, science, and reading literacy assessment. PISA 2012 also included an assessment of financial literacy 

given to an additional ~1,100 students. In order to focus the Validation Study on the main PISA domains, the proposed study will not include the students who 

took financial literacy except in the field trial. Also, in three states (Connecticut, Florida, and Massachusetts), additional samples of ~1,500 students per state took 

the PISA mathematics, science, and reading literacy assessment so that these states could obtain state-level PISA results (these three states provided funding for 

their supplemental samples). At this time, students in the state samples are not included in the Validation Study plans, although should states wish to partner with 

NCES to conduct a follow-up study with the students in the PISA 2012 state samples, the proposed study may be revised to include those students, in which case

a change request would be submitted.
4

A. JUSTIFICATION

A.1 Importance of Information

As part of a continuing cycle of international education studies, the United States, through the National 

Center for Education Statistics (NCES), is participating in several international assessments and surveys. The 

Program for International Student Assessment (PISA), sponsored by the Organization for Economic 

Cooperation and Development (OECD), is one of these studies (OMB# 1850-0755). 

In light of the growing concerns related to international economic competitiveness, the changing face of our 

workplace, and the expanding international marketplace in which we trade, knowing how our students and 

adults compare with their peers around the world has become an even more prominent issue than ever 

before. Nationwide, interest in understanding what other nations are doing to further the education 

achievement of their populations has increased beyond simple comparisons. Data at critical points during the 

education career of our students help inform policymakers in their efforts to guide and restructure the 

American education system. These critical points may occur during primary, secondary, or tertiary education, 

as well as extending into adult education and training programs. Consequently, generating comparative data 

about students in school, at the end of schooling, and about adults in the workplace and in the community 

has become an important focus for NCES.

PISA is part of the larger international program that NCES has actively participated in through collaboration 

with, and representation at, the OECD, the Asia-Pacific Economic Cooperation (APEC), and the 

International Association for the Evaluation of Educational Achievement (IEA). Collaboration with Statistics 

Canada, Eurostat, and ministries of education throughout the world helps to round out the portfolio of data 

NCES compiles. The United States has participated in PISA since its first administration in 2000. 

PISA measures students' knowledge, skills, and competencies primarily in three subject areas –mathematics, 

science, and reading literacy. The overall strategy is to collect in-depth information on student capabilities in 

one of these three domains every three years so that detailed information on each becomes available every 

nine years. During each three-year survey cycle, the major focus is on one content domain, with a minor focus 

on the other two content domains. The major focus for the data collection in 2012 was on mathematics 

literacy, with a minor focus on science and reading. PISA 2012 also included an assessment of problemsolving and financial literacy.

PISA is designed to measure “how well students, at age 15, are prepared to meet the challenges they may 

encounter in future life” (OECD 2009, pg. 9). PISA does this by identifying, with input from participating 

countries and subject-matter experts, the key competencies that are believed to contribute to students’ future 

success, and by measuring students’ knowledge and skills in these areas. In short, PISA is designed to assess 

how well students nearing the end of compulsory schooling are prepared for further education or entry into 

the workforce and, more fundamentally, to contribute to society as functioning young adults.

Unlike other large-scale international assessments that are based on school curricula, such as the Trends in 

International Mathematics and Science Study (TIMSS) or the Progress in International Reading Literacy Study 

(PIRLS), PISA assessments are designed to reflect what students have learned in and out of school and to 

measure students’ ability to apply that knowledge to real-world situations. For this reason, PISA has a unique 

validity challenge. TIMSS and PIRLS assessments can be validated by evaluating how well they cover 

mathematics, science, and reading curricula. However, comparing PISA measures with content standards or 

school curricula does not provide evidence that PISA assesses the competencies that matter for students’ 
5

“future success” and the “challenges they may encounter in the future.” To validate PISA measures, it is 

necessary to examine whether the competencies assessed by PISA predict future success.

Given the prominence of PISA in the United States, it is of policy interest to better understand how well 

PISA measures students’ preparedness for further education and the workforce. PISA is widely cited as 

evidence that U.S. students are not as prepared for the global economy as their peers in other highperforming countries, and such claims rest on the assumption that PISA is measuring the skills that students 

will need to succeed in the 21st century and the global economy.

NCES is requesting approval to conduct a follow-up study with students who participated in PISA 2012 to 

learn how performance on PISA relates to subsequent outcomes and skills of young adults. The follow-up 

study — referred to in materials to potential respondents as the PISA Young Adult Follow-Up Study, and in 

this clearance request in short as the PISA Validation Study — will provide information about how students’ 

skills and experiences at age 15, collected through PISA, relate to subsequent literacy, numeracy, and 

problem-solving skills, as well as educational attainment, education and work experiences, skills used in daily 

life, and aspects of health and well-being. The web-based version of the OECD’s Program for the 

International Assessment of Adult Competencies (PIAAC) will be used to measure young adult skills, 

outcomes, and experiences in the PISA Validation Study. The PIAAC assessment, administered in 25 

countries in 2011, focuses on cognitive and workplace skills needed for successful participation in 21st-

century society and the global economy (OECD 2012). Specifically, PIAAC measures relationships between 

individuals' educational background, workplace experiences and skills, occupational attainment, use of 

information and communications technology, and cognitive skills in the areas of literacy, numeracy, and 

problem solving.

The sample for the PISA Validation Study will consist of the approximately 4,700 15-year-old students from 

the PISA 2012 national sample who took the PISA mathematics, reading, and science test and completed a 

Student Information Form providing their contact information. PISA 2012 recruitment materials, including 

materials for parents, stated that students supplying contact information may be contacted by NCES for a 

future study. In fall 2015, when these students will be 18 years of age, they will be asked to take the webbased version of the PIAAC assessment and background questionnaire—the Education and Skills Online 

Assessment (ESO). The ESO is described below. A field trial of the Validation Study will be conducted in 

fall 2014 with a sample of 200 PISA 2012 students who provided contact information but took financial 

literacy test booklets and are not the target for the PISA Validation Study.

In 2011, PIAAC was administered for the first time (although it builds on previous international adult 

assessments) to a national sample of adults 16 to 65 years old. It examined the adults’ literacy and skills in the 

information age, focusing on what are deemed key skills for individuals to participate successfully in the 

economy and society of the 21st century. It assessed adults’ basic reading skills, reading literacy, numeracy, 

and problem solving in technology-rich environments (on or with a computer), and measured their ability to

use computer and web applications to find, gather, and use information, and to communicate with others. 

PIAAC used a “Job Requirements Approach” to ask employed adults about the types and levels of a number 

of specific skills used in the workplace, including not only the use of reading and numeracy skills on the job, 

but also the use of physical skills (e.g., stamina and manual dexterity), people skills (e.g., public speaking, 

negotiating, working in a team), and information technology skills (e.g., using spreadsheets, writing computer 

code). PIAAC also asked about the adults’ background, work, and educational experiences, including the 

requirements of their main job in terms of the intensity and frequency of the use of the aforementioned skills. 

The OECD has developed a web-based version of the PIAAC assessment and background questionnaire—

the Education and Skills Online Assessment (ESO) — that will be available, as of 2014, for use by countries, 
6

institutions, and individuals to support research and workforce development and training efforts. The ESO 

assessment, which is currently being piloted in multiple countries by the OECD, will be equated to PIAAC, 

allowing for international benchmarking. The ESO questionnaire is based on that used in PIAAC, with many 

of the same background items, plus additional optional modules that have been developed. 

In the PISA Validation Study, the ESO assessment and background questionnaire will be administered to 

sample respondents when they are 18 years old, three years after they took PISA. The assessment takes on 

average 90 minutes to complete and the background questionnaire approximately 30 minutes.

The ESO assessment measures a set of cognitive and non-cognitive skills that individuals need for full 

participation in modern societies. These skills and knowledge include being able to understand and use 

printed and electronic texts, reason with numbers, and solve problems using information and communication 

technologies (see exhibit A-1, below).

The ESO questionnaire, in turn, asks about personal and home background and education and workforce 

experiences, and includes optional modules related to (i) work/training history and skills transfer, (ii) career 

interests and intentionality, (iii) subjective well-being and health, and (iv) behavioral performance 

competencies. It is possible to use one or more of the modules developed by the OECD, but it is not 

possible to add or remove items or edit items within the modules. One or more of modules ii, iii, and iv may 

be used in the PISA Validation Study. Respondents will complete the instruments online. Appendix B 

includes the draft versions of items in the core part of the background questionnaire and all modules; these 

are the items currently being piloted by the OECD, many of which are based on PIAAC questionnaires items 

or are existing scales used in other surveys.

Exhibit A-1. Summary of the cognitive domains in the OECD Education and Skills Online (ESO) Assessment

Literacy Numeracy Problem solving in technologyrich environments 

Definition

Ability to understand, evaluate, use 

and engage with written texts to 

participate in society, achieve one’s 

goals, and develop one’s knowledge 

and potential. 

Ability to access, use, interpret and 

communicate mathematical 

information and ideas in order to 

engage in and manage the 

mathematical demands of a range of 

situations in adult life. 

Ability to use digital technology, 

communication tools and networks 

to acquire and evaluate information, 

communicate with others, and 

perform practical tasks. 

Content

Texts are characterized by 

Medium: 

 Print-based 

 Digital 

Format: 

 Continuous or prose texts 

(narration, argumentation, or 

descriptions) 

 Non-continuous or document 

texts (tables, lists, graphs) 

 Mixed texts (combination of 

prose and document elements) 

 Multiple texts (juxtaposition or 

linking of independently 

generated elements) 

Mathematical content, information 

and ideas: 

 Quantity and number 

 Dimension and shape 

 Pattern, relationships, change 

 Data and chance 

Representations of mathematical 

content: 

 Objects and pictures 

 Numbers and symbols 

 Diagrams, maps, graphs, tables 

 Texts 

 Technology-based displays 

Technology: 

 Hardware devices 

 Software applications 

 Commands and functions 

 Representations (text, graphics, 

video) 

Nature of problems: 

 Intrinsic complexity (number of 

steps, alternatives required for 

solution, complexity of 

computation and/or 

transformation, number of 

constrains) 

 Explicitness of the problem 

statement (largely unspecified or 

described in detail 
7

Literacy Numeracy Problem solving in technologyrich environments 

Cognitive 

Strategies

 Access and identify 

 Integrate and interpret (relating 

parts of text to one another) 

 Evaluate and reflect on 

 Identify, locate or access 

 Act upon and use (order, count, 

estimate, compute, measure, 

model) 

 Interpret, evaluate and analyze 

 Communicate 

 Setting goals and monitoring 

progress 

 Planning 

 Acquiring and evaluating 

information 

 Using information 

Context

 Personal 

 Work-related 

 Community 

 Education 

 Everyday life 

 Work-related 

 Society & Community 

 Education 

 Personal 

 Work-related 

 Community 

Source: http://www.oecd.org/site/piaac/ENGLISH_Brochure%20Education%20and%20Skills%20Online.pdf. 

A.2 Purposes and Uses of Data

PISA is intended to be an assessment at the end of compulsory education of how well students are prepared 

for further education or entry into the workforce and to contribute to society as functioning young adults. 

The PISA subject matter assessments are not developed by canvassing the curricula offered in participating 

countries, as is done in other large-scale international assessments. The PISA assessment frameworks and 

items are developed by conferring with subject matter experts around the world and representatives of 

participating countries about the mathematics, reading, and science competencies required for 15-year-olds to 

successfully transition to further education, the workforce, and otherwise as functional contributing members 

of society. For 2012, in which the mathematics framework was revised, the mathematics framework was also 

circulated internationally among representatives of employers and postsecondary institutions for their 

comments on the appropriateness of aspects of the PISA mathematics framework.

Because of its emphasis on assessing what is thought to be important for later success rather than assessing 

what has been taught, PISA must turn to sources of evidence other than participating countries’ curricula in 

order to check that it is assessing what it intends to assess. Currently, the main source for PISA for this sort 

of validity checking is in the form of experts drawn from across participating countries, as described earlier.

Also, two other countries have performed longitudinal studies based on PISA. A longitudinal study in Canada 

has followed 30,000 students who participated in PISA 2000, gathering information every two years on their 

educational experiences, transitions, and attainment and employment outcomes (OECD 2010). A similar 

study drawing on a smaller sample was conducted in Denmark (Schleicher 2007).

What is lacking is an empirical linkage between PISA and the sorts of skills and outcomes required for 

successful participation in adult life. The proposed study will provide data on how performance on PISA, at 

age 15, relates to literacy, numeracy, and problem-solving skills; educational attainment and education and 

workforces experiences; skills use in daily life; and other non-cognitive outcomes at age 18. For example, 

what thresholds of performance on PISA are associated the ability to successfully navigate the literacy, 

numeracy, and problem-solving demands of daily life in the adult world? What thresholds are associated with 

high school graduation and successful transition to higher education or entry-level workforce? How does 

performance on PISA relate to measures of subjective well-being at age 18, such as life satisfaction and 

healthy habits?

By linking performance on PISA to subsequent outcomes in young adulthood, the study will provide the sort 

of validity evidence needed to understand how well PISA is measuring the skills needed for successful 

participation in adult life, as it purports to do. The study will also strengthen our understanding of U.S. 

performance on PISA and its implications for U.S. college and career readiness and for the skills of our future 
8

workforce. This information can, in turn, also be used to inform the further development of PISA and future 

surveys of adult skills.

A.3 Use of Information Technology

Study instruments will be administered through the Internet. Respondents will be invited to log on to a secure 

website to take the assessment and questionnaire. The use of web-based instruments reduces the burden 

required for data collection, scoring, and data processing.

A.4 Efforts to Identify Duplication

Comparable U.S. data are not available from other data sources; nor would a PISA follow-up study make any 

data redundant. The purpose of this study is to look at the relationship of performance on PISA to 

performance on similar domains and to other outcomes after a period of time, to validate PISA’s measures. 

No other study in the United States. has collected follow-up data from or about PISA students so this would 

be a unique source of data to accomplish this objective. The current study could produce some information 

that is similar to a subset of information from the High School Longitudinal Study (HSLS), which follows a 

similar age group, however HSLS would not allow for validation of PISA measures, which is the goal of this 

study, and does not provide data directly comparable with data from other countries.

A.5 Minimizing Burden for Small Entities

This study will not impose burden on small businesses or small entities. 

A.6 Frequency of Data Collection

The main data collection will take place in fall 2015. A field trial of procedures will take place in 2014. 

Although there are currently no plans for a second follow-up study with the PISA 2012 students, NCES may 

decide at a later date to re-contact the sample to collect additional data at another point in time on education 

and workforce experiences and skills in order to evaluate PISA measures against longer term outcomes.

A.7 Special Circumstances of Data Collection

No special circumstances of data collection are anticipated.

A.8 Consultations Outside NCES

NCES has sought input from a wide range of individuals with knowledge of PISA and PIAAC. Both PISA 

and PIAAC are sponsored by the OECD and are coordinated internationally by a consortium of contractors 

responsible for various aspects of the studies. Key persons from the OECD and key contracting 

organizations who are involved in the international design, development, and operation of PISA and PIAAC

follow. 

Organization for Economic Cooperation and Development

Andreas Schleicher, Indicators and Analysis Division

2, rue André Pascal, 75775 Paris Cedex16, FRANCE

William Thorn, PIAAC

2, rue André Pascal, 75775 Paris Cedex16, FRANCE
9

Educational Testing Service

Irwin Kirsch, Project Director, ETS Corporate Headquarters

660 Rosedale Road, Princeton, NJ 08541 USA

A.9 Provision of Payments or Gifts to Respondents

As part of the planned efforts to reach response rate goals, NCES proposes giving sampled respondents the 

same incentive used for the PIAAC Main Study: $50 cash card for the respondent’s time and effort spent 

answering the survey. The amount of burden is equivalent to that of the PIAAC respondents. NCES 

received OMB approval to offer a $50 incentive for the PIAAC Main Study after the results of an incentive 

experiment during the 2010 PIAAC Field Test indicated that $50 improved responses rates compared with 

lower amounts. 

. Due to the unclustered sample, it would be prohibitively expensive to conduct an in-person data collection. 

A web approach has been designed to administer the ESO to PISA 2012 students. During recruitment, 

respondents will be supplied with an unloaded cash card that will be loaded upon their completion of the 

ESO assessment and questionnaire. The card can be used by the respondent anywhere a MasterCard can be,

once remuneration has been added to it. Until the survey is completed, the card acts as a visual incentive to 

the respondent to attempt to complete the survey.

A.10 Assurance of Confidentiality

Procedures for handling confidential aspects of the study that will be used in PISA 2015 will mirror those 

used in past administrations of PISA. Expertise in data security and confidentiality was a significant criterion 

in the selection of the PISA 2015 contractor. The plan for maintaining confidentiality includes signed 

confidentiality agreements and notarized nondisclosure affidavits obtained from all personnel who will have 

access to individual identifiers. Also included in the plan is personnel training regarding the meaning of 

confidentiality, particularly as it relates to handling requests for information and providing assurance to 

respondents about the protection of their responses; controlled and protected access to computer files under 

the control of a single data base manager; built-in safeguards concerning status monitoring and receipt control 

systems; and a secured and operator-manned in-house computing facility.

The physical and/or electronic transfer of personally identifiable information (particularly names and 

addresses) will be limited to the extent necessary to perform project requirements. This limitation includes 

both internal transfers (e.g., transfer of information between agents of Westat) and external transfers (e.g., 

transfers between Westat and NCES, or between Westat and another government agency or private entity 

assisting in data collection). All data files constructed to conduct the study will be maintained in secure 

network areas at Westat. These files will be subject to Westat’s regularly scheduled backup process. Backups 

are stored in secure facilities on site as well as off site. These data are stored and maintained in secure network 

and database locations where access is limited to those Westat staff who are specifically authorized access. 

Access is only granted once a staff member is assigned to the project and has completed the NCES Affidavit 

of Non-disclosure. Identifiers are maintained in files required to conduct survey operations that are physically 

separate from other research data and that are accessible only to sworn agency and contractor personnel.

All information identifying the individual respondents will be kept confidential, in compliance with the law 

(20 U.S. C. § 9573), which states that:

(c) (2) “No person may
10

i. use any individually identifiable information furnished under the provisions of this section for 

any purpose other than a research, statistics, or evaluation purpose under this subchapter; 

ii. make any publication whereby the data furnished by any particular person under this 

subchapter can be identified; or

iii. permit anyone other than the individuals authorized by the Director to examine the individual 

reports.”

The laws pertaining to the collection and use of personally identifiable information are clearly communicated 

in correspondence with participants, per NCES requirements (see materials in Appendix A). Materials will 

carry a statement addressing confidentiality as follows:

NCES is authorized to conduct this study under the Education Sciences Reform Act 

of 2002 (ESRA 2002, 20 U.S.C., § 9543). Participation is voluntary. Data collected 

are used only for statistical purposes and may not be disclosed or used, in identifiable 

form for any other purpose except as required by law (ESRA 2002, 20 U.S.C., § 

9573). Individual responses will be combined with those from other participants to 

produce summary statistics and reports.

Westat will deliver data files, accompanying software, and documentation to NCES at the end of the main 

study. Neither names nor addresses will be included on any data file. 

A. 11 Sensitive Questions

It is not anticipated that the survey will include questions usually considered to be of a highly sensitive nature, 

such as items concerning religion, substance abuse, or sexual activity.

A. 12 Estimates of Response Burden 

Table A-1. Estimated burden to respondents*

Activity Sample 

size

Estimated 

response 

rate

Number of 

Respondents

Number of 

Responses

Estimated average 

burden (minutes)

Total burden 

(hours)

Address update requests - main 

study sample (5) 4,729 1.00 4,729 4,729 5 min per request25 min total 1,970

Address update requests – field 

trial sample (3) 1,081 1.00 1,081 1,081 5 min per request15 min total 270

Total burden - this request 5,810 5,810 2,240

Field trial

 Communications 235 0.85 200 200 10 33

 Questionnaire 235 0.85 200 200 30 100

 Assessment 235 0.85 200 200 90 300

Data collection

 Communications 3,447 0.85 2930 2,930 10 488

 Questionnaire 3,447 0.85 2930 2,930 30 1,465

 Assessment 3,447 0.85 2930 2,930 90 4,395

Total burden - data collections

(communications and questionnaire) 3,130 6,260 2,086

*Total burden in the current request is for address updates only (estimated at 5 minutes each). Burden estimates for the field trial and main study 

communications and data collection (shown in gray font) are provided for information purposes and will be part of a Supporting Statement Part A

in subsequent approval requests for field trial and main study recruitment and data collections.
11

Table A-1 shows the estimated respondent burden for address updates until participation in the survey, as 

well as estimated respondent burden for field trial and main study recruitment and data collection. This 

submission requests approval of respondent burden for the address updates only. Five contacts are estimated 

for the main study sample, and three contacts for the field trial sample. Each contact has an estimated average 

burden of 5 minutes, for a total average burden of 25 minutes for the main sample and 15 for the field trial 

sample. The total burden in this request is 2,241 hours.

At an estimated $7.25 per hour (federal minimum wage), cost to respondents for the total burden associated 

with the address updates (2,241 hours) is $16,747.

A.13 Estimates of Cost to Respondents

Other than the burden associated with completing the address updates and study instruments, the study 

imposes no additional cost to respondents.

A.14 Cost to Federal Government

The cost to the Federal Government for conducting the validation study is estimated to be $1,945,602 over a 

four-year period, based on the cost for the Validation Study task in the PISA 2015 contract. This includes 

tracing and tracking sample respondents, data collection, use of the OECD’s ESO instruments, and analysis 

and reporting. These figures include all direct and indirect costs of the project.

A.15 Program Changes or Adjustments

The PISA Validation Study is a new data collection effort.

A.16 Publication Plans and Timetable

NCES will release a general audience report and technical/operations report, along with restricted-use data 

files. Electronic versions of each report will be made available on the NCES website. The expected data 

collection dates and a tentative reporting schedule are as follows:

Date Activity

Fall 2012 Collect contact information from PISA 2012 students (completed)

Fall 2013 – September 2015 Establish initial contact and conduct semi-annual contact with respondents for address 

updates

Fall 2014 Conduct field trial of procedures

September-December 2015 Web-based main study data collection 

2016-2017 Reporting: general audience report; survey operations report; restricted-use data-files

A.17 Display OMB Expiration Date

The OMB expiration date will be displayed on all data collection materials. 

A.18 Exceptions to Certification Statement

No exceptions are requested to the "Certification for Paperwork Reduction Act Submissions" of OMB Form 

83-I.
12

References

Organization for Economic Cooperation and Development (2012). Literacy, Numeracy and Problem Solving in 

Technology-Rich Environments - Framework for the OECD Survey of Adult Skills. Paris: Author.

Organization for Economic Cooperation and Development (2010). Pathways to Success: How Knowledge and Skills 

at Age 15 Shape Future Lives in Canada. Paris: Author.

Organization for Economic Cooperation and Development (2009). PISA 2009 Assessment Framework - Key 

Competencies in Reading, Mathematics and Science. Paris: Author.

Organization for Economic Cooperation and Development and Statistics Canada (2005). Learning a Living: 

First Results from the Adult Literacy and Life Skills Survey. Paris: OECD.

Organization for Economic Cooperation and Development and Statistics Canada (1997). Literacy Skills for the 

Knowledge Society: Further Results from the International Adult Literacy Survey. Paris: OECD.

Schleicher, A. (2007). Can competencies assessed by PISA be considered the fundamental school knowledge 

15-year-olds should possess? Journal of Educational Change, 8: 349-357 (published online July 24, 2007).