FY 2017 Annual Performance

Report and FY 2019 Annual

Performance Plan
U.S. Department of Education

Betsy DeVos

Secretary

February 12, 2018

This report is in the public domain. Authorization to reproduce it in whole or in part is granted. 

While permission to reprint this publication is not necessary, the citation should be: U.S. 

Department of Education, Fiscal Year 2017 Annual Performance Report and Fiscal Year 2019 

Annual Performance Plan: Washington, D.C., 2018. 

Notice of Availability of Alternate Formats

This report is available at http://www.ed.gov/about/reports/annual/index.html. On request, 

the report also is available in alternative formats, such as Braille, large print, or compact disk. 

For more information, please contact our Alternate Format Center at 202-260-0852 or the 504 

coordinator via e-mail at om_eeos@ed.gov. 

To become connected to the Department through social media, please visit the Department’s 

website at www.ed.gov. Our Twitter page is at @usedgov, and our blog is at Homeroom.

Notice to Limited English Proficient Persons

Notice of Language Assistance: If you have difficulty understanding English, you may request 

language assistance services, free of charge, for this Department information by calling 1-800-

USA-LEARN (1-800-872-5327); (TTY: 1-800-877-8339), or by e-mailing us at 

Ed.Language.Assistance@ed.gov. 

Please submit your comments and questions regarding this report, and any suggestions to 

improve its usefulness to APP_APRComments@ed.gov or write to:

U.S. Department of Education

Performance Improvement Officer 

400 Maryland Ave, SW

Washington, DC 20202
FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 3

Table of Contents

Table of Contents........................................................................................................................ 3

About the Department ................................................................................................................ 4

Operating Structure .................................................................................................................... 5

Overview...................................................................................................................................... 6

FY 2017 Performance ................................................................................................................. 6

FY 2018-22 Strategic Goals and Strategic Objectives...........................................................13

The Department’s Agency Priority Goals (APGs)..................................................................14

Cross-Agency Priority (CAP) Goals ........................................................................................15

Strategic Goal 1.........................................................................................................................16

Goal 1 Discretionary Resources..............................................................................................26

Major Discretionary Programs and Activities Supporting Goal 1.............................................26

Strategic Goal 2.........................................................................................................................27

Goal 2 Discretionary Resources..............................................................................................37

Major Discretionary Programs and Activities Supporting Goal 2.............................................37

Strategic Goal 3.........................................................................................................................39

Goal 3 Discretionary Resources..............................................................................................44

Major Discretionary Programs and Activities Supporting Goal 3.............................................44

Strategic Goal 4.........................................................................................................................45

Goal 4 Discretionary Resources..............................................................................................51

Major Discretionary Programs and Activities Supporting Goal 4.............................................51

Major Management Priorities and Challenges .......................................................................53

Appendices................................................................................................................................57

Appendix A: FY 2017 Performance Metrics ............................................................................58

Appendix B: Data Validity and Verification for FY 2017 Annual Performance Report.............74

Appendix C: Data Validity and Verification for FY 2019 Performance Metrics ........................93

Appendix D: Programs by Goal.............................................................................................113

Appendix E: Summary of Performance Evaluations Conducted During FY 2017 and Expected 

During FY 2018–19 ...............................................................................................................117

Appendix F: Glossary of Acronyms and Abbreviations .........................................................161
DEPARTMENT OF EDUCATION MISSION

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 4

About the Department 

The U.S. Department of Education’s mission is to promote student achievement and preparation 

for global competitiveness by fostering educational excellence and ensuring equal access.

The Department accomplishes its mission by administering programs that provide services 

throughout an individual’s entire lifespan–from early intervention services to employment 

programs. Many of these programs provide grants to state educational agencies (SEAs) or local 

educational agencies (LEAs) and support students and families from vulnerable populations, 

including children with disabilities and those from disadvantaged backgrounds. These programs 

also provide grants and loans to postsecondary students, and facilitate research that examines 

ways that states, schools, districts, and postsecondary institutions can improve America’s 

education system. In addition, the Department fulfills its mission through the enforcement for 

civil rights laws that provide equal access to Department programs for all individuals. 

The figure below reflects the operating structure of the U.S. Department of Education used in 

Fiscal Year (FY) 2017. Interactive and text versions of the operating structure of the Department 

are available online.
DEPARTMENT ORGANIZATION

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 5

Operating Structure1

 

1 Source https://www2.ed.gov/about/offices/or/index.html.
PERFORMANCE REPORT

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 6

Overview

The FY 2017 Annual Performance Report (APR) and FY 2019 Annual Performance Plan (APP), 

required by the Government Performance and Results Act Modernization Act of 2010, 

accompanies the Department’s Budget submission and links performance goals with resources 

for achieving targeted levels of performance. The APR reflects performance against the 

Department’s FY 2014-18 Strategic Plan. The APP identifies implementation strategies and 

performance measures to accomplish the strategic goals and objectives represented in the 

Department’s FY 2018–22 Strategic Plan. 

The U.S. Department of Education’s mission is to promote student achievement and preparation 

for global competitiveness by fostering educational excellence and ensuring equal access. This 

mission is manifested in the Department’s efforts to continually improve the educational 

environment for all students, addressing their educational needs. The Department’s National 

Center for Education Statistics (NCES) estimates that 50.7 million students were attending 

public elementary and secondary schools in 2017, with a projected 35.6 million in 

prekindergarten through grade 8 and a projected 15.1 million in grades 9 through 12. An 

additional 5.2 million students are projected to attend private elementary and secondary 

schools. NCES predicts that the total pre-K–12 enrollment will continue to grow to an all-time 

high of 56.8 million by 2026, indicating the increasing need for quality education throughout the 

United States. 

FY 2017 Performance

Education is primarily a state and local responsibility in the United States. It is states and 

communities, as well as public and private organizations of all kinds, that establish schools and 

colleges, develop curricula, and determine requirements for enrollment and graduation. As 

such, the Department is committed to supporting SEA, LEAs and other education institutions to 

help ensure every student in America has an equal opportunity for a great education. 

In March 2017, the Department released a revised consolidated state plan template to support 

states in meeting the requirements of the Elementary and Secondary Education Act of 1965 

(ESEA), as amended by the Every Student Succeeds Act (ESSA). The Department worked with 

SEAs, and other state and local stakeholders, to develop a revised template that was structured 

to reduce burden and promote innovation, flexibility, transparency, and accountability, while 

maintaining essential protections for all students. The revised template asked states only to 

provide details on their plans in areas (a) explicitly required by law and (b) deemed absolutely 

necessary for consideration of such a plan, consistent with ESEA section 8302(b)(3), leveraging 

input of states, local educators, and parents. State plans were submitted to the Department and

peer-reviewed. 

In FY 2017, the Department also focused on promoting evidence-based decision making with 

the intention to support states and districts in using and building evidence effectively. The 

Department published revised evidence definitions and related selection criteria for competitive 

grant programs in the Education Department General Administrative Regulations that align with 

ESSA, disseminated non-regulatory guidance on evidence in ESSA, Using Evidence to 

Strengthen Education Investments; awarded $16 million to support rigorous evaluations and 

researcher-practitioner partnerships focused on state and local education priorities. 

With regard to postsecondary education, the passage of the FY 2017 spending bill restored 

year-round Pell grants, and the Department announced that these grants would become 

available to college students beginning July 1, 2017. The Department recommended that unless 

a student had remaining eligibility from the 2016–17 award year, institutions should award Pell 
PERFORMANCE REPORT

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 7

Grant funds for this past summer out of the 2017–18 award year since the additional funding will 

be available later in the year (e.g., spring or summer of 2018). The change allows an eligible 

student to receive up to 150 percent of the student’s scheduled Pell Grant for an award year 

beginning with the 2017–18 award year, giving hundreds of thousands of students more 

resources to finish their coursework in a timeframe that meets their individual needs. 

In FY 2017, the Department also paused on two postsecondary regulations—Borrower Defense 

to Repayment, concerning forgiveness of student loan debt, and Gainful Employment, 

concerning educational programs that prepare students for gainful employment in a recognized 

occupation. Two negotiated rulemaking committees have been established to rethink these two 

higher education regulations, with the intent to develop fair, effective, and improved regulations 

to protect individual borrowers from fraud, ensure accountability across Institutions of Higher 

Education (IHEs), and protect taxpayer interests. It is the Department’s aim to protect students 

from predatory practices while also providing clear, fair, and balanced rules for colleges and 

universities to follow. 

The Department continued to improve the information and data it makes available to parents, 

students, and education stakeholders. A few examples include: 

• The Department’s College Scorecard, which supports postsecondary students by 

providing the public with clear, easily accessible information on college performance. 

• The Department’s National Assessment of Educational Progress (NAEP), the largest 

nationally representative and continuing assessment of student knowledge in various 

subject areas.

• The Department’s InformED initiative, which is intended to transform how the 

Department makes information available—and actionable—for internal users and for the 

public through the identification and development of open data initiatives. 

Finally, the Department continued to strengthen its internal operations throughout FY 2017. In 

response to President Trump’s Executive Order 13777, Enforcing the Regulatory Reform 

Agenda, the Department established a Regulatory Reform Task Force that catalogued over 150 

regulations and more than 1,700 items of policy guidance at the Department. The task force, 

comprised of agency political appointees and career staff, provided recommendations on which 

regulations and guidance documents to repeal, modify, or keep in an effort to ensure those 

items that remain protect students while giving states, institutions, teachers, parents, and 

students the flexibility to improve student achievement. 

Also, in response to Executive Order 13781, Comprehensive Plan for Reorganizing the 

Executive Branch, the Department submitted an Agency Reform Plan to the Office of 

Management and Budget (OMB). Department staff continue to assess reform factors that 

include: new activities or functions the Department should initiate; ways the agency can be more 

efficient in meeting the needs of students, families, and education partners; activities or 

functions the Department should consider combining or modifying; agency activities or functions 

that duplicate what others are doing; and how the Department could best deliver education 

services or products to students and educators. 

See Appendix A for detailed information on the FY 2017 performance measures.

The Department uses quarterly performance reviews, targeted strategic initiatives, and outreach 

to leaders and stakeholders to assess progress and garner engagement toward achieving 

strategic goals and outcomes. Looking to the future towards the FY 2018–22 Strategic Plan, the 

Department plans to focus in four key areas: (1) supporting state and local efforts to improve 

learning outcomes for all P–12 students in every community; (2) expanding postsecondary 
PERFORMANCE REPORT

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 8

education opportunities and improving outcomes to foster economic opportunity and informed, 

thoughtful and productive citizenry; (3) strengthening the quality, accessibility and use of 

education data through better management, increased privacy protections, and transparency; 

and (4) reforming the effectiveness, efficiency and accountability of the Department. 

Through its FY 2018-22 Strategic Plan, the Department is focusing on the students it serves. 

Students have unique needs, and the Department must strive to align and tailor its programs to 

meet their needs. When thinking about how education delivery and the support the Department 

provides to parents, teachers, school leadership, districts, institutions, and states must change, 

it is important to reflect on the current state of education in the United States. The following 

graphics provide information on public school funding, state performance compared to the 

Nation on the NAEP, high school graduation rate and degree attainment rate. 

Public Funding Sources
PERFORMANCE REPORT

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 9

National Assessment of Educational Progress – Fourth Grade
PERFORMANCE REPORT

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 10

National Assessment of Educational Progress – Eighth Grade
PERFORMANCE REPORT

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 11

National Assessment of Educational Progress – Twelfth Grade
PERFORMANCE REPORT

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 12

National High School Graduation Rate

National College Degree Attainment
PERFORMANCE PLAN

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 13

FY 2018-22 Strategic Goals and Strategic Objectives

Strategic Goal 1: Support state and local efforts to improve learning outcomes for all P-12 students in every 

community.

Strategic Objective 1.1 Increase high-quality educational options and empower students and parents to 

choose an education that meets their needs.

Strategic Objective 1.2 Provide all P-12 students with equal access to high-quality educational opportunities.

Strategic Objective 1.3 

Prepare all students for successful transitions to college and careers by supporting 

access to dual enrollment, job skills development and high-quality science, technology, 

engineering and mathematics (STEM).

Strategic Objective 1.4 

Support agencies and institutions in the implementation of evidence-based strategies 

and practices that build the capacity of school staff and families to support students’ 

academic performance.

Strategic Goal 2: Expand postsecondary educational opportunities, improve outcomes to foster economic 

opportunity and promote an informed, thoughtful and productive citizenry.

Strategic Objective 2.1 Support educational institutions, students, parents and communities to increase access 

and completion of college, lifelong learning and career, technical and adult education.

Strategic Objective 2.2 

Support agencies and educational institutions in identifying and using evidence-based 

strategies or other promising practices to improve educational opportunities and 

successfully prepare individuals to compete in the global economy.

Strategic Objective 2.3 

Support agencies and educational institutions as they create or expand innovative and 

affordable paths to relevant careers by providing postsecondary credentials or jobready skills.

Strategic Objective 2.4 Improve quality of service for customers across the entire student aid life cycle.

Strategic Objective 2.5 Enhance students’ and parents’ ability to repay their federal student loans by providing 

accurate and timely information, relevant tools and manageable repayment options.

Strategic Goal 3: Strengthen the quality, accessibility and use of education data through better management, 

increased privacy protections and transparency.

Strategic Objective 3.1 Improve the Department’s data governance, data life cycle management and the 

capacity to support education data.

Strategic Objective 3.2 Improve privacy protections for, and transparency of, education data both at the 

Department and in the education community.

Strategic Objective 3.3 Increase access to, and use of, education data to make informed decisions both at the 

Department and in the education community.

Strategic Goal 4: Reform the effectiveness, efficiency and accountability of the Department.

Strategic Objective 4.1

Provide regulatory relief to educational institutions and reduce burden by identifying timeconsuming regulations, processes and policies and working to improve or eliminate them, 

while continuing to protect taxpayers from waste and abuse.

Strategic Objective 4.2 Identify, assess, monitor and manage enterprise risks.

Strategic Objective 4.3 Strengthen the Department’s cybersecurity by enhancing protections for its information 

technology infrastructure, systems and data.

Strategic Objective 4.4 Improve the engagement and preparation of the Department’s workforce using 

professional development and accountability measures.
PERFORMANCE PLAN

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 14

The Department’s Agency Priority Goals (APGs) 

The Department identified four Agency Priority Goals (APGs) for FY 2018–19. Improving 

education starts with allowing greater decision-making authority at the state and local levels and 

empowering parents and students with educational options. These APGs aim to increase 

educational choice, improve the customer service the Department provides, ensure students are 

protected, and reduce red tape. The effective implementation of the Department’s APGs will 

depend, in part, on the effective use of high–quality and timely data, including evaluations and 

performance measures. Quarterly updates for the APGs will be available on 

www.Performance.gov. 

APG Related Strategic Objective

Improve the access to, and the quality and transparency of, 

school choice options for K-12 students.

By September 30, 2019, the Charter School Program (CSP) will 

support the creation and expansion of 300 new charter schools 

nationally. The CSP will also support the enrollment of 50,000 students 

in new charter schools. Additionally, by September 30, 2019, the 

Department will disseminate eight resources, at least one per quarter, 

on evidence-based and promising practices related to school choice.

Strategic Objective 1.1: Increase highquality educational options and empower 

students and parents to choose an 

education that meets their needs.

Improve borrowers’ access to quality customer service.

By September 30, 2019, Federal Student Aid will improve customers’ 

access to and availability of quality customer service by decreasing the 

overall average speed of answer to 60 seconds or less, decreasing 

abandoned rates to three percent or less and requiring all non-default 

federal student loan servicers to expand and standardize call center 

hours. 

Strategic Objective 2.4: Improve quality of 

service for customers across the entire 

student aid life cycle.

Improve student privacy and data security at Institutions of 

Higher Education (IHEs) through outreach and compliance efforts.

By September 30, 2019, the Department will increase information 

security program outreach activities to IHEs by 40% in order to help 

protect IT systems and data privacy and commence audits of IHEs 

subject to A-133 and Gramm-Leach-Bliley Act (GLBA), resulting in 36 

IHEs (from a baseline of zero) completing an audit of GLBA-related 

information security safeguards with no significant findings.

Strategic Objective 3.2: Improve privacy 

protections for, and transparency of, 

education data both at the Department and 

in the education community.

Provide regulatory relief to education stakeholders.

By September 30, 2019, the Department will reduce the regulatory 

burden on education stakeholders by submitting to OMB no less than 

25 deregulatory actions (against a baseline of zero (0) for FYs 2015 

and 2016).

Strategic Objective 4.1: Provide 

regulatory relief to educational institutions 

and reduce burden by identifying timeconsuming regulations, processes and 

policies and working to improve or 

eliminate them, while continuing to protect 

taxpayers from waste and abuse.
CROSS AGENCY PRIORITY (CAP) GOALS

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 15

Cross-Agency Priority (CAP) Goals

In addition to the APGs, the Department contributes to CAP Goals established by the 

Administration. These CAP Goals are used by leadership to accelerate progress on a limited 

number of Presidential priority areas where implementation requires active collaboration among

multiple agencies. The Department will contribute to the CAP goals, as appropriate, and will 

include information on its contributions in the FY 2018 Annual Performance Report and FY 2020 

Annual Performance Plan. Additionally, please refer to www.Performance.gov for the agency’s 

contributions to those goals and progress, where applicable.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 16

Strategic Goal 1

Support state and local efforts to improve learning outcomes for all 

P-12 students in every community.

Goal leader: Assistant Secretary for Elementary and Secondary Education

Strategic Objective 1.1

Increase high-quality educational options and empower students and 

parents to choose an education that meets their needs.

Strategic Objective Leader: Assistant Deputy Secretary for Innovation and Improvement

Implementation Strategies:

Parents and students need to know about and understand the educational options available to 

them. Further, parents and students should be able to select the educational experience that

best suits their needs. If every student had the opportunity to pursue personal development in 

the context of an organization aligned with their unique interests, our nation would be stronger 

as a result. To that end, the Department will support greater state and local flexibility in 

elementary and secondary education and encourage SEAs and LEAs to increase the number 

and quality of educational opportunities to provide for meaningful school choice. The two 

primary strategies the Department will use to achieve this strategic objective are the 

administration of federal programs that increase educational options (e.g., the Charter School 

Program (CSP) and the Magnet Schools Assistance Program (MSAP)) and the provision of 

technical assistance. 

Next Steps:

Several offices across the Department support this Strategic Objective, including the Office of 

Elementary and Secondary Education (OESE), the Office of Innovation and Improvement, the 

Office of Special Education and Rehabilitative Services (OSERS), the Office for Civil Rights

(OCR), and the Office of English Language Acquisition (OELA). During FY 2018 and FY 2019, 

the Department will:

• Administer federal programs and use the proposed Secretary’s Supplemental Priority on 

educational choice in discretionary grant programs, when appropriate and consistent 

with a program’s authorizing statute.

• Develop and disseminate quarterly resources on evidence-based and promising 

practices related to school choice.

• Deliver relevant technical assistance to states and communities on school choice topics.

• Apply effective oversight of CSP and MSAP on grant performance.

• Respond to risks identified by national audits and studies of charter schools and school 

choice.

• Provide information and technical assistance to support parents, including parents of 

children with disabilities, in choosing a high–quality education option for their child.

• Provide information to states regarding distributing federal Individuals with Disabilities

Education Act (IDEA) funds to all LEAs, including charter LEAs.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 17

• Provide information and technical assistance to support states in ensuring that the rights 

of children with disabilities are protected when parents exercise school choice. 

• Host a symposium in FY 2018 to explore weighted student funding (WSF) models, 

including lessons learned from early adopters, challenges, and opportunities, to inform 

the Department’s work in developing policy guidance and programmatic infrastructure 

that will support efforts by up to 50 districts in the 2018-19 school year to adopt WSF 

systems that improve funding equity and support expanded educational choice. 

• Launch the Flexibility for Equitable Per-Pupil (weighted student funding) demonstration 

program authorized by Title I, Part E of the ESEA. 

Performance Measures:
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 18

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Strategic Objective 1.2 

Provide all P-12 students with equal access to high-quality educational

opportunities.

Strategic Objective Leader: Assistant Secretary for Special Education and Rehabilitative 

Services

Implementation Strategies: 

Every child, regardless of his or her ZIP code or family income, should have access to a high–

quality education. The implementation of the ESSA, which reauthorized the ESEA, provides a 

framework for states and school districts to provide high–quality educational opportunities that 

meet the needs of their students, based on the following: 

• High academic standards and assessments. 

• A well-rounded education that includes a safe and healthy environment and integrated 

technology.

• Access to effective teachers and strong systems of collaboration among education 

stakeholders.

• Proper support and interventions for struggling students and opportunities for enrichment 

experiences before, after, and during the school day.

This Strategic Objective considers characteristics that are necessary to ensure both access to 

opportunities and the quality of such opportunities. The Department is committed to improving 

access to high–quality P-12 education opportunities for each student and will support 

educational institutions, parents, and communities in developing such opportunities as well as 

their capacity to improve the outcomes for each student.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 19

Next Steps:

Several offices across the Department support this Strategic Objective, including OESE, 

OSERS, OCR, the Office of Education Technology (OET), the Office of Career, Technical, and 

Adult Education (OCTAE), and the Office of Planning, Evaluation, and Policy Development 

(OPEPD). During FY 2018 and FY 2019, the Department will:

• Develop accountability system summaries of all approved ESSA state plans for internal 

use to help build staff capacity to support states as they implement and, as necessary, 

refine their plans. The Policy and Program Studies Service (PPSS) within OPEPD will 

develop these summaries based on state plans that the Department has approved. 

• Prepare and disseminate studies on K‐12 program or policy issues that could inform or 

support state implementation of ESSA state plans across programs. 

• Provide states, districts, schools, and families:

o Literacy instructional and assessment strategies and accessible digital 

instructional materials to students who are blind, visually impaired, or who have 

print disabilities.

o Resources and technical assistance that support preschool children with 

disabilities growth in social-emotional skills, acquisition and use of knowledge 

and skills, and use of appropriate behaviors to meet their needs.

o Resources and technical assistance that support the participation of children with 

disabilities in early childhood classrooms.

o Technical assistance on the development, implementation, and sustainability of 

multi-tiered systems of support, and intervention strategies to reduce 

inappropriate school behavior and improve school conditions.

o Technical assistance to ensure children with disabilities have appropriately 

ambitious individualized education program (IEP) goals and challenging 

objectives.

o Technical assistance to support improved results for economically disadvantaged 

students, children with disabilities, English learners, migrant, homeless, charter 

schools’ attendees, magnet schools’ attendees, and CTE concentrators.

• Provide targeted and intensive monitoring and support as well as general technical 

assistance based on risk assessments. 

• Conduct a National Educational Technology Survey to provide national estimates on the 

availability and use of educational technology in elementary and secondary schools.

Performance Measures: 
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 20
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 21

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Strategic Objective 1.3

Prepare all students for successful transition to college and careers by 

supporting access to dual enrollment, job skills development and highquality science, technology, engineering and mathematics (STEM).

Strategic Objective Leader: Assistant Secretary for Career, Technical and Adult Education

Implementation Strategies:

Well-rounded education programs are critical to preparing students for college and careers. 

Access to accelerated coursework to earn postsecondary credit while still in high school, such 

as Advanced Placement and International Baccalaureate courses, dual or concurrent enrollment 
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 22

programs, as well as to STEM opportunities and career and technical education is critical for 

students as they prepare for the transition to college and career and the constantly changing 

career demands of the technology-driven global economy. 

Next Steps:

Several offices across the Department support this Strategic Objective, including OCTAE, 

OESE, OPEPD, OCR and OSERS. During FY 2018 and FY 2019, the Department will:

• Provide technical assistance to recipients and subrecipients of funds under:

o The Adult Education and Family Literacy Act to improve program management 

and participant outcomes.

o The Carl D. Perkins Career and Technical Education Act to improve program 

management and participant outcomes.

• Conduct outreach related to the President’s Memorandum on STEM education, which 

calls for investing $200 million annually in high-quality STEM education, with an 

emphasis on computer science. 

• Use the proposed Secretary’s Supplemental Priority on STEM in discretionary grant 

programs, when appropriate and consistent with the program’s authorizing statute.

• Develop guidance and activities regarding transition services and pre-employment 

transition services for students and youth with disabilities. 

• Provide technical assistance to support state special education and Vocational 

Rehabilitative staff to provide transition services and pre-employment transition services 

to youth with disabilities. 

• Utilize the Parent Training and Information Centers to provide parents with technical 

assistance, information, and training about disabilities under IDEA, IDEA rights and 

protections and other relevant laws.

Performance Measures: 
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 23
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 24

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Strategic Objective 1.4

Support agencies and institutions in the implementation of evidencebased strategies and practices that build the capacity of school staff and 

families to support students’ academic performance.

Strategic Objective Leader: Director of Policy and Program Studies Service, OPEPD

Implementation Strategies:

A greater focus on evidence of what works for educators will help better serve students, 

families, and communities. This Strategic Objective aims to support the development of 

evidence about what works in P-12 education, primarily through expanded support for SEAs 

and LEAs as they implement provisions in the ESSA that require the use of evidence when 

determining what education interventions to implement.

Next Steps:

Several offices across the Department support this Strategic Objective, including OPEPD, 

OSERS, OESE and the Institute of Education Sciences (IES). During FY 2018 and FY 2019, the 

Department will:

• Collaborate with internal and external partners to disseminate resources related to use 

of evidence.

• Utilize the Parent Training and Information Centers to provide parents with technical 

assistance, information and training about disabilities, IDEA rights and protections, and 

other relevant laws.

• Support state work related to the State Systematic Improvement Plan (SSIP) to promote 

increased state capacity in evaluating the efficacy, fidelity of implementation and impact 

of evidence-based practices. 

• IES, through its National Center for Education Research and National Center for Special 

Education Research, will:

o Announce the FY 2018 funding opportunity for Small Business Innovation 

Research, and
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 25

o Announce FY 2019 funding opportunities for Education Research, Special 

Education Research, and Partnerships and Collaborations Focused on Problems 

of Practice or Policy.

• Release What Works Clearinghouse products on what is known about the effectiveness 

of various P-12 educational interventions.

Performance Measure: 

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Note on performance metrics and targets: These metrics were established as a part of the

development of FY 2018-22 Strategic Plan. Metrics may be updated or revised in the future. 

Such updates or revisions will be identified in footnotes.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 26

Goal 1 Discretionary Resources

36,635,410 

36,681,839 

32,436,494 

 - 10,000,000 20,000,000 30,000,000 40,000,000

(Dollars in Thousands)

Goal 1 Discretionary Resources

2019 President's Budget

2018 Annualized CR

2017 Appropriation

Major Discretionary Programs and Activities Supporting Goal 1

POC ACCT Goal 

# 

Obj

# Program FY 2017 Appropriation 

FY 2018 

Annualized 

CR

FY 2019 

President’s 

Budget 

OESE ED 1

1.2, 

1.3, 

1.4

Title I Grants 

to local 

educational 

agencies 

15,386,180 15,428,437 15,459,802

OESE SIP 1 1.2, 

1.4

State 

assessments 369,100 366,593 369,100

OII I&I 1 

1.1, 

1.2

Opportunity 

Grants 

(proposed 

legislation)

- - 500,000

OII I&I 1 1.1, 

1.2

Charter 

schools grants 342,172 339,848 500,000

OSERS SE 1 

1.1, 

1.2, 

1.4

Special 

education 

grants to 

States 

11,939,805 11,984,380 12,002,848

Note: On February 9, 2018, the President signed the Bipartisan Budget Act of 2018, which 

includes a two‐year cap deal that raises the FY 2019 spending caps significantly. As such, the 

Administration submitted an addendum to its FY 2019 Budget that includes additional funding 

for a limited set of Administration priorities under the new, higher cap levels. Please see 

https://www2.ed.gov/about/overview/budget/budget19/index.html for final levels in the FY 2019 

Budget.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 27

Strategic Goal 2

Expand postsecondary educational opportunities, improve 

outcomes to foster economic opportunity and promote an 

informed, thoughtful and productive citizenry.

Goal leader: Assistant Secretary for Postsecondary Education

Strategic Objective 2.1

Support educational institutions, students, parents and communities to 

increase access and completion of college, lifelong learning and career, 

technical and adult education.

Strategic Objective Leader: Deputy Assistant Secretary for Management and Planning, Office 

of Postsecondary Education (OPE)

Implementation Strategies:

The Department recognizes there are multiple, diverse routes to postsecondary education and 

employment and, under this Strategic Objective, seeks to support access to these educational 

opportunities and their completion for students at all stages of life. To that end, the Department 

will support the educational institutions that advance such opportunities through the 

administration of federal programs and the provision of technical assistance. Additionally, the 

Department will leverage opportunities to enhance the dissemination of information about these 

programs and to work collaboratively with partners to improve institutional oversight.

The Department will focus on improved utilization of discretionary grant funding so that grantees 

can fully achieve program goals and best serve students. A key indicator of program activity is 

drawdown of funds from grants that have been awarded. The Department will improve utilization 

of federal funding in monitoring the number of grantees with large available balances and 

providing technical assistance to those grantees to remedy such balances. 

A key indicator of successful administration of a grant program is fulfillment of all statutory 

requirements. The Department will monitor compliance with reporting and other requirements 

and assess whether adequate financial and human capital resources are in place to conduct 

mandated activities.

Next Steps:

Several offices across the Department support this Strategic Objective, including the OPE, 

OPEPD, FSA, and OCR. During FY 2018 and FY 2019, the Department will:

• Provide technical assistance to Congress, as requested, to inform the reauthorization of 

the Higher Education Act – such technical assistance includes reviewing draft legislation 

or policies for technical accuracy as well as providing information as to the impact of 

such proposals.

• Amend Title IX regulations to protect all students from harassment and violence by 

utilizing the federal rulemaking process to seek public comment on proposed 

regulations.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 28

• Strengthen grants management practices to maximize the impact of the Department’s 

discretionary grants portfolio. 

• Conduct, as appropriate and necessary, negotiated rulemaking to ensure the 

development of regulations that are fair, efficient, and effective for all affected parties. 

• Automate applicable College Scorecard measures within the National Student Loan 

Data System.

• Release College Scorecard information, including earnings, annually.

• Promote promising postsecondary enrollment strategies designed to improve 

postsecondary success.

• Provide resources and tools to parents and students to increase awareness and 

understanding of their rights, and to schools, colleges, and universities to increase 

awareness and understanding of their obligations, under federal civil rights laws by 

increasing the number of OCR technical assistance presentations and releases of public 

information.

• Promote first-time FAFSA filing among high school seniors, to increase access to 

postsecondary education.

• Improve the FAFSA-filing experience for first time aid recipients, to improve persistence 

among first-time FAFSA filers.

Performance Measures:
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 29

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.
Strategic Objective 2.2

STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 30

Support agencies and educational institutions in identifying and using 

evidence-based strategies or other promising practices to improve 

educational opportunities and successfully prepare individuals to compete 

in the global economy.

Strategic Objective Leader: Director of Policy and Program Studies Service, OPEPD 

Implementation Strategies:

The Department is committed to improving educational opportunities for the existing and future 

workforce. As such, this Strategic Objective focuses on the administration of federal programs 

that provide educational opportunities, training, and support services to the workforce as well as 

the use of evidence. It is also essential that the Department fulfills its commitment to individuals 

with disabilities by working with state and local agencies to provide job-driven training and 

support services, consistent with the Workforce Innovation and Opportunity Act.

Next Steps:

Several offices across the Department support this Strategic Objective, including the OPE, 

OPEPD, IES, OSERS, and OCTAE. During FY 2018 and FY 2019, the Department will:

• Strengthen grants management practices to maximize the impact of the Department’s 

discretionary and formula grants portfolio. 

• Leverage discretionary grant competitions to promote the use of evidence-based and 

promising practices. 

• Launch a CTE research network to increase the number of high-quality, causal studies 

on CTE practices, policies, and programs.

• Provide technical assistance to states and adult education providers to promote the 

effective implementation of Integrated English Literacy and Civics Education program 

activities that prepare individuals for unsubsidized employment in in-demand industries 

and occupations.

• Provide assistance to states to increase apprenticeship opportunities for high school 

CTE students. 

• Highlight Career Pathways for Individuals with Disabilities demonstration grant projects 

funded by OSERS.

• Announce FY 2019 funding opportunities for Education Research and Special Education 

Research and Partnerships and Collaborations Focused on Problems of Practice or 

Policy.

• Release reports and What Works Clearinghouse products that describe what is known 

about effective programs or practices in postsecondary education.

• Promote promising postsecondary enrollment strategies designed to improve 

postsecondary success.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 31

Performance Measures:

N/A = Not applicable

TBD = To be determined
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 32

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Strategic Objective 2.3

Support agencies and educational institutions as they create or expand 

innovative and affordable paths to relevant careers by providing 

postsecondary credentials or job-ready skills.

Strategic Objective Leader: Assistant Secretary for Career, Technical and Adult Education

Implementation Strategies:

It is critical that we ensure our nation’s workforce is prepared to meet the challenges of 

tomorrow with the skills and credentials that employers require, which can be achieved by 

creating or expanding innovative paths to recognized postsecondary credentials or obtainment 

of job-ready skills for careers in in-demand industry sectors or occupations. Such credentials 

may be obtained by students at a wide variety of education providers, such as traditional IHEs, 

non-traditional education providers, and other providers of self-guided learning. Through this 

Strategic Objective, the Department will provide grant funding and technical assistance 

resources to develop, evaluate, and replicate practices and programs that expand access to 

viable educational and career pathways that are innovative and affordable and lead to educated 

citizens with quality careers.

Next Steps:

Several offices across the Department support this Strategic Objective, including the OPE, 

OCTAE and OSERS. During FY 2018 and FY 2019, the Department will:

• Provide technical assistance to:

o Minority-serving community colleges in forming peer-to-peer communities of 

practice and sharing information.

o Community colleges in embedding cumulative, industry-recognized credentials 

within technical associate degree programs that enable students to advance 

along a career pathway to different and potentially higher-paying jobs.

o Recipients and subrecipients of funds under the Adult Education and Family 

Literacy Act to improve program management and participant employment 

outcomes.

o Improve the CTE and other outcomes of justice-involved young adults (ages 16–

24) in diversion programs that are alternatives to initial or continued processing in 

the criminal justice system.

• Assist state vocational rehabilitation agencies to develop the skills and processes 

needed to meet the requirements of the Workforce Innovation and Opportunity Act by 

providing training and technical assistance through various grants: (1) Workforce 

Innovation Technical Assistance Center; (2) Vocational Rehabilitation Technical 

Assistance Center – Youth with Disabilities; (3) Vocational Rehabilitation Technical 

Assistance Center – Targeted Communities; and (4) Vocational Rehabilitation Agency 

Program Evaluation and Quality Assurance.

• Support IHEs to provide support to individuals to obtain a degree in rehabilitation 

counseling.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 33

Performance Measures:

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Strategic Objective 2.4

Improve quality of service for customers across the entire student aid life

cycle.

Strategic Objective Leader: Chief Operating Officer, Federal Student Aid (FSA)

Implementation Strategies:

To improve the way the Department serves its customers, the Department will modernize the 

loan-servicing environment by taking the best ideas and capabilities available and putting them 
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 34

to work for Americans with federal student loans. This Strategic Objective focuses on the 

customer service provided by the Department to students, parents, borrowers, and institutions. 

The Department will deliver a best-in-class processing and servicing environment capable of 

sustaining the management of the growing federal student loan portfolio.

Next Steps:

During FY 2018 and FY 2019, the FSA will:

• Ensure accurate and timely information is available and promoted through the 

Department’s multiple engagement and information dissemination channels.

• Provide information geared toward the needs of students at each stage of the student 

aid life cycle.

• Oversee and monitor non-default federal student loan servicers to ensure they meet the 

requirements in their contracts.

• Ensure the Department’s non-default federal student loan servicers have websites that 

borrowers can use to log on, view their loan information, make a payment, and 

complete other transactions.

• Ensure the Department’s non-default federal student loan servicers communicate with 

borrowers who are in repayment and are past due. 

• Conduct outreach efforts to student loan borrowers and perform listening sessions of 

customer-call center interactions to ascertain areas for continual improvement in service 

and the customer experience.

Performance Measures:
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 35

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Strategic Objective 2.5

Enhance students’ and parents’ ability to repay their federal student 

loans by providing accurate and timely information, relevant tools and 

manageable repayment options.

Strategic Objective Leader: Chief Operating Officer, FSA

Implementation Strategies:

A comprehensive framework for default prevention that allows students to understand and 

access information about college options and associated costs, loan counseling and guidance, 

support for retention, loan repayment options and borrower benefits is critical. The Department 

will develop materials that clearly communicate students’ and parents’ options to repay federal 

student loans before and during the course of pursuing a postsecondary credential.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 36

Next Steps:

Several offices across the Department support this Strategic Objective, including FSA and OPE. 

During FY 2018 and FY 2019, the Department will:

• Monitor and report on non-default federal student loan servicers’ performance.

• Focus on reducing the number of delinquent borrowers.

• Address Borrower’s Defense regulations.

• Address Gainful Employment regulations. 

• Maintain and improve the Studentaid.gov, FinancialAidToolkit.ED.gov and 

CollegeCost.ED.gov websites.

• Leverage our digital engagement assets to inform, motivate and respond to students, 

parents, and financial aid professionals on the topic of student loans. 

• Engage with financial aid professionals to provide training and information on the tools, 

resources, and information they need to be successful in counseling current and 

prospective student loan borrowers. 

• Partner with minority serving institutions to help increase the financial aid acumen of 

students and parents.

• Ensure student borrowers receive exit counseling when they complete their degree 

program, withdraw, or enroll at less than half time.

Performance Measures:
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 37

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Note on performance metrics and targets: These metrics were established as a part of the 

development of FY 2018-22 Strategic Plan. Metrics may be updated or revised in the future. 

Such updates or revisions will be identified in footnotes.

Goal 2 Discretionary Resources

30,228,695 

30,034,276 

27,829,306 

 - 10,000,000 20,000,000 30,000,000 40,000,000

(Dollars in Thousands)

Goal 2 Discretionary Resources

2019 President's Budget

2018 Annualized CR

2017 Appropriation

Major Discretionary Programs and Activities Supporting Goal 2

POC ACCT Goal 

# 

Obj 

# Program FY 2017 Appropriation 

FY 2018 

Annualized 

CR

FY 2019 

President’s 

Budget 

FSA SFA 2

2.1, 

2.3

Federal Pell 

grants:

Discretionary 

22,475,352 22,322,722 22,475,352

OCTAE CTAE 2 

2.1, 

2.3

Career and 

technical 

education 

State grants 

1,112,226 1,115,380 1,117,598

OPE HE 2 2.1 Strengthening 

HBCUs 244,694 243,032 244,694

OPE HE 2 

2.1, 

2.2, 

2.3

Federal TRIO 

programs 950,000 943,549 550,000

Note: On February 9, 2018, the President signed the Bipartisan Budget Act of 2018, which 

includes a two‐year cap deal that raises the FY 2019 spending caps significantly. As such, the 
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 38

Administration submitted an addendum to its FY 2019 Budget that includes additional funding 

for a limited set of Administration priorities under the new, higher cap levels. Please see 

https://www2.ed.gov/about/overview/budget/budget19/index.html for final levels in the FY 2019 

Budget.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 39

Strategic Goal 3

Strengthen the quality, accessibility and use of education data 

through better management, increased privacy protections and 

transparency.

Goal leader: Assistant Secretary for Planning, Evaluation, and Policy Development

Strategic Objective 3.1

Improve the Department’s data governance, data life cycle management 

and the capacity to support education data.

Strategic Objective Leader: Associate Commissioner, Administrative Data Division, National 

Center for Education Statistics, IES 

Implementation Strategies: 

This Strategic Objective focuses on the data that the Department collects and opportunities to 

enhance the Department’s data management framework and internal capacity. The main goal of 

improving data management is to clarify the roles and procedures used to optimize data 

integrity and quality. The strategies behind this objective guide the Department’s activities to 

develop and support the highly skilled workforce capacity necessary to execute these 

improvements. As a result, parents, communities, and the Department’s state and local partners 

will be more inclined to make more informed decisions on behalf of their children and 

communities.

Next Steps:

Several offices across the Department support this Strategic Objective, including IES, OPEPD, 

and the Office of the Chief Information Officer. During FY 2018 and FY 2019, the Department 

will:

• Assess the current maturity of data management practices within the Department’s 

program offices and work with program offices to identify and prioritize areas of 

improvement.

• Deliver training curricula for data professionals within the Department to continue to 

develop a highly skilled workforce throughout the Department to facilitate the accurate 

and appropriate use of data.

• Explore improvements in the Department’s operating structure that would facilitate better 

data practice.

• Improve the Department’s data governance, data life management, and the capacity to 

support education data through varied efforts that include developing an open source 

data analysis software package called EdSurvey (currently available for NAEP data) for 

international assessment data.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 40

Performance Measures:

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Strategic Objective 3.2

Improve privacy protections for, and transparency of, education data 

both at the Department and in the education community.

Strategic Objective Leader: Chief Privacy Officer, Office of Management (OM)

Implementation Strategies:

The Department is committed to student privacy. While education data have transformative 

potential, the vast amount and sensitivity of these data make it imperative that the Department 

and educational institutions that maintain student data take steps to protect the data adequately. 

This Strategic Objective focuses on improving privacy protections through the administration of 

the Family Educational Rights and Privacy Act (FERPA), the development and dissemination of 

privacy and security training, and conducting technical assistance to SEAs, LEAs, and IHEs. 

The APG for this goal focuses on data security at institutions of higher education, using auditing, 

training, and reporting as tools.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 41

Next Steps:

Several offices across the Department support this Strategic Objective, including the Office of 

the Chief Privacy Officer (OCPO), FSA, IES, Office of the General Counsel (OGC) and all of the 

Department’s program offices. During FY 2018 and FY 2019, the Department will:

• Administer FERPA, with a focus on improving the timeliness of both enforcement actions 

and policy determinations.

• Provide needed assistance to educational institutions via the Privacy Technical 

Assistance Center. 

• Provide support, including outreach and compliance efforts, to external partners.

Performance Measures: 
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 42

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Strategic Objective 3.3

Increase access to, and use of, education data to make informed 

decisions both at the Department and in the education community.

Strategic Objective Leader: Director of Policy and Program Studies Service, OPEPD

Implementation Strategies:

For education data to be used to support informed decisions, the Department will need to 

develop and implement methods to analyze, visualize, interpret, and represent education data

and support education stakeholders in doing the same. This Strategic Objective focuses on 

increasing access to education data at all levels, and improving the tools necessary to support 

the appropriate use of education data for decision-making by the Department and education 

stakeholders.

Next Steps:

Several offices across the Department support this Strategic Objective, including IES, OPEPD,

OET, OCR, OCPO, and OSERS. During FY 2018 and FY 2019, the Department will:

• Improve the usefulness of the NAEP data for informed decision-making in the education 

community through analysis and reporting of a new type of data produced by the new 

NAEP digitally based assessments. The new digital format produces data on actions 

students take in responding to assessment questions. 

• Conduct outreach and provide support to LEAs to prepare for the 2017-18 CRDC data 

submission period to increase data access and use.

• Review data provided by grant recipients and provide technical assistance on data 

quality and use.

• Expand the InformED initiative to further develop the Department’s open data 

infrastructure.

• Ensure Accountability and Audit Resolution Tracking System (AARTS) system data are 

secure, accurate and reliable.

• Improve access to NCES’ extensive sample-survey data by integrating nine additional 

data sets on topics ranging from early childhood to postsecondary education into its 

public facing Datalab tool.

• Provide training, coaching and applied research assistance to SEA and LEA staff 

through the Regional Educational Laboratory program to assist with the analysis of 

education data.

• Increase the Department’s awareness and appropriate use of:

o The Open Licensing Requirement for Competitive Grant Programs and 

associated exemptions.

o Data dissemination requirements in grants competitions.

o Sessions dedicated to improved data use for external audiences.

• Complete a feasibility study on improvements to the open data IT infrastructure at the 

Department.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 43

• Ensure current links to machine readable data sources in the Public Data Listing are 

accurate and complete.

• Encourage offices to include releasing data in a machine-readable format into their data 

dissemination plans.

Performance Measures:

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Note on performance metrics and targets: These metrics were established as a part of the 

development of FY 2018-22 Strategic Plan. Metrics may be updated or revised in the future. 

Such updates or revisions will be identified in footnotes.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 44

Goal 3 Discretionary Resources

540,026 

536,358 

510,745 

 - 100,000 200,000 300,000 400,000 500,000 600,000

(Dollars in thousands)

Goal 3 Discretionary Resources

2019 President's Budget

2018 Annualized CR

2017 Appropriation

Major Discretionary Programs and Activities Supporting Goal 3 

POC ACCT Goal 

# 

Obj 

# Program FY 2017 Appropriation 

FY 2018 

Annualized 

CR

FY 2019 

President’s 

Budget 

IES IES 3

3.1, 

3.2, 

3.3

Research, 

development, 

and 

dissemination 

187,500 186,227 187,500

IES IES 3 

3.1, 

3.2, 

3.3

Statistics 109,500 108,756 112,500

IES IES 3 

3.1, 

3.2, 

3.3

National 

assessment 149,000 147,988 149,000

Note: On February 9, 2018, the President signed the Bipartisan Budget Act of 2018, which 

includes a two‐year cap deal that raises the FY 2019 spending caps significantly. As such, the 

Administration submitted an addendum to its FY 2019 Budget that includes additional funding 

for a limited set of Administration priorities under the new, higher cap levels. Please see 

https://www2.ed.gov/about/overview/budget/budget19/index.html for final levels in the FY 2019 

Budget.
STRATEGIC GOALS AND OBJECTIVES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 45

Strategic Goal 4

Reform the effectiveness, efficiency and accountability of the 

Department.

Goal Leader: Assistant Secretary for Management

Strategic Objective 4.1 

Provide regulatory relief to educational institutions and reduce burden by 

identifying time-consuming regulations, processes and policies and 

working to improve or eliminate them, while continuing to protect 

taxpayers from waste and abuse.

Strategic Objective Leader: Senior Counselor to the Secretary

Implementation Strategies:

Reducing regulatory burden on external stakeholders and improving internal decision-making 

processes will help to ensure greater efficiencies in Department operations and more effective 

and efficient service to the public.

Next Steps:

Given the importance of protecting taxpayers from waste and abuse, all Department offices 

support this Strategic Objective. During FY 2018 and FY 2019, the Department will continue to 

identify overly burdensome, outdated, unnecessary, or ineffective regulations and guidance that 

could be improved or eliminated.

Performance Measures:
STRATEGIC GOALS AND OBJECTIVES

 FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 46
STRATEGIC GOALS AND OBJECTIVES

 FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 47

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Strategic Objective 4.2

Strategic Objective Leader: Chief Financial Officer

Implementation Strategies:

Enterprise Risk Management (ERM) includes integrating existing risk management processes in 

day-to-day and strategic decision-making activities and creating a risk-aware culture. The 

Department will strengthen its ERM practices and foster open discussions about how those 

risks might impact the accomplishment of the Department’s mission and whether the 

Department’s allocation of resources is aligned to best mitigate risks.

Next Steps: 

Given its focus on enterprise risk, all Department offices support this Strategic Objective. During 

FY 2018 and FY 2019, the Department will:

Identify, assess, monitor and manage enterprise risks.

• Review and update the enterprise risk portfolio and develop an ERM framework aligned 

to the new Strategic Plan that incorporates other strategic risks as applicable.
STRATEGIC GOALS AND OBJECTIVES

 FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 48

• Assign accountability for managing enterprise risks to appropriate Department offices 

and senior officials.

• Integrate actions taken to address management challenges into the enterprise risk 

portfolio and in discussions about how they impact strategic objectives and performance.

• Develop and maintain an enterprise risk dashboard with leading indicators to inform risk 

management discussions.

• Hold routine Senior Management Council meetings to discuss key ERM documents, 

including the framework, profile, and dashboards, and how enterprise risks relate to 

management priorities, resource allocation decisions, and other critical issues such as 

cost estimates for student loans.

• Take actions to further strengthen the integration of the Department and FSA ERM 

efforts, leveraging consistent approaches to identify, assess, and determine mitigation 

strategies for enterprise risks.

• Improve the maturity level score of the Department’s Information Technology (IT) 

Security Program and Practices as they relate to the Federal Information Security 

Modernization Act of 2014.

• Take actions to integrate the Department’s ERM framework with the Internal Control 

Program. 

• Implement a “Risk Management Tool-Kit” website on ed.gov to assist external grant 

recipients and staff in identifying, mitigating, and managing risks. 

Performance Measure: 

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Strategic Objective 4.3 

Strengthen the Department’s cybersecurity by enhancing protections for 

its information technology infrastructure, systems and data.

Strategic Objective Leader: Chief Information Officer

Implementation Strategies:

Improved cybersecurity will be a key contributor in ensuring the Department’s systems and data 

are protected, which will help build a strong foundation for the Department’s information 
STRATEGIC GOALS AND OBJECTIVES

 FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 49

technology (IT) infrastructure. As such, the Department will provide proactive cybersecurity 

services, monitor and enhance threat intelligence capabilities, explore shared services and 

cloud capabilities, and improve its cybersecurity workforce.

Next Steps:

Given its focus on cybersecurity, all Department offices support this Strategic Objective. During 

FY 2018 and FY 2019, the Department will:

• Improve the maturity level score of the Department’s IT Security Program and Practices 

as they relate to the Federal Information Security Modernization Act of 2014.

• Assess the Department’s IT inventory. 

• Lead the Education Department Acquisition Regulations Program, focusing on IT to 

strengthen and enhance cybersecurity and support Federal Information Technology 

Acquisition Reform Act (FITARA) compliance.

• Ensure all public internet connections are compliant with the Department of Homeland 

Security Trusted Internet Connections Reference Architecture, as amended.

Performance Measure: 

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Strategic Objective 4.4 

Improve the engagement and preparation of the Department’s workforce 

using professional development and accountability measures.

Strategic Objective Leader: Chief Human Capital Officer, Office of Management

Implementation Strategies:

In order to best serve the public, the Department must maximize employee effectiveness 

through robust employee engagement and performance management. The Department will 
STRATEGIC GOALS AND OBJECTIVES

 FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 50

continue to build the skills and knowledge of its workforce and will focus on all employees and 

offices. Furthermore, the Department must focus on ways to thrive in an operational climate that 

is significantly resource constrained. This will spur opportunities to involve employees in thinking 

differently about strategies to reengineer, streamline, or even eliminate work that does not serve 

the efficiency of the Department in achieving its strategic goals and desired outcomes.

Next Steps:

Given its focus on the Department’s workforce, all Department offices support this Strategic 

Objective. During FY 2018 and FY 2019, the Department will:

• Implement a comprehensive internal communications strategy focused on employee 

engagement. 

• Expand professional development opportunities to further develop workforce skills by 

finalizing competency assessments for mission critical occupations and prioritizing 

training needs, developing a pilot performance management training course to address 

common performance challenges and launching an Administrative Professionals 

Development program. 

• Ensure that all supervisors, managers, and executives receive continual training and 

development to be successful in their roles as federal supervisors and to enhance their 

leadership skills.

• Restructure the Department’s workforce through enhanced succession planning.

• Develop strategies to leverage and retain critical employee skills through reassignments, 

on-the-job training, needs assessment and skills gap closure, and enhanced 

professional development opportunities. 

Performance Measures:
STRATEGIC GOALS AND OBJECTIVES

 FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 51

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Note on performance metrics and targets: These metrics were established as a part of the 

development of FY 2018-22 Strategic Plan. Metrics may be updated or revised in the future. 

Such updates or revisions will be identified in footnotes.

Goal 4 Discretionary Resources

599,756 

595,683 

630,113 

 - 100,000 200,000 300,000 400,000 500,000 600,000 700,000

(Dollars in thousands)

Goal 4 Discretionary Resources

2019 President's Budget

2018 Annualized CR

2017 Appropriation

Major Discretionary Programs and Activities Supporting Goal 4 

POC ACCT Goal 

# 

Obj 

# Program FY 2017 Appropriation 

FY 2018

Annualized 

CR

FY 2019 

President’s 

Budget 

ALL DM/PA 4

Program 

Administration

Salaries and 

Expenses

432,000 429,066 432,506
STRATEGIC GOALS AND OBJECTIVES

 FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 52

POC ACCT Goal 

# 

Obj 

# Program FY 2017 Appropriation 

FY 2018

Annualized 

CR

FY 2019 

President’s 

Budget 

OCR OCR 4 Office for Civil 

Rights 108,500 107,763 107,438

Note: On February 9, 2018, the President signed the Bipartisan Budget Act of 2018, which 

includes a two‐year cap deal that raises the FY 2019 spending caps significantly. As such, the 

Administration submitted an addendum to its FY 2019 Budget that includes additional funding 

for a limited set of Administration priorities under the new, higher cap levels. Please see 

https://www2.ed.gov/about/overview/budget/budget19/index.html for final levels in the FY 2019 

Budget.
MAJOR MANAGEMENT PRIORITIES AND CHALLENGES

Major Management Priorities and Challenges 

As summarized in the FY 2017 Agency Financial Report, the Office of Inspector General has 

identified four management challenges for the Department in FY 2018. These challenges are 

detailed in the FY 2018 Management Challenges Report. The Government Performance and 

Results Act Modernization Act of 2010 requires the Department to include in its APP the 

following information on those planned actions, including performance goals, indicators, and 

milestones, to address these challenges:

Management 

Challenge

Accountable 

Official Accomplishments Planned Actions

Performance Goals, 

Indicators, and 

Milestones

Improper 

Payments 

Chief Financial 

Officer (FSA) 

Obtained approval from OMB for 

alternative improper payment 

estimation methodology for 2017, 

which included refinements over 

prior years

-Updated the grant and contract risk 

assessment processes to include 

consideration of all required risk 

factors 

-Implemented security control 

improvements and reactivated the 

Data Retrieval Tool (DRT) to enable 

applicants to securely obtain 

necessary financial information to 

complete the FAFSA or apply for an 

income-driven repayment (IDR) 

plan

-Documented and tested thousands 

of entity-level, process and 

information technology system 

controls, including more than 500 

controls designed to prevent or 

detect improper payments; 

determined that 96% of controls are 

designed and operating effectively, 

with 4% that have immaterial 

deficiencies that are being 

corrected 

-FSA Program Compliance 

performed ongoing program 

reviews of more than 150 schools 

to ensure compliance and identify 

root cause issues to inform 

guidance for schools

Continue to coordinate 

with OMB on 

refinements to 

estimation methodology 

for student aid programs

-Develop a statisticallyvalid estimation 

methodology

-Continuously monitor 

processes and systems 

to strengthen controls, 

including work with IRS 

to implement system 

upgrades 

-Maintain improper 

payment compliance 

requirements as a 

significant enterprise risk 

in FSA’s risk profile

Goal: Minimize the risk 

of improper payments 

without unduly 

burdening students 

-Indicator: % improper 

payments 

Key milestones: 

Integrate improper 

payment risk 

management 

strategies with 

updated enterprise risk 

portfolio aligned with 

Strategic Plan 
MAJOR MANAGEMENT PRIORITIES AND CHALLENGES

 FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 54

Management 

Challenge

Accountable 

Official Accomplishments Planned Actions

Performance Goals, 

Indicators, and 

Milestones

Information 

Technology 

Security 

Chief 

Information 

Security 

Officer (OCIO) 

Completed the Department’s

cybersecurity workforce capability 

assessment to identify current gaps 

in Department cybersecurity 

workforce skills and certifications

-Completed the Department’s 

Cybersecurity Workforce 

Development Plan to address gaps 

identified in the Department’s 

cybersecurity workforce capability 

assessment and completed 

implementation plan for coding of 

all cyber positions 

-Established Department/FSA 

Cybersecurity Committee 

Developed a Departmental

Cybersecurity Strategy and 

Implementation Plan (ED-CSIP)

-Submitted Cybersecurity 

Framework Implementation Plan to 

OMB, as required by President’s 

Executive Order 

-Completed major updates to the 

Department’s primary cybersecurity 

policy guidance 

-Re-validated the list of the 

Department’s High Value Assets 

(HVAs), to include plans of actions 

to strengthen security controls for 

all HVAs

Improve the maturity 

level score of the 

Department’s 

Information Technology 

(IT) Security Program 

and Practices as they 

relate to the Federal 

Information Security 

Modernization Act of 

2014 (FISMA)

-Maintain information 

technology security as a 

significant enterprise risk 

in FSA’s risk profile

Goal: Mature the 

Department’s 

cybersecurity 

operations to achieve 

the next level of 

effectiveness as 

defined by the FISMA 

maturity model 

-Indicator: 60% of 

security functions will 

be scored at or above 

Maturity Level 4 – 

Managed and 

Measurable

-Key milestones: 

Calibrate IT security 

risks with other 

enterprise risks in ED’s 

updated risk profile
MAJOR MANAGEMENT PRIORITIES AND CHALLENGES

 FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 55

Management 

Challenge

Accountable 

Official Accomplishments Planned Actions

Performance Goals, 

Indicators, and 

Milestones

Oversight and 

Monitoring 

Chief 

Enterprise 

Risk Officer 

(FSA) 

ED announced a stronger approach 

to FSA compliance enforcement, 

creating stronger consumer 

protections for students, parents 

and borrowers against "bad actors"

-FSA’s Annual Risk Assessment 

was the first version of the system 

to be built on the EDWA platform, 

increasing efficiency and accuracy, 

and allowing for greater flexibility in 

scoring in future years

-Initiated study to determine if 

manipulation of financial statement 

composite scores is a widespread 

issue impacting FSA risk 

assessments 

-FSA revamped REACH 

agreements for monitoring to align 

with organization performance and 

improvement efforts including timely 

determinations and improved 

customer service

-Grant-making offices developed a 

standard discretionary grant site 

visit monitoring tool and related 

training

-Implemented new cross-program 

risk tools including TAGR and 

Grantworks 

-OPE included periodic random 

checks of grant files in managers’ 

REACH agreements

-OESE expanded fiscal monitoring 

pilot across programs

Increase expertise 

among Department staff 

to effectively monitor 

student aid program 

participants and grant 

recipients 

-Improve monitoring and 

support processes, 

including increased 

information-sharing 

across offices and more 

integrated and targeted 

monitoring approaches 

-Expand data analysis 

capabilities and continue 

risk-based programmatic 

and fiscal monitoring of 

grant recipients, 

providing assistance in 

meeting requirements

-Highlight areas of 

ambiguity or common 

misunderstanding for 

program participants and 

provide additional 

technical assistance in 

those areas 

-Maintain monitoring and 

oversight as a significant 

enterprise risk in FSA’s 

risk profile 

Goal: Support 

agencies and 

educational institutions 

to improve outcomes 

for students 

-Indicator: % grants 

not making substantial 

progress 

-Key milestones: 

update risk 

assessments for 

student aid 

participants; gather 

feedback from annual 

student aid conference 

attendees; calibrate 

oversight and 

monitoring risks with 

other enterprise risks 

in ED’s updated risk 

profile
MAJOR MANAGEMENT PRIORITIES AND CHALLENGES

 FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 56

Management 

Challenge

Accountable 

Official Accomplishments Planned Actions

Performance Goals, 

Indicators, and 

Milestones

Data Quality 

and Reporting 

Director, Risk 

Management 

Service (ODS) 

OESE and the Office of Special 

Education Program applied the new 

SharePoint tracking tool to 

collaboratively document and 

follow-up on data quality concerns 

identified in the initial submissions 

of the 2015-16 Consolidated State 

Performance Reports (CSPR), with 

a focus on academic assessments 

and graduation rates, which are 

shared areas of stewardship and 

programmatic use

-EDFacts state Submission plans 

were modified to allow states to 

proactively identify issues affecting 

data submissions

-EDFacts Data Quality Summary 

Reports can be generated on every 

file collected in EDFacts; Summary 

Reports present high level 

information about timeliness, 

completeness, accuracy and 

alignment with automated business 

rules

-Policy on Data Management has 

been developed and is being 

finalized 

-A new contract was awarded to 

implement streamlined data 

management procedures and 

templates 

-EDFacts Data Governance Board

(EDGB) continues to promote and 

support stewardship of data 

-New certification language was 

updated in CSPR 

-The EDGB committed meeting 

time to exchange best practices, 

including NCES data review and 

quality control practices 

Identify efforts which 

could result in a more 

streamlined 

implementation of the 

tracking tool using key 

elements on the 2016-17 

CSPR to improve 

monitoring and 

correction of data quality 

issues 

-Finalize data 

management strategy 

and continuation with 

principal offices 

stewarding data quality, 

as part of the EDFacts

Data Governance Board 

-Adopt new standard 

certification language in 

each data collection 

system; require states to 

provide additional 

justification for 

questionable data 

-Identify promising 

practices in monitoring 

grantee controls over 

data accuracy and share 

across principal offices 

-Leverage Single Audits 

to help assess grantee 

data quality

Goal: Strengthen data 

management practices 

to support improved 

quality and efficient 

reporting to better 

enable evidencebased decision-making 

-Indicators: # of 

procedures or 

templates reviewed by 

the Department’s Data 

Strategy Team for use 

across program office 

collections; # of data 

use sessions with 

stakeholders 

-Key milestones: 

Continue use of data 

tracking tool; identify 

data management 

framework used to 

identify procedures 

and templates for use 

across data life cycle 

of active collections; 

disseminate promising 

practices 

In FY 2017, the Department continued to make progress improving Information Technology 

System Development and Implementation, and the Office of Inspector General removed this 

issue from the list of FY 2018 Management Challenges. The Department has begun the 

Maturity Phase of the FITARA plan and, at the time of this report, the Department has 

completed seven of the seventeen maturity tasks. The continued improvements have been 

made evident on the most recent Biannual FITARA Scorecard in which the Department’s score 

increased a full letter grade, from a C+ to a B+. In an effort to continue improvement and gain 

efficiencies in IT spend, the Department has completed an “As-Is” visualization map of all IT 

systems and services. This effort will enable the Department to rationalize the IT portfolio by 

eliminating redundant IT services and consolidate existing services when possible.
Appendices

Appendix A: FY 2017 Performance Metrics 

Appendix B: Data Validity and Verification for FY 2017 Annual Performance Report 

Appendix C: Data Validity and Verification for FY 2019 Performance Metrics 

Appendix D: Programs by Goal 

Appendix E: Summary of Performance Evaluations Conducted During FY 2017 and Expected 

During FY 2018 and FY 2019 

Appendix F: Glossary of Acronyms and Abbreviations 
Appendix A: FY 2017 Performance Metrics2

 

2 These performance metrics were established in relation to the FY 2014-18 Strategic Plan and 

their FY 2017 performance targets were included in the FY 2016 Annual Performance Report 

and FY 2018 Annual Performance Plan. Now that the Department has established the FY 2018-

22 Strategic Plan, this Appendix represents the final report on these performance metrics and 

the prior Strategic Plan.

GOAL 1: POSTSECONDARY EDUCATION, CAREER AND TECHNICAL EDUCATION, 

AND ADULT EDUCATION. Increase college access, affordability, quality, and 

completion by improving postsecondary education and lifelong learning opportunities 

for youths and adults.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 59
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 60
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 61

GOAL 2: ELEMENTARY AND SECONDARY EDUCATION. Improve the elementary and 

secondary education system’s ability to consistently deliver excellent instruction 

aligned with rigorous academic standards while providing effective support services to 

close achievement and opportunity gaps, and ensure all students graduate high school 

college- and career-ready.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 62
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 63
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 64

GOAL 3: EARLY LEARNING. Improve the health, social-emotional, and cognitive 

outcomes for all children from birth through 3rd grade, so that all children, particularly 

those with high needs, are on track for graduating from high school college- and careerready.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 65

GOAL 4: EQUITY. Increase educational opportunities for underserved students and 

reduce discrimination so that all students are well-positioned to succeed.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 66
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 67

GOAL 5: CONTINUOUS IMPROVEMENT OF THE U.S. EDUCATION SYSTEM. Enhance 

the education system’s ability to continuously improve through better and more 

widespread use of data, research and evaluation, evidence, transparency, innovation, 

and technology.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 68
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 69
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 70

GOAL 6: U.S. DEPARTMENT OF EDUCATION CAPACITY. Improve the organizational 

capacities of the Department to implement this strategic plan.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 71
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 72
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 73

N/A = Not applicable

TBD = To be determined

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans AugustJuly and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 

year; Calendar Year (CY) spans January-December.

Note on performance metrics and targets: These metrics were established as a part of the 

FY 2014-18 Strategic Plan. Metrics may be updated from previous reports or revised to reflect 

awareness of more accurate data or clarifications. Such updates or revisions are identified in 

footnotes.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 74

Appendix B: Data Validity and Verification for FY 2017

Annual Performance Report 

The Government Performance and Results Act Modernization Act of 2010 requires 

agencies to describe the accuracy and reliability of data presented. Details of how the 

Department assesses the completeness and reliability of the data reported for FY 

2017 are presented as part of this appendix, and known limitations of the data are also 

included.

To the best of my knowledge, the data verification and validation process and the data 

sources used provide, to the extent practicable, complete, and reliable performance 

data pertaining to goals and objectives in our FY 2014-18 Strategic Plan. Through a 

process of continuous improvement, the Department regularly assesses its validation 

process and welcomes input from stakeholders.

Joseph C. Conaty 

Senior Policy Advisor

Office of the Deputy Secretary

February 12, 2018

 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 75

GOAL 1: POSTSECONDARY EDUCATION, CAREER AND TECHNICAL EDUCATION, AND 

ADULT EDUCATION. Increase college access, affordability, quality, and completion by 

improving postsecondary education and lifelong learning opportunities for youths and 

adults.

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

1.1.A Federal student 

loan delinquency 

rate

Federal Student 

Aid (FSA) Data 

Center

National Student Loan Data System (NSLDS) is currently the 

source of the data. Verification and validation occur in three 

aspects of the process:

(1) Data providers (servicers in this case) transmit data to 

NSLDS where the data are subject to edits and screening.

FSA works with the servicers to resolve anomalies.

(2) Users who analyze the data sometimes identify 

anomalies in the course of their analytic roles and work with 

experts in NSLDS to resolve. In some cases, NSLDS will 

again work with the servicer(s) to resolve.

(3) The report providing the data for this metric has been 

validated. This means that the programming and logic have 

been independently reviewed.

1.1.B Number of firsttime Free 

Application for 

Federal Student 

Aid (FAFSA) filers 

among high school 

seniors

FSA's Central 

Processing 

System

The data we use originate from FSA operational systems. 

These systems, as a part of their operations, have 

procedures in place to address potential data quality issues. 

These metrics report figures about data that our operational 

systems maintain, and we do not manipulate it in any way. If 

anomalies persist, they exist in the operational systems.

We have a consistent and disciplined process in place for 

querying our systems for data. The process includes having 

a separate data analyst from a different office within FSA 

validate the accuracy of the query and the resulting data. 

This process also is used for anomalous data.

In addition to the above controls, queries and calculations are 

simultaneously conducted on data from previous years to 

ensure technical definitions remain consistent and 

calculations by the business intelligence team as part of the 

validation.

The Customer Analytics Group is responsible for the primary 

calculation of the metric using the data from FSA. Business 

Intelligence is responsible for the technical validation of the 

metric, which is done by reviewing for accuracy the query 

used to pull the data and all calculations made with the data. 

Finally, the Financial Reporting and Analysis Branch is 

responsible for ensuring that documentation is complete and 

archived.

Since year of high school graduation is not asked on the 

FAFSA, several assumptions are made and specific criteria 

were used (such as age of applicant) to identify those likely 

to be high school seniors. These assumptions and criteria 

are applied consistently across all baseline years and future 

calculations.

These calculations also restrict the application period to the 

first nine months of the application cycle (the close of the 

fiscal year), rather than the entire 18 months. Since most 

applicants, including high school seniors, file their FAFSA 

prior to the start of the upcoming academic year (usually 

before fiscal year end), this decision better aligns the 

performance metric with the fiscal year where most of the 

performance occurred. The alternative is waiting for the close 

of the 18-month cycle, where a performance metric would 

mostly reflect performance from an earlier fiscal year.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 76

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

1.1.C Index of national 

aggregate annual 

earnings of 

Vocation 

Rehabilitation (VR) 

consumers (based 

on the number of 

competitive 

integrated 

employment 

outcomes, hours 

worked, and hourly 

wages of VR 

consumers)

RSA-911 Each grantee’s RSA-911 is submitted as a text file to RSA 

via an SFTP server. We run an edit program that converts 

the text file into MS Access and verifies the accuracy of the 

data. Each of the 215 data elements is verified. If errors are 

identified by the application, the grantee is requested to 

correct the errors and resubmit the data. Once the data file is 

corrected and resubmitted, it is checked again using the 

same process. If anomalies are identified, the grantee must 

provide an explanation and verify that the data are correct. 

RSA provides the agencies with the edit program that we use 

and are encouraged to run their data using the program 

often, at least quarterly, during the year. In addition, agencies 

have edits in their own systems which run additional checks.

1.1.D Index of national 

aggregate 

earnings of 

Transition-Age 

Youth (based on 

the number of 

competitive 

integrated 

employment 

outcomes, hours 

works, and hours 

wages of VR 

Transition-Age 

Youth)

1.1.E Number of data 

points or other 

information reports 

released on the 

FSA Data Center

Federal Student 

Aid (FSA) Data 

Center

The Validation Subject Matter Expert (SME) works with the 

Request SME to understand the solution used to fulfill the 

request, checking all results, code, and documentation 

produced by the Request SME. Additionally, the Validation 

SME ensures there are no data disclosure issues that need 

to be addressed. In the event the Validation SME cannot 

validate the request results, he or she contacts the Request 

SME and the Data Request Team (DRT) to resolve any 

issues necessary to successfully complete the validation.

The Validation SME completes his or her portion of the 

Request Template and notifies the DRT that the validation 

step is complete. Upon receipt of the data that was 

requested, the DRT confirms with the assigned Validation 

SME that the results have been validated and documented 

per their selected solution and enters it into the DRT

database

1.2.A Pell enrollment at 

IHEs with high 

graduation rates

Data from college 

Scorecard 

Validation of the data is conducted by the National Center for 

Education Statistics (NCES) through its annual IPEDS data 

validation process. NCES will ensure accuracy, while 

OPEPD/ Policy and Program Studies Service (PPSS) will 

incorporate the data into the Scorecard datasets.

Analysis will be conducted to identify anomalous data. They 

will be handled on a case-by-case basis, as most anomalies 

are due to variation from year-to-year or small n-size and are 

already pooled and suppressed before publishing the data. 

Other data problems may necessitate additional runs, 

qualifications, or suppression for stability reasons. 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 77

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

1.2.B Number of states 

that develop or 

strengthen career 

pathways policies, 

guidance, and or 

legislation

Development data 

from the National 

Skills Coalition, 

possibly 

supplemented with 

data from the 

Association of 

State Legislators 

and Center for 

Law and Social 

Policy

The Department cannot independently verify the data, but 

relies on the tracking of the National Skills Coalition. There 

are no processes in place to check for anomalous data.

1.3.A Degree attainment 

among 25-34 year 

old age cohort

NCES Digest of 

Education 

Statistics, Table 

104.30

Data available at 

https://nces.ed.gov/programs/digest/d16/tables/dt16_104.30.

asp, Number of persons age 18 and over, by highest level of 

educational attainment, sex, race/ethnicity, and age: 2016. 

Tabulated from Current Population Survey data, U.S. 

Census.

The underlying data is verified by Census prior to publication. 

Data quality and limitations are documented in:

http://www.census.gov/prod/2006pubs/tp-66.pdf. 

NCES tabulates the data, which are verified prior to

publication according to NCES guidelines.

1.3.B Persistence 

among firs-time 

filing aid recipients

FSA’s Common 

Origination and 

Distribution system

Data used for these calculations are based on counts from 

operational systems (number of recipients and number of 

applicants). Moreover, standardized queries are used to 

rerun and match calculations for earlier cycles. Therefore, the 

metric as defined should be considered verified and valid.

Data used for these calculations are based on counts from 

operational systems (number of recipients and number of 

applicants meeting certain criteria). Anomalous data would 

suggest there are recipients that are not recipients (or vice 

versa) or applicants that are not applicants (or vice versa). A 

variety of internal controls are in place tracking both of these 

processes.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 78

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

1.4.A Number of 

Science, 

Technology, 

Engineering and 

Math (STEM) postsecondary 

credentials 

awarded

Integrated 

Postsecondary 

Education Data 

System (IPEDS) 

Data Center

Doctor's Degrees: 

(324.25, 324.30 

and 324.35)

Doctor's Degrees: (324.25, 324.30 and 324.35)

https://nces.ed.gov/programs/digest/d16/tables/dt16_324.25.asp

https://nces.ed.gov/programs/digest/d16/tables/dt16_324.30.asp

https://nces.ed.gov/programs/digest/d16/tables/dt16_324.35.asp

Master's Degrees: (323.30, 323.40, 323.50)

https://nces.ed.gov/programs/digest/d16/tables/dt16_323.30.asp

https://nces.ed.gov/programs/digest/d16/tables/dt16_323.40.asp

https://nces.ed.gov/programs/digest/d16/tables/dt16_323.50.asp

Bachelor's Degrees: (322.30, 322.40, 322.50)

https://nces.ed.gov/programs/digest/d16/tables/dt16_322.30.asp

https://nces.ed.gov/programs/digest/d16/tables/dt16_322.40.asp

https://nces.ed.gov/programs/digest/d16/tables/dt16_322.50.asp

Associate's Degrees: (321.30, 321.40, 321.50)

https://nces.ed.gov/programs/digest/d16/tables/dt16_321.30.asp

https://nces.ed.gov/programs/digest/d16/tables/dt16_321.40.asp

https://nces.ed.gov/programs/digest/d16/tables/dt16_321.50.asp

Certificates: (320.20)

https://nces.ed.gov/programs/digest/d16/tables/dt16_320.20.asp

IPEDS collects completions by Classification of Instructional 

Programs (CIP) Code which may be aggregated into STEM 

and non- STEM counts. The Department of Education does 

not currently have a single definition for which CIP codes are 

STEM. 

Data quality and limitations are identified in IPEDS First Look 

Publications, “Data Collection Procedures,” and IPEDS 

methodology available at 

https://nces.ed.gov/pubs2015/2015098.pdf. Data are 

checked by NCES for consistency.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 79

GOAL 2: ELEMENTARY AND SECONDARY EDUCATION. Improve the elementary and 

secondary education system’s ability to consistently deliver excellent instruction aligned 

with rigorous academic standards while providing effective support services to close 

achievement and opportunity gaps, and ensure all students graduate high school 

college- and career-ready.

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

2.1.A Number of 

states/territories 

that have adopted 

college- and 

career-ready 

standards

ESEA Flexibility 

Monitoring

With the passage of the ESSA, the Department no longer 

collects these data.

2.1.B Number of 

states/territories 

that are 

implementing nextgeneration reading 

and mathematics 

assessments, 

aligned with 

college- and 

career-ready 

standards

ESEA Flexibility 

Monitoring

With the passage of the ESSA, the Department no longer 

collects these data.

2.3.A Disparity in the 

rates of out-ofschool 

suspensions for 

students with 

disabilities and 

youth of color 

(youth of color 

metric)

CRDC OCR strives to ensure CRDC data are an accurate and 

comprehensive depiction of student access to educational 

opportunities in school districts. The CRDC submission 

system conducts automatic edit checks to validate the format 

of the data and the system also performs business rule 

validations. For suspensions, the business rule validations 

include comparisons of disaggregated data to totals and 

enrollment counts. Districts must address the quality issues 

identified before the accuracy of the data can be certified.

Only the district superintendent, or the superintendent’s 

designee, may certify the data submission. Ultimately, the 

quality of the CRDC data depends on accurate collection and 

reporting by the participating districts.

2.3.B Disparity in the 

rates of out-ofschool 

suspensions for 

students with 

disabilities and 

youth of color 

(Students With 

Disabilities (SWD), 

IDEA only metric)

CRDC

2.4.A Number of 

persistently low 

graduation rate 

high schools

EDFacts With the passage of the ESSA, the Department no longer 

collects these data.

2.4.B Percentage of 

School 

Improvement 

Grant (SIG) 

schools in Cohort 

5 that are above 

the 25th percentile 

in mathematics, as 

measured by their 

state assessments

Analytic dataset 

produced by the 

contractor for the 

SIG National 

Summary 

With the passage of the ESSA, the Department no longer 

collects these data.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 80

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

2.4.C Percentage of SIG 

schools in Cohort 

5 that are above 

the 25th percentile 

in reading 

language art, as 

measured by their 

state assessments

Analytic dataset 

produced by the 

contractor for the 

SIG National 

Summary 

With the passage of the ESSA, the Department no longer 

collects these data.

2.5.A Number of public 

high school 

students who have 

taken at least one 

AP STEM exam

College Board/AP 

administrative 

records

The College Board does not collect or report race/ethnicity 

based on the federal guidelines. Examinees are asked to 

select one of the options noted in the data. The College 

Board Public School List is updated annually by state 

educational agencies; thus, small changes to the list over 

time are to be expected as schools open, close, and/or 

merge. Students are assigned to graduating cohorts based 

on self-reported information (i.e., grade level and/or 

graduation year) provided at the time of registration (in the 

case of SAT) or test administration (in the case of AP and 

PSAT). The College Board matches students’ data across 

programs to identify the most recent valid value when 

assigning students to cohorts.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 81

GOAL 3: EARLY LEARNING. Improve the health, social-emotional, and cognitive outcomes 

for all children from birth through 3rd grade, so that all children, particularly those with 

high needs, are on track for graduating from high school college- and career-ready.

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

3.1.A Percent of 4-year 

olds enrolled in 

state preschool 

programs

Preschool 

Development 

Grants Annual 

Performance 

Reports, 

Data Workbook, 

Table A(1)(a) and 

(b)

Annual Performance Report data are collected externally. The 

data are collected primarily through a performance report 

template and a data workbook submitted by State Project 

Directors for the grants.

Data are collected by program offices through a technical 

assistance contractor using a performance report template and 

self- administered electronic submission tool. The data 

collection period for one year is reported in the following year’s 

Annual Performance Report submission. For example, school 

year 2015-2016 data is reported in February of 2016. The 

Technical Assistance (TA) contractor performs an analysis and 

data anomalies are clarified and verified with the State Project 

Director. 

 

3.1.B Number of states 

with high–quality 

preschool 

programs 

standards

Preschool 

Development 

Grants Annual 

Performance 

Reports, 

Data Workbook, 

Table 8

 

3.2A Number of states 

that require a 

teacher with a 

bachelor’s degree 

in a state preschool 

program

3.3.A Number of states

collecting and 

reporting 

disaggregated data 

on the status of 

children at 

kindergarten entry 

using a common 

measure

EDFacts The project officers and our TA provider work with the Race to 

the Top-Early Learning Challenge grantees to verify and 

validate the data. The project officers and our TA provider work 

with the Race to the Top-Early Learning Challenge grantees to 

check for anomalous data. If anomalous data are discovered, 

project officers, the TA provider and the grantee meet and 

discuss. 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 82

GOAL 4: EQUITY. Increase educational opportunities for underserved students and reduce 

discrimination so that all students are well-positioned to succeed.

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

4.1.A National high 

school graduation 

rate

EDFacts EDFacts works with the data stewards to determine the 

appropriate business rule checks for these data. The checks 

that were done on these data include:

File validation and format checks: Identify file submissions that 

do not conform to the accepted file format, the data universe 

(school/LEA/state directory), or the reporting requirements 

(mandatory reporting fields).

Submission Edit Business Rules: Submission edits and 

warnings ensure that the data meet or exceed an acceptable 

level of reasonability by checking the values entered in a field 

against other similar values in the same file or across files. If a 

discrepancy is found (i.e., a value falls outside of the 

acceptable range), a submission error or warning is issued. 

Unlike format and validation edits, submission edits and 

warnings are programmed into the EDFacts Submission 

System and applied to the data after they are in the staging 

database.

Coordinated Data Quality Reviews (CDQR): Identify potential 

errors and anomalies related to completeness, consistency, 

and comparability in the file submissions that would affect the 

quality and usability of data in the files.

CDQR Process: At the end of a collection period, EDFacts runs 

checks to validate data quality of submitted data and presents 

all results to program offices. It is the responsibility of the 

program offices, in consultation with the EDFacts staff, to 

determine which identified errors to escalate to states for 

further review. The EDFacts Partner Support Center (PSC) 

sends these errors to states for remediation, which take the 

form of explanations for data anomalies, data file submission 

updates, and data corrections. OESE should be considered the 

point-of-contact for identifying which errors were escalated and 

the result of those escalations.

Path to public release and national rates: For each collection 

cycle states report their calculated cohort counts and ACGR 

graduation rates. These counts and rates are tested against a 

number of submission edit business rules and the Adjusted 

Cohort Graduation Rate (ACGR) Coordinated Data Quality 

Review for format, consistency, completeness, and 

comparability. Those rule checks are delivered to the OESE for 

follow-up correspondence through the EDFacts PSC with the 

states. The program office, in consultation with NCES EDFacts

staff and other stakeholders, meet to determine which issues 

identified by the business rules should be raised with the state 

for explanation, update, or correction. Following that review, 

data are then aggregated to the national level for the purpose 

of calculating and publishing a national rate. NCES processes 

these data to force conformity of reporting categories (mapping 

reported “Major Racial/Ethnic Groups” to the traditional five 

racial/ethnic groups) and addresses any issues raised by 

missing data. After imputations are made, NCES produces a 

national rate for the country as a whole; a rate which is 

representative of every state. Once produced, NCES 

documents any remaining issues with these data, and the 

aggregation and imputation methodology in a public report. 

That report, documentation, and the associated data tables are 

put through several stages of review including independent 

4.1.B Gap in the 

graduation rate 

between all 

students and 

students from lowincome families

EDFacts

4.1.C Number of schools 

that do not have a 

gap or that 

decreased the gap 

between students 

from low-income 

families and the 

state average for 

all students

EDFacts
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 83

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

reviews at the division (Administrative Data), center (NCES), 

and principal office (IES) levels prior to public dissemination.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 84

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

4.2.A Average number of 

cases substantively 

resolved, per 

investigative staff 

member

Office for Civil 

Rights Case 

Management 

System (CMS) 

system and human 

resources 

dashboard

OCR captures up-to-date information related to substantive 

closures of cases through its Case and Activity Management 

System (CAMS). The CAMS is a formal IT Investment, which 

includes a business case and its own performance metrics 

tracked through ED’s IT Investment Review process. The 

CAMS includes the Case Management System (CMS), Activity 

Management System, and Document/Record Management 

(DM/RM) components. In order to track this Metric, the staff in 

OCR’s 12 enforcement offices are responsible for updating and 

maintaining accurate case statuses in the CAMS’s CMS. This 

specific Metric data is extracted from the CAMS for reporting 

purposes by OCR’s IT Specialist who is also the CAMS Project 

Manager and technical expert. This staff person then delivers 

the data and analysis to the Goal 4 Strategic Goal Leader for 

reporting.

The verification and validation of CAMS data utilize (1) rules 

built into the CAMS’s CMS component; (2) periodic checks on 

questionable data; (3) guidance and reference on data entries; 

and (4) regular training and guidance for primary users 

imputing the data.

(1) Rules built in CMS: 

Samples of CMS validations: 

- Docket number is assigned by the CMS based on four 

required data (Active Office, TA Type, Nature/Type of 

Assignment, and Initial Contact/Target).

- Entry date must be validated.

(2) Periodic checks on questionable data: (quarterly)

- Missing entries: such as TA completion without (a) date of 

final activity, (b) activity log including subject area, completion 

date and location.

(3) Guidance and reference materials for use by primary users 

making data entries are posted at the OCR SharePoint site and 

are available to all OCR staff:

- CMS Data Entries requirements.

- CMS User Manual.

(4) Users Training:

User training is available as needed. OCR also provides 

training through a “peer helper” or “train the trainer” model for 

staff in the field offices and headquarters.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 85

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

4.2.B Number of 

technical 

assistance 

presentations, 

issuance of press 

releases, and 

issuance of 

resolution 

agreements on 

OCR work

Office for Civil 

Rights Case 

Management 

System (CMS) and 

website

The verification and validation of CAMS data utilize (1) rules 

built into the CAMS’s CMS component; (2) periodically checks 

on questionable data; (3) guidance and reference on data 

entries; and (4) regular training and guidance for primary users 

inputting the data.

(1) Rules built in CMS: 

Samples of CMS validations: 

- Docket number is assigned by the CMS based on four 

required data (Active Office, TA Type, Nature/Type of 

Assignment, and Initial Contact/Target).

- Entry date must be validated.

(2) Periodic checks on questionable data: (quarterly)

- Missing entries: such as TA completion without (a) date of 

final activity, (b) activity log including subject area, completion 

date and location.

3) Guidance and reference materials for use by primary users 

making data entries are posted at the OCR SharePoint site and 

are available to all OCR staff:

- CMS Data Entries requirements.

- CMS User Manual.

(4) Users Training: User training is available as needed. OCR 

also provides training through a “peer helper” or “train the 

trainer” model for staff in the field offices and headquarters.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 86

GOAL 5: CONTINUOUS IMPROVEMENT OF THE U.S. EDUCATION SYSTEM. Enhance the 

education system’s ability to continuously improve through better and more widespread 

use of data, research and evaluation, evidence, transparency, innovation, and technology.

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

5.1.A Number of public 

data sets included 

in the Department’s

Data Inventory and 

thus linked to 

Data.gov or 

ED.gov websites

Data Strategy 

Team Data 

Inventory and the 

public 

Department’s Data 

Inventory at 

http://datainventory

.ed.gov

The data are validated with a crosswalk between Inventory 

entries and the listing of the Department’s public datasets, 

ensuring that the data described in the Department’s Data 

Inventory are publicly available at the identified web address.

5.1.B Number of states 

linking K-12 and 

postsecondary 

data with workforce 

data

Statewide

Longitudinal Data 

Systems (SLDS) 

grant monitoring 

(monthly updates 

from states, annual 

performance 

reports, final 

performance 

reports, and site 

visits) 

Data are collected through monitoring of states with active 

SLDS grants. The data are limited to this population. While 47 

states and territories have received these grants over time, by 

June 2015, there will be fewer than 25 states with active 

grants, which leads to incomplete and not up-to-date data from 

states that either have not received grants or that do not have 

active grants.

If the state is focusing on an early childhood data system, the 

state’s reporting might reflect only those development efforts, 

and not statewide P20W (preschool to college workforce) 

development efforts. There are additional sources for 

information about state data linkages in the field. For example, 

the Data Quality Campaign (DQC) conducts surveys of state 

capacity to collect, store, link, and use data. Because the 

Department does not use the same definitions as DQC, our 

figures tend to be lower than theirs. 

For example, we require that a state possess the capacity to 

follow its own students in order for us to report that the state 

has the linkage in place. Similarly, we enable states to report 

on whether particular linkages are planned, in progress, or 

complete, and report that a state has a linkage when the state 

reports that the project is complete; DQC might give a state 

credit for an ‘in progress’ or pilot-stage linkage.

A survey administered to the universe of states and territories 

would enable more a systematic collection data about all 

states’ capacity for data linkages and data use. There is a 

concern, however, that if those data were to be used for public 

reporting, states might begin to overstate their capacities, 

particularly on data that are also publicly reported by 

organizations such as DQC.

Currently, data from monitoring are used in an iterative, 

formative approach to program improvement; our technical 

assistance program is designed to support states’ efforts to 

improve their systems. This relies on states being honest about 

their own internal capacities.

5.1.C Number of states 

actively using data 

systems to support 

and inform 

improvements

Statewide

Longitudinal Data 

Systems (SLDS) 

grant monitoring 

(monthly updates 

from states, annual 

performance 

reports, final 

performance 

reports, and site 

visits)

Data for this metric were independently assessed by SLDS 

program staff using state progress reports, monitoring call 

summaries and site visit write-ups to determine which states 

are actively using their data systems to inform improvements. 

An operational definition for ”active use” was agreed upon and 

is used consistently by SLDS staff to avoid bias.

Definitions for active use are based upon tangible deliverables 

within the grant program. By explicitly counting the number of 

grantee states with deliverables that meet the definition’s 

criteria we are able to accurately measure the explicit behavior 

described in the metric.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 87

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

5.1.D Web traffic to the 

College Scorecard

Google Analytics 

data from College 

Scorecard 

The data are simply pulled from the GSA Google Analytics 

account. The data are checked for reasonability, and 

incorporated alongside the existing baseline data. The data 

need to be matched with baseline data and collated 

accordingly. There is limited analysis required.

5.2.A Average time to 

close “cases” 

(PTAC + FPCO)

Case Tracking 

System (CTS) 

Monthly Metric 

Reports

The term “case” refers to requests for quick, informal 

responses to routine questions related to student privacy. 

These requests are received via email, the Family Policy 

Compliance Office (FPCO) / Privacy Technical Assistance 

Center (PTAC) resource website, or by telephone and 

subsequently entered into the CTS. In contrast, 

“correspondence and complaints” refers to written complaints 

of alleged failures to comply with Family Educational Rights 

and Privacy Act / Protection of Pupil Rights Amendment filed 

with FPCO; requests for formal written guidance/interpretation 

laws administered by FPCO; and, to the reporting of instances 

of data breaches by educational agencies and institutions. 

These inquiries are logged into the Correspondence Control 

Manager System, given a tracking control number, and 

assigned to FPCO staff.

The preliminary data for this metric are reviewed at least 

weekly for verification. If anomalous data are identified in the 

periodic reviews or when anomalies are suspected, individual 

cases are examined individually to identify if they were properly 

closed or if their status was entered incorrectly. When 

appropriate, corrections are made. Staff responsible for 

entering data into the CTS will continue to be trained on 

policies and procedures.

The monthly metric reports are scrutinized by the Director of 

FPCO, the Contracting Officer Representative for the PTAC 

contractors, and the Department's Chief Privacy Officer, to 

assure completeness and reliability of data and to recommend 

any improvements to the CTS or modifications to the standard 

operating procedures. The quarter entry represents the fiscal 

year to date average days to close as of the end of that quarter 

taken from the corresponding monthly report.

5.3.A Percentage of 

select new (noncontinuation) 

discretionary grant 

dollars that reward 

evidence

Forecast Report 

issued by the 

Office of the Chief 

Financial Officer 

(OCFO) and final 

Funding Reports 

from relevant 

programs

In determining which discretionary grants are considered 

“evidence-based” (i.e., the numerator when calculating the 

percentage), the Department includes all programs that 

rewarded applicants with supporting evidence of promise or 

better (per the Education Department General Administrative 

Regulations (EDGAR) evidence framework). This could be 

done through a competitive preference or absolute priority, an 

eligibility requirement, or a selection factor. Only the amounts 

of the grants awarded for those projects were counted. In 

determining what counts as discretionary funding (i.e., the 

denominator when calculating the percentage), the Department 

includes all programs for which the EDGAR evidence 

framework could conceivably work. In Fiscal Year 2015, the 

Department counted all discretionary grant programs except for 

those programs run through the Institute of Education Sciences 

(IES), which are already evidence-based and would not be 

candidates for the EDGAR evidence framework). 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 88

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

5.3.D Number of 

completed project 

evaluations from 

grantees of select 

discretionary grant 

programs in a 

given fiscal year 

that meet What 

Works 

Clearinghouse 

(WWC) Evidence 

Standards

Discretionary grant 

slate memoranda, 

discretionary grant 

financial forecasts 

and reports from 

OCFO, and the 

What Works 

Clearinghouse

We expect that it will take about five minutes to search the 

WWC database for new Investing in Innovation Program (i3) 

studies each quarter. We are also considering ways in which 

IES staff and OII staff can work closely to report these data 

quickly.

We do not have formal verification or validation processes in 

place. We do not have formal processes in place to check for 

anomalous data. So far, we have not encountered anomalous 

data. 

We note that we will report out on studies that are funded by i3 

that are in the WWC database and meet WWC Evidence 

Standards. We need to be careful not to imply that these 

studies all reflect positive results – in actuality, the WWC 

Evidence Standards only measure the rigor of impact 

evaluations and do not consider the actual outcomes reported.

5.4.A Percentage of 

schools in the 

country that have 

actual internet 

bandwidth speeds 

of at least 100 

Mbps

Education 

Superhighway

The Department uses an external data source for this metric 

and relies on the external, third party’s verification and 

validation methodology. We believe these data are the best 

currently available and provide useful information to gauge 

progress on this metric. ESH analyzes publicly available data 

from the Universal Service Administrative Company, an 

independent, not-for-profit corporation created by the Federal 

Communications Commission, confirms data accuracy through 

outreach and verification processes, and further analyzes the 

total IA bandwidth as compared to the National Center for 

Education Statistics data on student population in districts.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 89

GOAL 6: U.S. DEPARTMENT OF EDUCATION CAPACITY. Improve the organizational 

capacities of the Department to implement this strategic plan.

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

6.1.A Percent of 

selections made 

per job opportunity 

announcement

EDHires (Monster’s 

electronic hiring 

management 

system) 

The data are entered into the EDHires system by customers, 

contractors and staff, so quality of data is only as reliable as 

the information entered and/or updated in the system. 

Data are reviewed by HR Specialists for relevance and 

completeness to ensure correct parameters and filters have 

been applied. 

If anomalies are identified, HR Specialists will compare 

contractor generated reports against ad-hoc hiring reports 

generated from the system of record and other linked HR 

systems. Any questionable results would be brought to the 

contractor’ attention; in-turn, Office of Human Resources 

(OHR) would work with the contractor to obtain clarity and/or 

resolve.

6.1.B EVS Employee 

Engagement Index

OPM Federal 

Employee 

Viewpoint Survey 

(FEVS) 

Any questionable FEVS results would be brought to the 

attention of OPM; the Department would then work with the 

OPM point of contact to obtain clarity. 

The Engagement index score is calculated by OPM by first 

determining the percent positive for each of the 15 items in the 

sub-indices (i.e., Leaders Lead, Supervisors, and Intrinsic Work 

Experiences). Then the unrounded percent positive scores are 

averaged to obtain individual sub-index scores. The unrounded 

sub-index scores are then averaged to obtain the overall index 

score. The overall index score is then rounded for reporting 

purposes.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 90

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

6.1.C Time to hire Federal 

Personnel/Payroll 

System (FPPS) 

Datamart

The Office of Management (OM) identified a gap between 

hiring procedures and the 90 Day Hiring Model’s time to hire 

calculations which necessitated an adjustment in the way time 

to hire is calculated for some actions. OM found that 

recruitment work is often initiated prior to the receipt of an 

“official” action. While this is a proactive customer-centric 

approach, especially prevalent in ED’s non-competitive hiring 

activities, it can sometimes result in a negative time to hire or a 

time to hire of zero – neither of which reflects a realistic or 

meaningful hiring lead time. 

To account for, track and document this upfront work, OM 

developed an adjustment mechanism to better gauge hiring 

lead time in these cases. For ease of comparison, the 

adjustment was applied to all FY16 hiring actions and resulted 

in a revised Q1 actual. 

To ensure clarity and consistency in the application of time to 

hire methodology moving forward, OM also clarified time to hire 

calculation protocols as follows: 

Individual time to hire:

• Time to hire for individual hiring actions is determined 

by calculating the number of days between the 

Request to Recruit (ROE) Date and the Verbal 

(tentative) Offer Date, as reflected in FPPS DataMart.

o If time to hire results in a negative or zero, it 

is replaced with the value 1.

o If time to hire cannot be determined (i.e., 

date not available), it is excluded from 

agency calculations.

• Time to hire is not calculated for actions In-Progress; 

it is only calculated for Completed actions (i.e., 

estimated EOD established).

Agency time to hire:

• The Department’s time to hire is an annual rate 

determined by calculating the percent of hiring actions 

within a fiscal year completed within 90 days.

• Q1, Q2, and Q3 actuals are cumulative, measured 

from the beginning of the fiscal year. 

• Q4 actuals represent the Department’s annual time to 

hire rate and are reported in the Annual Performance 

Report.

• Periodic snapshots of time to hire may be computed 

monthly, quarterly or by principal office to help gauge 

progress toward the annual target; however, care 

should be taken to ensure periodic updates are clearly 

identified as snapshots or progress indicators and not 

represented as the agency’s annual rate.

6.1.D Effective 

Communication 

Index

Office of Personnel 

Management 

(OPM) Federal 

Employee 

Viewpoint Survey 

(FEVS)

Any questionable FEVS results would be brought to the 

attention of OPM; the Department would then work with the 

OPM point of contact to obtain clarity. 

This index score is calculated by the Department by averaging 

the percent positive scores from OPM FEVS questions 53, 58, 

and 64.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 91

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

6.2.A Percentage of 

Department grant 

recipients without 

any Single Audit 

Findings

Federal Audit 

Clearinghouse 

(FAC) 

The FAC is the primary data source. OCFO and FSA import 

and manage Single Audits in Accountability and Audit 

Resolution Tracking System (AARTS) and EZ-Audit 

respectively. Reporting for this metric is based on downloading 

FAC data from their web portal.

Simple calculations are made with data that are downloaded 

from FAC that include subtracting the total number of audits 

with findings from all audits received, then dividing to determine 

the overall percentage. We do not expect the calculations to be 

time consuming, so data can be reported to the Performance 

Improvement Office each quarter. However, quarterly data will 

be more variable than year-end data given the cycle of when 

audits are submitted to the FAC.

Metric relies on public data from FAC database. The FAC 

maintains controls to verify and validate data, including 

procedures to review the completeness of audits before they 

are formally accepted. OCFO will obtain a copy of those 

procedures to support this metric.

OCFO maintains an interagency agreement with the FAC and 

internal procedures to review and verify the accuracy of all 

audits with findings that are imported into AARTS for 

resolution. FAC staff manually review and correct coding errors 

in the audit data collection form. OCFO staff review and correct 

any errors in the imported audit file before it is assigned for 

resolution. These procedures are documented. Moving 

forward, OCFO also will use AARTS and EZ-audit data to 

reconcile the total number of audits that have findings with the 

total generated through the FAC report.

6.2.B Compliance rate of 

contractor 

evaluation 

performance 

reports

Past Performance 

Information 

Retrieval System 

(PPIRS) 

Compliance Metric 

Report

Compliance rates of contractor performance evaluations are 

set by OMB and are calculated by use of a Government wide 

reporting tool available in the PPIRS (www.ppirs.gov). 

Office of the Chief Financial Officer (OCFO) staff analyzes the 

PPIRS report weekly to ensure that each contractor 

performance evaluation reflected on that report should appear 

on that report, and to rectify any errors on the report.

6.3.A Overall average 

impact score of the 

Department’s 

technical 

assistance in 

helping build state 

capacity to 

implement 

education reforms

Annual Grantee 

Satisfaction Survey

The contractor has a quality control system and the results are 

also provided to all of the programs that participate in the 

survey to identify issues/anomalies with the data. 

Program staff report anomalies to the contractor to correct.

The data comes from the Annual Grantee Satisfaction Survey 

Report (See 

http://www2.ed.gov/about/reports/annual/gss/index.html). The 

report is produced by a contractor; the contract is managed by

OCFO. 

One major issue with the Grantee Satisfaction Survey is the 

timing of the survey. The survey is administered in late 

spring/early summer when many staff are out of the office 

which suppresses the response rate. Also, some grant 

programs do a better job than others of promoting the survey 

and showing how the survey results have been used to make 

improvements. We’ve been told that the survey is administered 

late each year for the following reasons:

The Interagency Agreement can’t be put into clearance until 

the budget is approved each year, getting the Interagency 

Agreement through Departmental clearances takes several 

months, program staff are asked to review their customized 

survey questions and submit their grantee contact lists at the 

busiest time of the year which makes it difficult to turn them 

around quickly. 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 92

Metric 

No.

Performance 

Metric (Indicator) Data Source(s) Data Verification and Validation Overview

6.4.A Number of the 

Department’s IT 

security incidents

Cyber Security 

Assessment and 

Management 

(CSAM) and RSA 

Security 

Operations 

management 

(SecOps) 

Office of the Chief Information Officer (OCIO) has standard 

operating procedures (SOP) to verify and validate the data: 1) 

CSAM SOP, 2) Incident Response SOP, and 3) SecOps SOP. 

This process is executed on a weekly basis. Further, the data 

are pulled from the Department’s authoritative source (system 

of record) for Incident Response and Tracking. In addition, 

standard operate procedures (SOPs) enforce the use of this 

capability for entering, tracking, and managing all incident 

related activity. The data are also audited on a routine basis to 

ensure consistency and accuracy.

6.4.B EVS Results 

Oriented 

Performance 

Culture Index

OPM Federal 

Employee 

Viewpoint Survey 

(FEVS) 

Data verification and validation process is integrated into the 

OPM FEVS survey results validation process. The ResultsOriented Performance Culture Index score is calculated by 

OPM by first determining the percent positive for each of the 13 

items in the index. Then the unrounded percent positive scores 

are averaged across all index items to get the index score. The 

index score is then rounded for reporting purposes. Any 

questionable FEVS results would be brought to the attention of 

OPM; the Department would then work with the OPM point of 

contact to obtain clarity.

6.4.C EVS Leadership 

and Knowledge 

Management Index

OPM Federal 

Employee 

Viewpoint Survey 

(FEVS) 

Data verification and validation process is integrated into the 

OPM FEVS survey results validation process. The Leadership 

and Knowledge Management Index score is calculated by 

OPM by first determining the percent positive for each of the 12 

items in the index. Then the unrounded percent positive scores 

are averaged across all index items to get the index score. The 

index score is then rounded for reporting purposes. Any 

questionable FEVS results would be brought to the attention of 

OPM; the Department would then work with the OPM point of 

contact to obtain clarity.

6.4.D Total usable 

square footage

Department's 

Master Space 

Management Plan

The Department reconciles its rent bills per building on a 

monthly basis. Every six months the usable square footage is 

verified with GSA.

Because usable square footage relates directly to rent costs, 

the Department uses the same data verification and validation 

procedures. Specifically, the data are collected directly from 

Occupancy Agreements and rent bills per building.

The data are derived from historic examples and relevant 

experience. Department leadership has agreed to a set of 

assumptions by which the data are based. Leadership has 

reached out to subject matter experts to broaden the scope of 

the data set, and lower risks of missing contingencies that may 

affect the data. At each step, the data are reviewed 

independently to double check the work of each team member 

and provide quality control. These processes help ensure the 

data’s completeness and reliability.

6.4.E Rent cost Department's 

Master Space 

Management Plan

Data are collected directly from Occupancy Agreements and 

rent bills per building. The actual rent may vary significantly if 

the Department relocates to a new leased building and/or signs 

short lease extensions. The Department is leveraging the 

examples and experience of the mobility labs and building 

consolidations programs.

The Department reconciles its rent bills per building on a 

monthly basis. Every six months, leadership will re-evaluate the 

data, the assumptions on which it is based, and incorporate 

actual costs and project schedules. These steps will become 

part of our quality assurance program and procedures. 

Leadership looks to improve completeness, reliability, and 

quality of the data at these milestones.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 93

Appendix C: Data Validity and Verification for FY 2019 

Performance Metrics

The Government Performance and Results Act Modernization Act of 2010 requires agencies to 

describe the accuracy and reliability of data presented. Details of how the Department plans to 

assess the completeness and reliability of the data reported for the FY 2019 performance 

metrics are presented as part of this appendix, and known limitations of the data are also 

included.

To the best of my knowledge, the data verification and validation process and the data sources 

used provide, to the extent practicable, complete, and reliable performance data pertaining to 

goals and objectives in our FY 2018-22 Strategic Plan. Through a process of continuous 

improvement, the Department regularly assesses its validation process and welcomes input 

from stakeholders.

Joseph C. Conaty 

Senior Policy Advisor

Office of the Deputy Secretary

February 12, 2018
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 94

Strategic Goal 1: Support state and local efforts to improve learning outcomes for P-12 

students in every community.

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

1.1.A Number of open and 

operating charter schools 

supported by CSP

National Center 

for Education 

Statistics, CCD

(used to identify 

all charter 

schools and 

enrollees) and 

grantee annual 

performance 

reports (used to 

identify the 

subset of charter 

schools in CCD 

that receive CSP 

support)

Program officers will review and verify data as part 

of their project monitoring. Program officers will 

review reported data and confirm the accuracy with 

the grantees. The Charter School Program (CSP) 

team will work closely with their contractor to 

compare data in the CCD with data in grantees’ 

Annual Performance Reports (APRs) to confirm the 

data. The CSP director is responsible for certifying 

that the data are accurate.

1.1.B Number of students 

enrolled in charter schools 

supported by CSP

1.1.C Number of new resources 

on evidence-based and 

promising practices related 

to school choice 

disseminated

National Charter 

School Resource 

Center and IES 

sponsored 

materials

Verification and validation of data is based on 

review and approval of resources (promising 

practices documents, papers, etc.). The CSP 

Director is responsible for certifying that the data 

are accurate.

1.1.D Number of students 

enrolled in magnet schools 

Grantee annual 

performance 

reports

The Department relies on a contractor to conduct 

data reviews of grantee performance data. The 

contractor is responsible for locating, assessing, 

and recording annual performance data. Review 

protocols include checking and documenting the 

presence and completeness of performance 

measure data for analysis. The contractor 

maintains databases with classification and coding 

schemes in Microsoft Excel and SPSS to analyze 

grantee data, including student enrollment. The 

contractor addresses anomalous data during the 

data verification and validation process. The 

contractor’s research staff discusses anomalous 

data with grantees, who verify or correct the data. If 

the data are correct as reported, the contractor will 

document and clarify the anomaly in the report. The 

Parental Options and Improvement Programs (POI) 

Director is responsible for certifying that the data 

are accurate.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 95

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

1.2.A Percentage of states that 

show improvement in the 

percentage of students in 

grades 3-8 scoring at or 

above proficient on state 

assessments in reading in 

all of the following 

subgroups: economically 

disadvantaged, children 

with disabilities, English 

learners, migrant, 

homeless, and major 

racial/ethnic groups

Assessment Data 

File that includes 

state reported 

data pulled from 

EDFacts files 

C175, C178, 

C185, and C188

The Office of Elementary and Secondary Education 

(OESE), Office of Special Education Program 

(OSEP), and EDFacts conduct a thorough, 

coordinated data quality review of the assessment 

data submitted by states. OSEP reviews this set of 

assessment data files for the purposes of IDEA 

Section 618 and OESE reviews this set of 

assessment data files for the purposes of the 

Consolidated State Performance Report (CSPR). 

The review includes the following three areas: 

timeliness of the data submission, completeness of 

the data files, and accuracy of the data. 

OESE, OSEP, and EDFacts conduct three data 

quality reviews of states’ assessment data 

submissions. The first two data quality reviews 

result in data quality inquiries, comments, and 

questions that are sent to the states for responses. 

A state may resubmit the data to address the data 

quality inquiries, provide a written response, or a 

data note to address the data quality inquiry. The 

three-data quality review is conducted to ensure 

that the Department is publishing and using a set of 

assessment data that meets our data quality 

criteria. 

Additionally, OESE, OSEP, and EDFacts will hold 

conference calls with states to gain better 

understanding of anomalies that are identified in 

the data submissions or to provide technical 

assistance to the state to submit higher quality 

assessment data

These data quality reviews are conducted after 

each of the due date/ resubmission dates for these 

data. If there are questions regarding whether the 

data submitted by a state are accurate remain then 

the Department may decide not to publish or use 

the data.

Note that this metric is impacted by changes to 

state assessment systems. If states change 

assessments, performance levels, cut scores, etc., 

it will invalidate the year to year analysis to identify 

states that showed improvements in the 

percentage of students proficient. Since statewide 

assessment systems have been influx for the last 

few years, it makes it challenging to establish 

baselines and set targets.

States sign a certification when submitting their 

CSPR.

If questions remain upon completion of the 

Department’s data quality review process regarding 

whether the data submitted by a state are accurate, 

the Department may decide not to publish or use 

the data.

Note that this metric is impacted by changes to 

state assessment systems. If states change 

assessments, performance levels, cut scores, etc., 

it will invalidate the year to year analysis to identify 

states that showed improvements in the 

percentage of proficient students. Since statewide 

assessment systems have been influx for the last 

few years, it makes it challenging to establish 

baselines and set targets.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 96

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

1.2.B Percentage of states that 

show improvement in the 

percentage of students in 

grades 3-8 scoring at or 

above proficient on state 

assessments in 

mathematics in all of the 

following subgroups:

economically 

disadvantaged, children 

with disabilities, English 

learners, migrant, 

homeless, and major 

racial/ethnic groups

Assessment Data 

File that includes 

state reported 

data pulled from 

EDFacts files 

C175, C178, 

C185, and C188

OESE, OSERS, and EDFacts conduct a very 

thorough CDQR of the assessment data submitted 

by states. OSERS reviews this set of assessment 

data files for the purposes of IDEA Section 618 and 

OESE reviews this set of assessment data files for 

the purposes of the CSPR. The review includes the 

following three areas: timeliness of the data 

submission, completeness of the data files, and 

accuracy of the data. Through the coordinated 

review, the states receive one set of data quality

comments or inquiries associated with the 

assessment data from the Department. 

OESE, OSERS, and EDFacts conduct three data 

quality reviews of states’ assessment data 

submissions. The first two data quality reviews 

result in data quality inquiries, comments, and 

questions that we sent back to states for 

responses. A state may resubmit the data to 

address the data quality inquiries and/or provide a 

written response and/or a data note to address the 

data quality inquiry. The three-data quality review is 

conducted to ensure that we are publishing and 

using a set of assessment data that meets our data 

quality criteria. 

These data quality reviews are conducted after 

each of the due date/ resubmission dates for these 

data. Typically, these due date/ resubmission dates 

occur in December, February/ March, and April. 

OESE, OSERS, and EDFacts identify anomalous 

data via the coordinated data quality review noted 

in question 11 (above). Once anomalous data is 

identified via the data quality review and analyses, 

states are given the opportunity to resubmit the 

data or provide an explanation for the anomalous 

data. Additionally, OESE, OSERS, and EDFacts

will hold conference calls with states to gain better 

understanding of anomalies that are identified in 

the data submissions and/or to provide technical 

assistance to the state to submit higher quality 

assessment data. 

If there are questions regarding whether the data 

submitted by a state are accurate after the midApril freeze date, OESE and/or OSERS may 

decide not to publish or use the data.

This measure will be impacted by changes to state 

assessment systems. If states change 

assessments, performance levels, cut scores, etc., 

it will invalidate the year to year analysis to identify 

states that showed improvements in the 

percentage of students proficient. Since statewide 

assessment systems have been influx for the last 

few years, it makes it challenging to establish 

baselines and set targets.

For states that have missing or inaccurate data, 

there are opportunities to resubmit their data files 

and have them reviewed prior to publication. 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 97

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

1.2.C Percentage of states with 

80% or more of 

preschoolers with 

disabilities that showed 

greater than expected 

growth in positive socialemotional skills (including 

social relationships) by the 

time they exit IDEA Part B, 

section 619 services

IDEA Part B 

State Annual 

Performance 

Reports

OSEP State Leads review each states’ data for 

data quality issues once the annual performance 

report is received. The MSIP Data Implementation 

Team look across all states to review the data for 

any anomalies. OSEP’s technical assistance 

center, which has expertise on early childhood 

outcome measurement systems, also reviews the 

data for quality issues. The center publishes a brief 

on how they analyze the data 

http://ectacenter.org/eco/. The number of states 

collecting high-quality data has increased over 

time. In addition, states continue to build their 

capacity to collect valid and reliable data. These 

efforts are supported by the aforementioned TA 

center, which helps states build and improve their 

outcome measurement systems, collect and 

analyze data, and use data to make program 

improvements. States certify that the data they turn 

in to OSEP is accurate. The OSEP Director signs 

the determination letter for each state.

1.2.D Percentage of schools in 

the country that have actual 

internet bandwidth speeds 

of at least 100 Mbps

Education 

Superhighway

Verification and validation of data is managed by 

Education Superhighway. 

https://s3-us-west-1.amazonaws.com/esh-sotspdfs/methodology_2017_state_of_the_states.pdf. 

The process for checking for anomalous data is 

Education Superhighway. The Office of Education 

Technology (OET) is responsible for certifying that 

the data are accurate. 

1.2.E Percentage of rural schools 

connected to a broadband 

infrastructure capable of 

scaling to 10 gigabits per 

second
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 98

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

1.2.F Percentage of states 

publishing report cards on 

the preceding school year 

in a timely manner* 

*Timeliness defined as 

report cards that are 

published by January 15th 

of the year following the 

reporting year (e.g., report 

cards on the 2017-18 

school year would be timely

if published by January 15, 

2019) 

Consolidated 

State 

Performance 

Report (CSPR)

The data will be collected through the CSPR. There 

is a formal, extensive process for the verification 

and validation of any information submitted through 

EDFacts/CSPR which occurs annually, beginning 

immediately following the submission due date, in a 

series of review windows between December and 

May. Data relevant to OSS are reviewed by OSS 

program staff, and the results of the process are 

further reviewed by the OSS Data Team, OESE 

Front Office, and EDFacts team. All staff engaged 

in this process are trained in the content and review 

process for these data. Due to the scope and 

complexity of the process, most data included in 

the CSPR are not considered final and available for 

use until May. However, since the review of the 

report card link will only involve navigating to the 

web location and confirming that a current report 

card is posted, OSS will be able to use preliminary 

data, which will be available earlier in the year, in 

order to produce this metric. Data will be reviewed 

by OESE prior to being submitted. Anomalous data 

will be checked and verified by OESE or other 

supporting offices. The ESSA report card data 

metrics are new to states and some states may be 

working through the programming necessary to 

report them and therefore may be delayed for the 

first couple of years of implantation in the release of 

their data. However, the data to be collected by the 

Department via CSPR should be standard. States 

are responsible for certifying the accuracy of data 

submitted through the CSPR. The data are further 

reviewed by OSS to identify any anomalies before 

being used in any public products.

1.2.G Percentage of monitored 

states reporting Information 

on each indicator in the 

state’s accountability 

system and the list of 

schools identified for 

comprehensive support and 

improvement and targeted 

support and improvement.

OESE Title I 

monitoring 

reports of ESSA 

State report cards

The data will be collected during Title I monitoring 

by OESE. Monitoring reports are reviewed both 

internally by staff in OSS and OESE and submitted 

to states as part of the monitoring process.

Program officers doing the monitoring are trained 

for consistent and standard application of 

monitoring protocols. Data will be reviewed by 

OESE prior to being submitted. Anomalous data 

will be checked and verified by OESE or other 

supporting offices. The ESSA report card data 

metrics are new to states and some states may be 

working through the programming necessary to 

report them. However, the data to be collected by 

ED via monitoring should be standard. OESE is 

responsible for monitoring the Title I program and 

follow the protocols necessary to ensure the data 

are accurate.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 99

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

1.3.A Number of discretionary 

grant notices with STEM as 

a priority 

Program offices 

holding 

discretionary 

grant 

competitions 

each year 

(including OESE, 

OII, OSERS, 

OPE, OCTAE, 

IES, and OELA)

Reports can be checked against the NIA for each 

competition. Each program office listed in the data 

source is responsible for certifying that the data are 

accurate. 

1.3.B Number of public high 

school graduates who have 

taken at least one 

Advanced Placement (AP) 

Science, Technology, 

Engineering, Mathematics 

(STEM) exam* while in high 

school

*Including the following 

exams: Biology, Chemistry, 

Environmental Science, 

Physics B, Physics C: 

Electricity and Magnetism, 

Physics C: Mechanics 

Based, Calculus AB, 

Calculus BC, Computer 

Science A, and Statistics

College Board College Board addresses all data quality issues 

and is responsible for certifying that all data are 

accurate.

The College Board does not collect or report 

race/ethnicity based on the federal guidelines. 

Examinees are asked to select one of the options 

noted in the data. The College Board Public School 

List is updated annually by state educational 

agencies; thus, small changes to the list over time 

are to be expected as schools open, close, and/or 

merge. Students are assigned to graduating 

cohorts based on self-reported information (i.e., 

grade level and/or graduation year) provided at the 

time of registration (in the case of SAT) or test 

administration (in the case of AP and PSAT). The 

College Board matches students’ data across 

programs to identify the most recent valid value 

when assigning students to cohorts.

1.3.C Number of public high 

school graduates who have 

taken at least one 

Advanced Placement (AP) 

Science, Technology, 

Engineering, Mathematics 

(STEM) exam* while in high 

school and score a 3 or 

better

*Including the following 

exams: Biology, Chemistry, 

Environmental Science, 

Physics B, Physics C: 

Electricity and Magnetism, 

Physics C: Mechanics 

Based, Calculus AB, 

Calculus BC, Computer 

Science A, and Statistics

College Board College Board addresses all data quality issues 

and is responsible for certifying that all data are 

accurate.

The College Board does not collect or report 

race/ethnicity based on the federal guidelines. 

Examinees are asked to select one of the options 

noted in the data. The College Board Public School 

List is updated annually by state educational 

agencies; thus, small changes to the list over time 

are to be expected as schools open, close, and/or 

merge. Students are assigned to graduating 

cohorts based on self-reported information (i.e., 

grade level and/or graduation year) provided at the 

time of registration (in the case of SAT) or test 

administration (in the case of AP and PSAT). The 

College Board matches students’ data across 

programs to identify the most recent valid value 

when assigning students to cohorts.

1.3.D Percentage of adult 

education participants in 

adult basic education 

programs who achieve a 

measurable skill gain

NRS for Adult 

Education 

Office of Career, Technical and Adult Education 

(OCTAE) provides guidance and technical 

assistance to states on data validity, reliability, and 

completeness. States are required to certify data 

quality annually. Sufficient edits checks are a 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 100

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

1.3.E Percentage of adult 

education participants who 

obtain a secondary school 

diploma or its equivalent 

and are employed or 

enrolled in an education or 

training program within one 

year following exit 

requirement that must be addressed as part of the 

process for states certifying data quality. States 

with an overall data quality level that falls below 

“superior” must submit a data quality improvement 

plan. OCTAE conducts ongoing desk and on-site 

monitoring to verify data quality. The director of the 

Division of Adult Education and Literacy in OCTAE 

is responsible for ensuring that the data are 

accurate.

1.3.F Percentage of secondary 

career and technical 

education (CTE) 

concentrators who attain a 

secondary school diploma, 

a General Education 

Development credential or 

other state-recognized 

equivalent (including 

recognized alternative 

standards for individuals 

with disabilities), or a 

proficiency credential, 

certificate, or degree, in 

conjunction with a 

secondary school diploma

State 

Consolidated 

Annual Reports 

(CARs) for the 

Carl D. Perkins 

Career and 

Technical 

Education Act 

OCTAE performs a 10-step check for validity, 

reliability, and completeness of data. Under the 

current Carl D. Perkins Career and Technical 

Education Act, states have the authority to 

determine definitions of CTE concentrator and the 

definitions of their measures. Director of the 

Division of Academic and Technical Education 

(DATE) in OCTAE is responsible for certifying that 

the data are accurate.

1.4.A Number of technical 

assistance engagements, 

events or related activities 

or products focused on the 

grantees' use of evidence 

in kindergarten through 

grade 12 education

Department of 

Education offices 

that deliver 

technical 

assistance

The Department has established criteria to 

accommodate the range of internal and external 

technical assistance provided across offices. 

Establishing a common definition of key terms and 

program offices applying such definitions 

consistently and in a standard way will be a 

challenge. PPSS will follow up with program offices 

as necessary to address any anomalous data. This 

metric provides descriptive information and no 

causal relationship could be attributed to the data 

and the performance of grantees. Contacts will be 

identified in individual program offices to be 

responsible for certifying accuracy of the data.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 101

Strategic Goal 2: Expand postsecondary education opportunities, improve outcomes to 

foster economic opportunity and promote an informed, thoughtful and productive 

citizenry.

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

2.1.A Percentage of OPE grantees 

with large available balances

The 

Department’s 

Grants 

Management 

System G5

The universe for this metric includes Higher 

Education Program (HEP) grants and select 

International and Foreign Language Education 

(IFLE). G5 data are historically accurate. Grantees 

tell program staff when they think there is a 

discrepancy in the balances. This is an independent 

source of validation. OPE managers will spot check 

data and address anomalies with program staff. 

Given that program cycles are in different stages 

from year to year and each program is unique, the 

data for this metric may not be comparable each 

quarter or from year to year. The Deputy Assistant 

Secretary of HEP and the Assistant Secretary of 

OPE are responsible for certifying that the data are 

accurate.

2.1.B Percentage of OPE grantees 

with large available balances 

that received technical 

assistance resulting in 

“Resolved with Good 

Explanation”

2.1.C Percentage of annual 

statutory requirements for 

OPE programs that are 

fulfilled by OPE

G5 will be updated to list statutory requirements for 

each program. Managers will certify fulfillment. 

Directors will spot check data and address 

anomalies with managers. Office of the General 

Counsel (OGC) will be consulted if there is a 

question about whether a requirement has truly been 

fulfilled. The Deputy Assistant Secretary of HEP and 

the Assistant Secretary of OPE are responsible for 

certifying that the data are accurate. 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 102

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

2.1.D Percentage of First-Time 

FAFSA Filers Among High 

School Seniors

FSA’s Central 

Processing 

System

The data we use originate from FSA operational 

systems. These systems, as a part of their 

operations, have procedures in place to address 

potential data quality issues. These metrics report 

figures about data that our operational systems 

maintain, and we do not manipulate it in any way. If 

anomalies persist, they exist in the operational 

systems.

We have a consistent and disciplined process in 

place for querying our systems for data. The process 

includes having a separate data analyst from a 

different office within FSA validate the accuracy of 

the query and the resulting data. This process also is 

used for anomalous data.

In addition to the above controls, queries and 

calculations are simultaneously conducted on data 

from previous years to ensure technical definitions 

remain consistent and calculations by the business 

intelligence team as part of the validation.

The Customer Analytics Group is responsible for the 

primary calculation of the metric using the data from 

FSA. Business Intelligence is responsible for the 

technical validation of the metric, which is done by 

reviewing for accuracy the query used to pull the 

data and all calculations made with the data. Finally, 

the Financial Reporting and Analysis Branch is 

responsible for ensuring that documentation is 

complete and archived.

Since year of high school graduation is not asked on 

the FAFSA, several assumptions are made and 

specific criteria were used (such as age of applicant) 

to identify those likely to be high school seniors. For 

the 2018-19 FAFSA, which was available beginning 

October 1, 2017, the age assumption for high school 

seniors was increased from 18 to 19 years of age.

Using the age criteria of 18 years of age (used in 

previous years) resulted in FSA undercounting high 

school seniors by approximately 23%. Relaxing the 

age assumption to include FAFSAs from slightly 

older high school seniors produces more accurate 

data that also better reflect nationwide awareness 

and outreach efforts.

These calculations also restrict the application period 

to the first nine months of the application cycle (the 

close of the fiscal year), rather than the entire 18 

months. Since most applicants, including high school 

seniors, file their FAFSA prior to the start of the 

upcoming academic year (usually before fiscal year 

end), this decision better aligns the performance 

metric with the fiscal year where most of the 

performance occurred. The alternative is waiting for 

the close of the 18-month cycle, where a 

performance metric would mostly reflect 

performance from an earlier fiscal year.

2.1.E Persistence Among FirstTime Filing Aid RecipientsFSA’s Common 

Origination and 

Distribution 

system

Data used for these calculations are based on 

counts from operational systems (number of 

recipients and number of applicants). Moreover, 

standardized queries are used to rerun and match 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 103

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

calculations for earlier cycles. Therefore, the metric 

as defined should be considered verified and valid.

2.2.A Number of technical 

assistance events or 

activities and products 

focused on the use of 

evidence in federal 

programs that promote 

educational opportunities, 

training, and support 

services for the workforce.

Department of 

Education 

offices that 

deliver technical 

assistance

Criteria for and examples of technical assistance 

events/activities and products are provided above.

The examples provided are not exhaustive, so 

program offices may be able to “count” additional 

events/activities and products that meet the other 

criteria. The criteria are flexible to accommodate 

different technical assistance offerings provided 

across the Department. PPSS will follow up with 

program offices as necessary to address any 

anomalous data. Establishing a common definition of 

key terms and program offices applying such 

definitions consistently and in a standard way will be 

a challenge. ED could conduct a few “level setting” 

meetings with staff charged with reporting from each 

program office to try to support collection of standard 

and meaningful data. In addition, this will be 

descriptive information and no causal relationship 

could be attributed to the data and the performance 

of grantees. Contacts will be identified in Individual 

program offices to be responsible for certifying 

accuracy of the data.

2.2.B Percentage of adult 

education program 

participants who are in 

unsubsidized employment 

during the second quarter 

after exit from the program

NRS for Adult 

Education

OCTAE provides guidance and technical assistance 

to states on data validity, reliability, and 

completeness. States are required to certify data 

quality annually. Sufficient edits checks are a 

requirement that must be addressed as part of the 

process for states certifying data quality. States with 

an overall data quality level that falls below 

“superior” must submit a data quality improvement 

plan. OCTAE conducts ongoing desk and on-site 

monitoring to verify data quality. The director of the 

DAEL in OCTAE is responsible for ensuring that the 

data are accurate.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 104

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

2.2.C Percentage of VR 

participants who are in 

unsubsidized employment 

during the second quarter 

after exit from the program

RSA 911 

Vocational 

Rehabilitation 

Case Service 

Report

The numerator for this metric is the participants who 

were employed in the 2nd quarter after program exit 

(RSA-911 Element 383).

The denominator for this metric is the total number of 

participants who exited during the reporting period 

(RSA-911 Element 355).

RSA will maintain a comprehensive edit check table 

on its website. The table will detail, by data element, 

the edit checks required to ensure the integrity of 

data submissions. Edits describe constraints that 

should be satisfied by the data. Each data 

submission will be analyzed to determine whether 

the data are consistent with the edit checks. Data 

submissions that fail to pass the edit check will be 

returned to the VR agency for correction and 

resubmission. Any corrections made must be 

consistent with the agency’s electronic case 

management system and the supporting 

documentation maintained by the agency. RSA’s 

data editing process will utilize both hard and soft 

edits. Hard edits identify records that “fail” based 

upon erroneous combinations or missing values. 

Soft edits are constraints that identify (combinations 

of) values that are suspicious but not necessarily 

incorrect. RSA data staff conduct macro-level checks 

to uncover inconsistencies in the data reported by 

state VR agencies (e.g., formatting errors, necessary 

relationships between data elements, and 

completeness of submissions). RSA will request 

agencies investigate and correct any anomalies in 

subsequent report submissions. Data for this metric 

are based on unemployment insurance data from the 

state’s Unemployment Insurance Tax System. There 

is a two-quarter lag in the availability of those data. 

As a result, the employment rate data reported is, in 

effect, reported four quarters after the individual exits 

the program. VR agencies are responsible for 

ensuring that any data submitted conforms to edit 

check and data submission requirements. VR 

Directors, or individuals formally delegated the 

authority to submit the data on behalf of the VR 

Directors, are required to certify the accuracy of the 

data that are submitted.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 105

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

2.2.D Percentage of VR program 

participants who, during a 

program year, are in an 

education or training 

program that leads to a 

recognized postsecondary 

credential or employment 

and who are achieving 

measurable skill gains, 

defined as documented 

academic, technical, 

occupational, or other forms 

of progress, towards such a 

credential or employment

RSA 911 

Vocational 

Rehabilitation 

Case Service 

Report 

The numerator for this metric is the unique count of 

the most recent date on which participants achieved 

a Measurable Skill Gains in the reporting period, via 

one of the following:

•Educational Functioning Level (RSA-911 

Element 343) 

•Postsecondary Transcript/Report Card (RSA911 Element 345) 

•Secondary Transcript/Report Card (RSA-911 

Element 344)

•Training Milestone (RSA-911 Element 346) 

•Skills Progression (RSA-911 Element 347)

The denominator for this metric is all participants 

enrolled in an education or training program leading 

to a recognized postsecondary credential or 

employment (RSA-911 Element 85). 

RSA will maintain a comprehensive edit check table 

on its website. The table will detail, by data element, 

the edit checks required to ensure the integrity of 

data submissions. Edits describe constraints that 

should be satisfied by the data. Each data 

submission will be analyzed to determine whether 

the data is consistent with the edit checks. Data 

submissions that fail to pass the edit check will be 

returned to the VR agency for correction and 

resubmission. Any corrections made must be 

consistent with the agency’s electronic case 

management system and the supporting 

documentation maintained by the agency. RSA’s 

data editing process will utilize both hard and soft 

edits. Hard edits identify records that “fail” based 

upon erroneous combinations or missing values. 

Soft edits are constraints that identify (combinations 

of) values that are suspicious but not necessarily 

incorrect. RSA data staff conduct macro-level checks 

to uncover inconsistencies in the data reported by 

state VR agencies (e.g., formatting errors, necessary 

relationships between data elements, completeness 

of submissions). RSA will request agencies 

investigate and correct any anomalies in subsequent 

report submissions. VR agencies are responsible for 

ensuring that any data submitted conforms to edit 

check and data submission requirements. VR 

Directors, or individuals formally delegated the 

authority to submit the data on behalf of the VR 

Directors, are required to certify the accuracy of the 

data that are submitted.

2.3.A Number of technical 

assistance activities, 

sponsored by the 

Department, intended to 

expand or enhance the 

integration of workforce 

preparation activities within 

academic instruction in adult 

education classrooms 

Quarterly 

Progress 

Reports from 

Contractor

The contractor progress reports include, as 

applicable, links to virtual events and meeting 

materials from face-to-face events. OCTAE staff vet 

all event materials prior to the actual event. For 

virtual events, contractor provides relevant analytics 

as supporting documentation. For face-to-face 

events, contractor collects sign-in sheets and event 

evaluation forms. OCTAE staff review contractor 

documentation for anomalous or unclear 

submissions and follow up with the appropriate 

contractor. The Deputy Director of DAEL in OCTAE 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 106

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

is responsible for certifying that the data are 

accurate.

2.3.B Percentage of adult 

education participants 

enrolled in an integrated 

education and training (IET) 

program

NRS for Adult 

Education

OCTAE provides guidance and technical assistance 

to states on data validity, reliability, and 

completeness. States are required to certify data 

quality annually. Sufficient edits checks are a 

requirement that must be addressed as part of the 

process for states certifying data quality. States with 

an overall data quality level that falls below 

“superior” must submit a data quality improvement 

plan. OCTAE conducts ongoing desk and on-site 

monitoring to verify data quality. The director of the 

DAEL in OCTAE is responsible for ensuring that the 

data are accurate.

2.3.C Percentage of 

postsecondary CTE 

concentrators who received 

an industry-recognized 

credential, a certificate, or a 

degree

State 

Consolidated 

Annual Reports 

(CARs) for the 

Carl D. Perkins 

Career and 

Technical 

Education Act

OCTAE performs a 10-step check for validity, 

reliability, and completeness under the current Carl 

D. Perkins Career and Technical Education Act, 

states have the authority to determine the definitions 

of CTE concentrator and the definitions of the 

measures and so performance data may not be 

comparable across states. DATE in OCTAE, is 

responsible for certifying that the data are accurate.

2.4.A Number of non-default 

federal student loan 

servicers’ call centers with 

expanded hours

Federal 

servicers 

quarterly reports

Data will be collected from federal servicers through 

non-default federal student loan servicers’ quarterly 

reports. For this metric the query will be “how many 

call centers have expanded hours meeting or 

exceeding the defined times?” The verification and 

validation of performance by the non-default federal 

student loan servicers’ will be conducted by Federal

Student Aid (FSA) including, but not limited to, (a) 

review and validation of federal servicer reports, (b) 

ongoing/recurring quality assurance discussion with 

federal servicers, (c) site visits to federal servicer call 

center sites, and (c) documented on-phone 

(“mystery caller”) evaluations of services. Because 

the agency directive is succinct and builds upon 

current contractor operational capabilities, FSA does 

not anticipate anomalous data or issues with 

implementation. However, in cases where 

verification and validation detect anomalies that 

suggest less than complete information, FSA will 

address any deficiencies through direct contact with 

federal servicers, requests for information, audits, 

site visits, and/or other assessment measures of 

performance, as applicable. FSA is not presently 

aware of any limitations or constraints of the data. 

Business Operations Officer, FSA Business 

Operations, which oversees all of FSA’s non-default 

federal student loan servicers, is responsible for 

certifying that the data are accurate.

2.4.B Number of call centers that 

meet or exceed the quality 

standard for average speed 

to answer (ASA)

Data will be collected from non-default federal 

student loan servicers through servicers’ quarterly 

reports. For this metric, the query will be “how many 

servicers met the standards for quality?” The 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 107

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

2.4.C Number of call centers that 

meet or exceed the quality 

standard for average 

abandon rate (AR) for 

incoming calls

verification and validation of performance by the 

non-default federal student loan servicers will be 

conducted by FSA including, but not limited to, (a) 

review and validation of federal servicer reports, (b) 

ongoing/recurring quality assurance discussion with 

federal servicers, (c) site visits to federal servicer call 

center sites, and (c) documented on-phone 

(“mystery caller”) evaluations of services. Because 

the agency directive is succinct and builds upon 

current contractor operational capabilities, FSA does 

not anticipate anomalous data or issues with 

implementation. However, in cases where 

verification and validation detect anomalies that 

suggest less than complete information, FSA will 

address any deficiencies through direct contact with 

federal servicers, requests for information, audits, 

site visits, and/or other assessment measures of 

performance, as applicable. FSA is not presently 

aware of any limitations or constraints of the data. 

Business Operations Officer, FSA Business 

Operations, which oversees all of FSA’s non-default 

federal student loan servicers, is responsible for 

certifying that the data are accurate.

2.4.D ACSI Aid Life Cycle Surveys FSA’s Customer 

Satisfaction 

Surveys 

Traditionally, the ACSI Aid Life Cycle metric has 

been based on the following proportional weights:

.65 – Borrower (in Repayment) Satisfaction – based 

on a survey of ED held Serviced Borrowers

.29 – Applicant Satisfaction – based on the FAFSA 

on the Web Survey of Applicants who complete the 

FAFSA

.06 – Borrower (in School) Satisfaction – based on 

our survey of borrowers still enrolled.

For 2017, our Borrower in School Survey was not 

funded while applicants were still in school, so the 

survey could not be conducted. As a result, last 

year’s score was based on the following proportional 

weights:

.76 – Borrower (in Repayment) Satisfaction – based 

on a survey of ED-held Serviced Borrowers

.24 – Applicant Satisfaction – based on the FAFSA 

on the Web Survey.

FSA plans to fund the 2018 Borrower in School 

survey this winter/spring in order to be able to 

conduct the Borrowers in School Survey and use 

that information as part of this metric for 2018. The 

ACSI Survey results have declined in the past few 

years, partly attributed to unexpected interruptions 

with the IRS Data Retrieval Tool during FAFSA 

completion. While the Borrower in School Survey is 

an important part of the overall Customer 

Satisfaction Survey, its inclusion is not a significant 

contributor to the overall score. Therefore, FSA 

expects the FY2018 ACSI Aid Life Cycle Survey 

score to be largely unchanged for the coming year.

As FSA approaches changes to the FAFSA, we do 

expect sharp increases in customer satisfaction in 

future years.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 108

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

2.5.A Percentage of borrowers 

that are more than 30 days 

delinquent

FSA’s Data 

Warehouse 

Data will be collected on a quarterly basis from 

FSA’s Data Warehouse. The verification and 

validation of the rate(s) of delinquency will be 

conducted by FSA’s Office of Performance 

Management. Because the query has been 

performed before, successfully, FSA does not 

anticipate anomalous data or issues with 

interpretation. However, in cases where verification 

and validation detect anomalies that suggest less 

than complete information, FSA will address any 

deficiencies through collaboration with subject matter 

experts within the Office of Performance 

Management. No limitations, other than macroeconomic situations (i.e., economic downturn), have 

been identified. Deputy Chief Operating Officer, FSA 

is responsible for certifying that the data are 

accurate.

2.5.B Percentage of borrowers 

that are more than 90 days 

delinquent
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 109

Strategic Goal 3: Strengthen the quality, accessibility, and use of education data through 

better management, increased privacy protections and transparency.

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

3.1.A Number of data 

management activities for 

which Department-wide 

procedures or templates 

have been created and 

reviewed through the Data 

Strategy Team (DST)

Minutes from 

Monthly DST 

meetings

At the time minutes from the Monthly Meeting are 

submitted by the DST Support contractor to the DST 

Co-Chairs, the information presented in support of 

this metric will be reviewed and accepted by the DST 

Co-Chairs. These meetings occur every two-weeks 

providing both the DST Support contractor and the 

DST Co-Chairs with an opportunity to review and 

finalize documentation of the Meeting in a timely 

fashion, and before a subsequent DST Monthly 

Meeting is held. Any data identified by the DST CoChairs as anomalous would be discussed at one of 

the bi-weekly meetings between the Co-Chairs and 

the DST Support contractor. Anomalous data would 

be researched using the original meeting notes from 

the monthly meeting in question and followed up with 

calls from the Co-Chairs to any points of contact 

whose approval or whose attendance at the Monthly 

Meeting was in question. The DST Co-Chairs are 

responsible for certifying that the data are accurate.

3.1.B Number of the Department’s

program offices participating 

in DST offered data 

management trainings

3.2.A Number of Institutions of 

Higher Education (IHE) that 

have an audit of GLBArelated information security 

safeguards which result in 

no significant findings

IHE-provided 

auditor reports

The data will be verified and validated by the FSA 

Senior Advisor for Cybersecurity in collaboration with 

the Department’ Chief Privacy Officer, on at least a 

quarterly basis, and compared to report data, FSA 

records, and ongoing compliance and investigations 

regarding data breaches. Anomalous data resulting 

from GLBA-related audits will be brought to the 

attention of the auditing entities and reconciled 

where possible. There are no identified nuances or 

limitations to the data. FSA’s Deputy Chief Operating 

Officer will certify that all data are accurate.

3.2.B Number of outreach 

activities targeting data 

privacy and IT security 

requirements of institutions 

of higher education

Outreach activity 

records 

maintained by 

the Privacy 

Technical 

Assistance 

Center

The data will be verified and validated by the FSA 

Senior Advisor for Cybersecurity in collaboration with 

the Department’s Chief Privacy Officer on at least a 

quarterly basis, and compared to report data, FSA 

records, and ongoing compliance and investigations 

regarding data breaches. Anomalous data related to 

outreach activities will be noted and staff and 

contractors will be questioned to ensure that 

outreach activities and programs fall within the 

defined, qualifying activities. There are no identified 

nuances or limitations to the data. FSA’s Deputy 

Chief Operating Officer will certify that all data are 

accurate.

3.2.C Percentage of LEA websites 

from statistically 

representative sample 

reviewed for inclusion of 

transparency best practices 

and compliance with legal 

requirements relating to third 

party contracting

Selected LEA 

public websites

The data will be verified and validated by the 

Director of Student Privacy Policy in collaboration 

with the Department’ Chief Privacy Officer on at least 

a quarterly basis. Anomalous data will be noted in 

the comments field, and staff and contractors will be 

questioned to ensure that district website reviews 

are accurate and complete. No limitations are 

anticipated. The Chief Privacy Officer is responsible 

for certifying that the data are accurate.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 110

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

3.3.A Number of sessions 

dedicated to improved data 

use provided to external 

grantees and stakeholders 

presented by Department

employees or their 

contractors, or occurring at 

Departmental hosted events

PPSS records The sessions will be tracked on a dashboard on

PPSS’s SharePoint site. Program offices submitting 

the data will be responsible for verifying its accuracy. 

The dashboard will include a statement that Program 

Offices submitting data must agree to ensure the 

accuracy of their information prior to entering it into 

the dashboard. OPEPD-PPSS would investigate 

further any anomalous data. Anomalous data would 

be researched using the original data submitted and 

follow-up calls to the program office will be made 

when necessary. One limitation of the collection 

could be Program Offices not capturing all qualifying 

sessions in the dashboard. The Department can use 

its multiple data focused teams, such as DST, 

EDFacts Data Governance Board, and InformED, to 

continuously remind Program Offices to enter their 

data. OPEPD-PPSS is responsible for certifying that 

the data are accurate to the best of their knowledge.

3.3.B Number of newly added 

publicly available datasets in 

machine readable formats

The 

Department’s 

Public Data 

Listing

The Office of the Associate Commissioner for the 

Administrative Data Division in NCES will verify that 

the extract matches the inventory at the time of 

extraction. NCES and OPEPD-PPSS will discuss 

and review jointly and data identified as anomalous. 

Anomalous data would be researched using the 

original metadata submitted and the extraction. 

Follow-up calls to the contact person listed for the 

dataset in question will be made if necessary. The 

universe of the data used in this reporting does not 

currently reflect the full universe of data that are or 

could be made publicly available by the Department. 

There may also be access points to machine 

readable data not listed in the Public Data Listing. 

These gaps will continue to close as the Public Data 

Listing becomes more complete. Additionally, there 

may be a time in the future when the Public Data 

Listing is no longer the central access point for ED’s 

machine readable data. If this happens, we may 

need to revise the data source, but otherwise the 

metric will remain the same. OPEPD-PPSS is 

responsible for certifying that the data are accurate 

to the best of their knowledge.

3.3.C Percentage of discretionary 

grant competitions that 

include the requirement to 

openly license to the public 

copyrightable grant 

deliverables created with 

Department grant funds

Information 

Collection 

Packages

OM-ICCD is responsible for ensuring the information 

is accurate, based on information received through 

the Information Collection Packages. If in reviewing 

the packages, OET has any questions about any 

information included in the packages, OET will first 

ask OM-ICCD. If OM-ICCD cannot find the answer 

they will facilitate communications with the 

appropriate Program Office to find the answer. The 

numerator and denominator are clearly defined, as is 

the process for calculating a percentage; therefore,

there are no limitations or nuances that need to be 

documented at this time. The OET is responsible for 

certifying that the data are accurate to the best of 

their knowledge.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 111

Strategic Goal 4: Strategic Goal 4: Reform the effectiveness, efficiency and accountability 

of the Department.

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

4.1.A Number of evaluations to 

identify potential EO 13771 

deregulatory actions that 

included opportunity for 

public input and/or peer 

review

Report to OMB Division of Regulatory Services (DRS) will monitor 

the data and verify numbers. The Assistant General 

Counsel for Regulatory Services in OGC is 

responsible for certifying that the data are accurate.

4.1.B Number of EO 13771 

deregulatory actions 

recommended by the 

Regulatory Reform Task 

Force to the agency head, 

consistent with applicable 

law

Department 

Records

DRS will monitor the data and verify numbers. The 

Assistant General Counsel for Regulatory Services 

in OGC is responsible for certifying that the data 

are accurate. 

4.1.C Number of EO 13771 

deregulatory actions issued 

that address 

recommendations by the 

Regulatory Reform Task 

Force

Federal Register 

and ed.gov

DRS will monitor the data and verify numbers. The 

Assistant General Counsel for Regulatory Services 

in OGC is responsible for certifying that the data 

are accurate. 

4.1.D Number of EO 13771 

regulatory actions and, 

separately, EO 13771 

deregulatory actions issued

Federal Register 

and ed.gov

DRS will monitor the data and verify numbers. The 

Assistant General Counsel for Regulatory Services 

in OGC is responsible for certifying that the data 

are accurate. 

4.1.E Total incremental cost of all 

EO 13771 regulatory actions 

and EO 13771 deregulatory 

actions (including costs or 

cost savings carried over 

from previous fiscal years)

Report to OMB

4.1.F Number of significant 

deregulatory actions 

submitted to OMB, the 

number of actions reflected 

on Reginfo.gov

Reginfo.gov

4.1.G Number of requests for 

significance determination of 

deregulatory actions 

submitted to OMB, the 

number of such requests 

submitted to OMB by e-mail

E-mails between 

the Department 

and OMB

4.2.A Percentage of key indicators 

that show improvement in 

performance and risk 

management 

Office of the 

Chief Financial 

Officer will 

calculate using 

data from 

various sources

Each office that is responsible for a risk indicator 

within this composite metric maintains data 

verification and validation procedures for that 

indicator. Office of the Chief Financial Officer 

(OCFO) requires Sections I and III of this 

questionnaire be completed for all indicators and 

maintains copies as supporting documentation to 

help ensure data integrity. Judgement is required to 

define the list of risk indicators that comprise this 

composite metric, since it does not include all 

Strategic Plan metrics. OCFO leverages the Senior 

Management Council to decide on the final list of 

indicators that comprise this metric. OCFO is 

responsible for certifying that the data are accurate. 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 112

Associated 

Strategic 

Objective

Performance Metric 

(Indicator) Data Source(s) Data Verification and Validation Overview

4.3.A Maturity level score of the 

Department’s Information 

Technology (IT) Security 

Program and Practices as 

they relate to the Federal 

Information Security 

Modernization Act of 2014

(FISMA)

Annual (FY) 

Inspector 

General FISMA 

Reporting 

Metrics Audit 

Report 

As part of the annual audit process, the OIG 

disseminates draft reports for the Department to 

review and comment on its findings, 

recommendations, and assessments of maturity. 

OCIO leads coordination efforts across Department 

stakeholders and participates in review meetings 

with the OIG to verify and validate the information 

contained in the OIG’s report that provide the basis 

for its determination of the maturity levels. Each 

year the Office of Management and Budget 

publishes new FISMA guidance. OCIO reviews the 

new guidance and identifies areas that may impact 

how the Department’s IT security program will be 

evaluated. OCIO leads coordination and 

communication efforts to ensure stakeholders and 

Department leadership understand the new 

guidance and any steps that need to be taken. The 

CIO is responsible for certifying that the data are 

accurate.

4.4.A FEVS employee 

engagement index score

Office of 

Personnel 

Management 

(OPM) Federal 

Employee 

Viewpoint 

Survey (FEVS)

Data verification and validation processes are 

integrated into the OPM FEVS survey results 

validation process. Any questionable FEVS results 

identified in annual reports would be brought to the 

attention of OPM. The Department would then work 

with the OPM point of contact to obtain clarity or 

resolve. Any questionable or anomalous FEVS data 

are brought to the attention of OPM. In turn, the 

OPM point of contact would research the matter 

and work with the Department to obtain clarity or 

resolve. There are no known data limitations. OPM 

is responsible for certifying that the data are 

accurate. 

4.4.B Percentage of positions with 

competencies identified

Talent 

Management 

System (TMS)

Data are processed and reviewed by LDD staff for 

accuracy. LDD staff will monitor progress of 

assessments. OM, OHR and LDD are responsible 

for certifying that the data are accurate.

4.4.C Percentage of supervisors 

and managers with a 

performance plan critical 

element related to employee 

engagement

TMS and USA 

Performance

To ensure quality control, verification processors 

will self-check and cross check one another. The 

validation and affirmation of final numbers will be 

done by the Principal Operating Component (POC) 

EXO leadership to reconcile system reports. The 

Director of Workforce Relations Division will be 

responsible for certifying the data are accurate.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 113

Appendix D: Programs by Goal

Most of the Department’s programs are funded through discretionary appropriation acts enacted 

each fiscal year. However, there are many education programs—some of them large—that are 

funded directly through their authorizing statutes. For many budgeting purposes, these 

programs are classified as mandatory. For the purposes of this document, resources by goal 

are discretionary funds only. Mandatory programs that contribute to each goal are listed below.

Strategic Goal 1

Support state and local efforts to improve learning outcomes for all P12 students in every community.

Other discretionary Goal 1 programs/activities include the following: 

POC ACCT Goal # Objective # Program

OESE ED 1 1.2, 1.3, 1.4 Title I Grants to local educational agencies 

OESE ED 1 1.2 State agency programs: Migrant 

OESE ED 1 1.2 State agency programs: Neglected and 

delinquent

OESE ED 1 1.2 Special programs for migrant students: High 

School Equivalency Program

OESE IA 1 1.2 Impact Aid, Payments for federally connected 

children: Basic support payments 

OESE IA 1 1.2 Impact Aid, Payments for federally connected 

children: Payments for children with disabilities 

OESE IA 1 1.2 Impact Aid, Facilities maintenance 

OESE IA 1 1.2 Impact Aid, Construction 

OESE SIP 1 1.2, 1.4 State assessments 

OESE SIP 1 1.2 Education for homeless children and youth 

education 

OESE SIP 1 1.2 Training and advisory services 

OESE SIP 1 1.2 Rural education 

OESE SIP 1 1.2 Supplemental education grants 

OESE SSCE 1 1.2 School safety national activities

OESE IE 1 1.2 Indian Education: Grants to local educational 

agencies 

OESE IE 1 1.2 Indian Education: Special programs for Indian 

children 

OESE IE 1 1.2 Indian Education: National activities 

OII I&I 1 1.1, 1.2 Opportunity Grants (proposed legislation)

OII I&I 1 1.3, 1.4 Education innovation and research

OII I&I 1 1.1, 1.2 Charter schools grants 

OII I&I 1 1.1, 1.2 Magnet schools assistance 

OESE/

OLEA

ELA 1 1.2 English language acquisition 

OSERS SE 1 1.1, 1.2, 1.4 Special education grants to States 

OSERS SE 1 1.1, 1.2 Preschool grants 

OSERS SE 1 1.1 Grants for infants and families 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 114

POC ACCT Goal # Objective # Program

OSERS SE 1 1.1, 1.2, 1.4 Technical assistance and dissemination 

OSERS SE 1 1.1 Parent information centers 

OSERS SE 1 1.3 Education technology, media, and materials 

Mandatory programs supporting Goal 1 include: 

No additional programs.

Strategic Goal 2

Expand postsecondary educational opportunities, improve outcomes to 

foster economic opportunity and promote an informed, thoughtful and 

productive citizenry.

Other discretionary Goal 2 programs/activities include the following:

POC ACCT Goal # Objective # Program

OESE ED 2 2.1 Special programs for migrant students: College 

Assistance Migrant Program

OSERS SE 2 2.2 State personnel development 

OSERS SE 2 2.2 Personnel preparation 

OSERS REHAB 2 2.1, 2.2 VR Training 

OSERS REHAB 2 2.1, 2.2 Demonstration and training programs 

OSERS REHAB 2 2.2 Independent living services for older blind 

individuals 

OSERS REHAB 2 2.2 Helen Keller National Center for Deaf-Blind 

Youths and Adults 

OSERS APBH 2 2.1, 2.3 American Printing House for the Blind 

OSERS NTID 2 2.1, 2.2 National Technical Institute for the Deaf

OSERS GU 2 2,1, 2.3 Gallaudet University

OCTAE CTAE 2 2.1, 2.3 Career and technical education State grants 

OCTAE CTAE 2 2.1, 2.4 Career and technical national programs 

OCTAE CTAE 2 2.1 Adult basic and literacy education State grants 

OCTAE CTAE 2 2.1 Adult education national leadership activities 

FSA SFA 2 2,1, 2.3 Federal Pell grants: Discretionary 

FSA SFA 2 2,1, 2.3 Federal work-study 

OPE HE 2 2.1 Aid for institutional development: Strengthening 

tribally controlled colleges and universities 

OPE HE 2 2.1 Strengthening HBCUs 

OPE HE 2 2.1 Strengthening historically Black graduate 

institutions 

OPE HE 2 2.1 Strengthening HBCU masters program 

OPE HE 2 2.2, 2.3 Minority science and engineering improvement 

OPE HE 2 2.1 Model transition programs for students with 

intellectual disabilities into higher education

OCTAE HE 2 2.1 Tribally controlled postsecondary career and 

technical institutions 

OPE HE 2 2.1, 2.2, 2.3 Federal TRIO programs

OPE HE 2 2.1 Child care access means parents in school 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 115

POC ACCT Goal # Objective # Program

OPE HE 2 2.1 Howard University: General support 

OPE HE 2 College Housing and Academic Facilities Loans 

Program Account: Federal administration 

OPE HE 2 Historically Black College and University Capital 

Financing Program Account: Federal 

administration 

OPE HE 2 Historically Black College and University Capital 

Financing Program Account: Loan subsidies

FSA DM/

SAA

2 2.4, 2.5 Student Aid Administration: Salaries and 

expenses 

FSA DM/

SAA

2 2.4, 2.4 Student Aid Administration: Servicing activities

Mandatory programs supporting Goal 2 include: 

POC ACCT Goal # Objective # Program

OSERS REHAB 2 2.1, 2.2, 2.3 Vocational rehabilitation, State grants 

OSERS REHAB 2 2.1 Vocational rehabilitation, Grants to Indians 

FSA SFA 2 2.1, 2.3 Federal Pell grants: Mandatory 

FSA SFA 2 2.1, 2.3 Federal Pell grants: Mandatory funding for 

discretionary program costs 

FSA TEACH 2 2.1, 2.3 TEACH Grants: New loan subsidy

FSA FDSL 2 2.1, 2.3 Federal Direct Student Loans Program Account:

New loan subsidy 

FSA FDSL 2 2.1, 2.3 Federal Direct Student Loans Program Account:

New net loan subsidy (non-add)

FSA FFEL 2 2.1 Federal Family Education Loans Program 

Account: Downward modification of existing 

loans (non-add)

FSA FFEL 2 2.1 Federal Family Education Loans Program 

Account: Net modification of existing loans (nonadd)

FSA FFEL 2 2.1 Federal Family Education Loans Liquidating 

Account: Pre-1992 student loans

FSA HEAL 2 2.1 Health Education Assistance Loans Liquidating 

Account 

OPE HE 2 2.1 Aid for institutional development: Mandatory 

strengthening tribally controlled colleges and 

universities 

OPE HE 2 2.1 Mandatory strengthening Alaska Native and 

Native Hawaiian-serving institutions 

OPE HE 2 2.1 Mandatory strengthening HBCUs 

OPE HE 2 2.1 Mandatory strengthening predominantly Black 

institutions 

OPE HE 2 2.1 Mandatory strengthening Asian American- and 

Native American Pacific Islander-serving 

institutions 

OPE HE 2 2.1 Mandatory strengthening Native Americanserving nontribal institutions 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 116

POC ACCT Goal # Objective # Program

OPE HE 2 2.1, 2.2 Mandatory developing HSI STEM and articulation 

programs 

SFA 2 2.1 Perkins loan repayments 

SFA FDSL 2 2.1, 2.3 FDSL downward modification/negative loan 

subsidies

SFA FFEL 2 2.1 FFEL downward modification/negative loan 

subsidies

SFA 2 2.1, 2.3 Student Financial Assistance debt collection 

Strategic Goal 3

Strengthen the quality, accessibility, and use of education data through 

better management, increased privacy protections and transparency.

Other discretionary Goal 3 programs/activities include the following:

POC ACCT Goal # Objective # Program

IES IES 3 3.1, 3.2, 3.3 Research, development, and dissemination 

IES IES 3 3.1, 3.2, 3.3 Statistics 

IES IES 3 3.1, 3.2, 3.3 National assessment 

IES IES 3 3.1, 3.2, 3.3 National Assessment Governing Board

IES IES 3 3.3 Research in special education 

Mandatory programs supporting Goal 3 include: 

No additional programs.

Strategic Goal 4

Reform the effectiveness, efficiency and accountability of the 

Department.

Other discretionary Goal 4 programs/activities include the following:

POC ACCT Goal # Objective # Program

ALL DM/PA 4 Program Administration: Salaries and expenses

ALL DM/PA 4 Program Administration: Building Modernization 

OCR OCR 4 Office for Civil Rights

OIG OIG 4 Office of Inspector General 

Mandatory programs supporting Goal 4 include: 

No additional programs. 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 117

Appendix E: Summary of Performance Evaluations 

Conducted During FY 2017 and Expected During FY 2018–19

For a complete list of program evaluations and studies from the Office of Planning, Evaluation, 

and Policy Development, please visit

http://www2.ed.gov/about/offices/list/opepd/ppss/reports.html. For a complete list of evaluation 

studies of the National Center for Education Evaluation and Regional Assistance, please visit 

http://ies.ed.gov/ncee/projects/evaluation/index.asp. 

Evaluation Reports From FY 2017–18 

National Center for Education Evaluation and Regional Assistance

Literacy 

An Exploration of Instructional Practices that Foster Language Development and 

Comprehension: Evidence from Prekindergarten through Grade 3 in Title I 

Schools 

Study Purpose: 

The National Assessment of Educational Progress (NAEP) found that one-third of fourth-grade 

students performed below a basic level in reading in 2011. Gaps in reading and language 

development between disadvantaged children and their more advantaged peers begin early, as 

documented by the Early Childhood Longitudinal Study. Closing the gap in language 

development and background knowledge is critical if children are to comprehend text because 

the research literature indicates that these areas of development are linked. Given the modest 

and inconsistent effects of large-scale early literacy interventions, the purpose of the Early 

Childhood Language Development study was to investigate types of instructional practices that 

hold potential promise for promoting children's language development and comprehension. 

Key Question(s) Addressed:

• What classroom practices are associated with children's development of language skills, 

background knowledge, and comprehension in prekindergarten through 3rd grade in Title I 

schools?

Design: 

This exploratory study included a sample of 83 Title I schools with prekindergarten programs to 

identify instructional practices associated with improved language development, background 

knowledge, and comprehension outcomes. In the 2011-12 school year, the study collected data 

for five grade cohorts (prekindergarten, kindergarten, first, second, and third grades) and 

classroom and student samples were selected for each. Data collection included a battery of 

student assessments and classroom observations. This study is exploratory, and its analyses 

estimated the associations between instructional practices and student outcomes to inform 

future rigorous evaluation of strategies to improve language and comprehension outcomes for 

disadvantaged children. 

Actual Completion Date:

The final report was released in August 2017. 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 118

Key Findings: 

The exploratory analysis identified four instructional practices that show promise for improving 

young children’s language development and comprehension. The practices that were most 

consistently related to student growth include:

• Engaging students in defining new words during or after reading a text

• Helping students make connections between their prior knowledge and the texts they read

• Promoting higher-order thinking by asking questions that require students to analyze 

information, explain their thinking, and develop new ideas

• Focusing students’ attention on the meaning of a text before reading it, such as by 

introducing the topic and encouraging predictions

Link to Additional Information: http://ies.ed.gov/ncee/pubs/20174024

Teacher Effectiveness

Study of the Distribution of Effective Teaching

Study Purpose:

There is a persistent achievement gap in the United States where students from high-income 

families outperform those from low-income families on achievement tests. There is also 

substantial variation in the effectiveness of teachers. A key question for policy makers is 

whether policy initiatives focused on providing low-income students with equal access to 

effective teachers can address the achievement gap. This study provided information about the 

extent to which disadvantaged students received less-effective teaching than other students. 

The study also examined teacher mobility in participating districts and how patterns of mobility 

might contribute to unequal access.

Key Question(s) Addressed:

• Are low-income students taught by less effective teachers than high-income students? If so, 

to what extent would providing equal access to effective teachers reduce the student 

achievement gap?

• Are there differences between high- and low-poverty schools in teacher hiring, transfer, and 

attrition? If so, are they consistent with inequitable access to effective teachers for lowincome students?

Design:

The study documented low-income students’ access to effective teachers, as measured by 

value added across the 2008–09 through 2012–13 school years. The study also described 

district polices designed to address inequitable distribution of effective teaching implemented 

during those years. Lastly, the study examined teacher mobility patterns within participating 

districts. Annual data collection included district administrative records, such as student 

achievement to conduct value added analyses, as well as annual semi-structured interviews 

with district leadership to provide information on district policies. District personnel data were 

also collected to examine teacher mobility within participating districts. The study was 

conducted in 29 geographically dispersed school districts.

Actual Completion Date:

The final report on school years 2008–09 through 2012–13 was released in October 2016.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 119

Key Findings from the Final Report:

The final report focused on low-income students’ access to effective teachers and teacher 

mobility patterns in 26 districts and found that:

• There are small differences in the effectiveness of teachers of high- and low-income 

students, on average. The average teacher of a low-income student is just below the 50th 

percentile of effectiveness based on value-added, while the average teacher of a highincome student is at the 51st percentile. Providing low-income students with equally 

effective teachers would not substantively reduce the achievement gap.

• In a subset of the study districts, there is meaningful inequity in teacher effectiveness in 

math. In 3 of the 26 study districts, providing low-income students with teachers whose 

effectiveness is equal to that of high-income students over a five-year period would reduce 

the math achievement gap by at least a tenth of a standard deviation of student 

achievement, the equivalent of about 4 percentile points.

• Teacher hiring patterns are consistent with small inequities in access to effective teachers. 

High-poverty schools have more newly hired teachers than low-poverty schools, but this 

difference is likely to have a small influence on equity because (1) relatively few teachers 

are new hires (11 percent of teachers in high-poverty schools and 5 percent in low-poverty 

schools), and (2) performance of newly hired teachers improves quickly. On average, newly 

hired teachers become as effective as the average teacher after one year.

• Teacher transfer patterns are also consistent with small inequities in access to effective 

teachers. Teachers who transfer to schools in a higher poverty category are less effective 

(43rd percentile) than the average district teacher. Teachers who transfer to schools in a 

lower poverty category are nearly as effective (48th percentile) as the average district 

teacher. These patterns likely have a small influence on equity since just under 4 percent of 

all teachers transfer across poverty categories each year.

• Teacher attrition patterns do not contribute to inequity. Teachers who leave a district are 

less effective (44th percentile) than the average teacher, and more teachers leave highpoverty schools than low-poverty schools (10 percent versus 7 percent, respectively).

Link to Additional Information: https://ies.ed.gov/ncee/pubs/20174008/

Impact Evaluation of Teacher and Leader Evaluation Systems

Study Purpose:

The study is designed to examine the implementation of a package of performance evaluation 

system components and the impact of their use for formative purposes. These are components 

that states and districts might elect to include in their evaluation systems, with support from Title 

II Part A funds under the Every Student Succeeds Act (ESSA). The components include 

measures of student achievement growth, classroom practices, and principal leadership.

Key Question(s) Addressed:

• What is the impact of the performance evaluation system on teachers’ classroom practices 

and student achievement? What are districts’ and educators’ experiences with 

implementation? 

Design:

Within each of eight districts, approximately 15 elementary and middle schools were randomly 

assigned to receive the study’s measures of student achievement growth, classroom practices, 

and principal leadership during 2012–13 and 2013–14 (treatment group) or to participate only in 

the district’s usual performance evaluation system during the same time period. In treatment 

schools, each year, teachers received four rounds of classroom observations and feedback 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 120

sessions and information about their students’ achievement growth in math and/or reading using 

a statistical model known as value-added. Principals in treatment schools received feedback on 

their leadership practices twice per year. Study data collection included surveys of teachers and 

principals, observations of teachers’ classroom practices, and collection of student records data. 

The study also looks at the extent to which the measures were implemented as intended and 

whether the ratings from the measures reliably distinguish educator performance.

Estimated or Actual Completion Date:

A report on first year implementation findings was released in November 2016. A final report on 

impacts on educator practices and student achievement as well as implementation during the 

study’s second year is expected in fall 2017. 

Key Findings from the First Report: 

• The study’s performance measures were implemented generally as planned. Teachers and 

principals received multiple rounds of ratings and feedback on their practices. However, 

fewer principals and teachers accessed their student growth reports than the study 

intended. 

• Both classroom observation and student growth measures differentiated teacher 

performance, although observation scores were mostly at the upper end of the scale. 

Overall, observation scores varied across teachers, and both value-added scores and 

average classroom observation scores over the year had sufficient reliability to capture 

performance differences among some teachers. 

• The principal leadership measure differentiated performance, but principal self-ratings, 

teachers’ ratings of the principal, and the principal’s supervisor’s ratings of the principal 

often differed.

• Both teachers and principals in treatment schools reported receiving more feedback on their 

performance than did their counterparts. For example, teachers and principals in treatment 

schools reported spending more total time in performance feedback sessions across the 

year than teachers and principals in the control schools. 

Link to Additional Information: https://ies.ed.gov/ncee/pubs/20174004/

Does Content-Focused Teacher Professional Development Work? Findings from 

Three Institute of Education Sciences (IES) Studies

Study Purpose:

Federal and local governments continue to invest billions of dollars each year in professional 

development for teachers. Until recently, there has been little rigorous evidence to inform the 

design and delivery of these professional development programs. Nevertheless, there has been 

growing consensus that deepening teachers’ content knowledge is an essential component of 

effective professional development in both reading and mathematics. Over the past decade, IES 

conducted three large-scale random assignment studies of teacher professional development in 

different grades in reading and math. These studies reveal a common pattern of findings on the 

impact of intensive, content-focused professional development on teaching and learning. The 

findings also highlight unresolved issues that future research might explore to advance our 

understanding and inform professional development policy and practice.

Key Question(s) Addressed:

• This brief synthesizes findings from three large-scale random assignment studies of 

professional development that were conducted by the National Center for Education 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 121

Evaluation and Regional Assistance in IES. Each study examined the impact of teacher 

professional development on teacher knowledge, practice, and student achievement.

Design:

Although the professional development programs in each study were different, they all 

emphasized building teachers’ content knowledge or knowledge about content-specific 

pedagogy. The programs combined summer institutes with periodic teacher meetings and 

coaching during the school year. These programs were compared to the substantially less 

intensive professional development that teachers typically received in study districts. The three 

studies included 270 second-grade reading teachers, 165 fourth-grade math teachers, and 195 

seventh-grade math teachers. 

Actual Completion Date:

The brief was released in November 2016.

Key Findings: 

• The professional development improved teachers’ knowledge and some aspects of their 

practice;

• Improving teachers’ knowledge and practice did not lead to positive impacts on student 

achievement;

• Most of the measured aspects of teachers’ knowledge and practice were not correlated with 

student achievement; and

• The consistent pattern of findings suggests that future studies might seek to better 

understand on what aspects of teacher knowledge and practice professional development

should focus, and how professional development can achieve a larger impact on knowledge 

and practice that also impacts student achievement.

Link to Additional Information: http://ies.ed.gov/ncee/pubs/20174010/

School Choice 

Evaluation of the DC Opportunity Scholarship Program (OSP): Impacts One Year 

After Application

Study Purpose:

The April 2011 Scholarships and Opportunities for Results Act (SOAR Act) provided for a fiveyear continuation of a school choice program for low-income residents of Washington, DC. The 

program, still titled the Opportunity Scholarship Program (OSP), now provides annual 

scholarships of $8,000 (for grades K–8) or $12,000 (for grades 9–12), plus a cumulative inflation 

adjustment, to enable low-income students to attend private schools in DC in lieu of the public 

schools already available to them. The new law also mandated another independent, rigorous 

evaluation of the effectiveness of the program.

Key Question(s) Addressed:

• What is the impact of the OSP on student academic achievement and other measures of 

student success, overall and for subgroups of students identified in the statute as high 

priority?

• What effect does the program have on student and parent perceptions of school safety and 

satisfaction, and on parents’ involvement in education of their children?

• Does the program change students’ instructional environments and opportunities?
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 122

Design:

The evaluation will primarily compare outcomes of approximately 1,800 student applicants 

randomly assigned by lottery to either receive a scholarship or not receive a scholarship. 

Lotteries of program applicants were conducted in spring 2012 (cohort 1), spring 2013 (cohort 

2), and spring 2014 (cohort 3). Data were collected for three follow-up years for each of the 

cohorts and, for students in both the scholarship and nonscholarship groups. The contractor 

administered academic assessments, and conducted student, parent, and principal surveys 

each spring (spring 2013–17). 

Estimated or Actual Completion Date:

The first impact report was released in spring 2017. The second and final impact reports are 

planned for early spring 2018 and late fall 2018.

Key Findings from the First Impact Report: 

• The OSP had a statistically significant negative impact on mathematics achievement after 

one year. Math scores were lower for students who were offered or used OSP scholarships, 

compared to students that applied for but were not offered scholarships. There were no 

statistically significant impacts on reading scores for students in the evaluation overall.

• The program did not have a statistically significant impact on parents' or students' general 

satisfaction with the school the child attended in that first year. The percentage of parents 

giving their child's school a grade of A or B was not statistically different when comparing 

parents of students who were offered or used OSP scholarships with the parents of students 

not selected for the scholarship offer. There were also no statistically significant differences 

when looking at student satisfaction with schools.

• The program had a statistically significant positive impact on parents' perceptions of safety 

at the school their child attended in that first year. Parents of students who were offered or 

used OSP scholarships were more likely to indicate that their child's school was "very safe," 

compared with the parents of students not selected for the scholarship offer. Student 

perceptions of school safety were not significantly different between the groups.

• The OSP did not have a statistically significant impact on parent involvement in education 

overall. However, for parents of students in grades 6–12, the program had statistically 

significant positive impacts on involvement in education-related activities and events at 

home after one year.

Link to Additional Information: https://ies.ed.gov/ncee/pubs/20174022/

Students with Disabilities

National Longitudinal Transition Study (NLTS) 2012

Study Purpose:

Supporting youth’s successful transition to adult life is a priority under the Individuals with 

Disabilities Education Act (IDEA). IDEA provides federal funds to states and through them to 

school districts to assist in providing special education and related services. States and school 

districts must make a free appropriate public education (FAPE) available to all eligible children 

with disabilities. FAPE includes the provision of special education and related services provided 

in conformity with an individualized education program. IDEA requires that the first IEP to be in 

effect when a child turns 16 include postsecondary goals and transition services needed to help 

youth with disabilities complete high school prepared to achieve appropriate post-school 

outcomes, such as postsecondary education, jobs, and independent living. The NLTS 2012 is 

the third in a series examining the characteristics, school experiences, and post-high school 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 123

outcomes of a nationally representative sample of youth with disabilities. NLTS 2012 focuses on 

youth ages 13 to 21 (in December 2011), but also includes a small sample of students without 

disabilities to enable direct comparisons of students with and without an IEP. This study is part 

of the congressionally mandated National Assessment of the IDEA and is supported with funds 

authorized under Section 664 of IDEA.

Key Question(s) Addressed:

• How do the personal, family, and school characteristics and experiences of youth with 

disabilities differ from those of youth not served under IDEA? (Volume 1; March 2017)

• How do the characteristics and experiences of youth vary across disability groups? (Volume 

2; March 2017) 

• How have the characteristics and experiences of youth with disabilities changed over time? 

• To what extent do youth with disabilities make progress through high school compared to 

other youth?

• Are youth with disabilities achieving the post-high school outcomes envisioned by IDEA, and 

how do their college, training, and employment rates compare to those of other youth? 

• How do these high school and postsecondary outcomes vary with student characteristics?

Design:

This descriptive study includes a nationally representative sample drawn from 432 school 

districts and special schools randomly sampled in 2011 and students randomly sampled within 

those districts. Survey data were collected in 2012-2013 from approximately 12,000 in-school 

youth and their parents, of which about 10,000 are students with IEPs representing each of the 

federal disability categories. The surveys asked about the background characteristics of 

secondary school youth and the schools they attend, their health, functional abilities, and 

engagement in school, the academic supports they receive, and their expectations for and steps 

to achieve transitions beyond high school. High school records and postsecondary enrollment 

information from the National Student Clearinghouse are being collected (from 2017-2019) to 

follow the sample of students through high school and beyond. This administrative data will be 

linked with the 2012-2013 survey data to examine high school course-taking and completion, 

and youth’s experiences with college, training, and employment.

Actual Completion Date:

The first two volumes of the study report describing survey results were released in March 2017.

Key Findings (from the March 2017 volumes):

• Youth with an IEP are more likely than their peers to be socioeconomically disadvantaged. 

Youth with an IEP are 12 percentage points more likely to live in low-income households 

and are less likely to have parents who are employed or have a college education. Among 

disability groups, youth with intellectual disability and youth with emotional disturbance are 

more socioeconomically disadvantaged and more likely to attend a lower-performing school 

than youth with an IEP overall.

• The vast majority of youth with and without an IEP feel positive about school, but those with 

an IEP experience bullying and are suspended at higher rates. Like their peers, more than 

80 percent of youth in special education report that they are happy with school and with 

school staff. However, not only do youth with an IEP more commonly experience some 

types of bullying (e.g., being teased or called names) but, according to parent reports, they 

are more than twice as likely to be suspended or expelled from school.

• Youth with an IEP are more likely than other youth to struggle academically, yet less likely to 

receive some forms of school-based support. Half of all youth with an IEP report they have 

trouble with their classes, about 15 percentage points more than reported by their peers. 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 124

However, they are less likely to report receiving school-based academic help before or after 

regular hours, although their parents more commonly help with homework and attend a 

parent-teacher conference. 

• Youth with an IEP lag their peers in planning and taking steps to obtain postsecondary 

education and jobs. Substantially fewer youth with an IEP expect to enroll in postsecondary 

education or training, compared to youth without an IEP. Reflecting these gaps, youth in 

special education are almost half as likely as their peers to report taking college entrance 

and placement tests. Forty percent report having recent paid work experience while in high 

school, compared with 50 percent of youth without an IEP. 

• Youth with autism, deaf-blindness, intellectual disability, multiple disabilities, and orthopedic 

impairments are most at-risk for not transitioning successfully beyond high school. Youth in 

these groups are less likely than all youth with an IEP to have key characteristics and 

experiences linked to success after high school, such as performing typical daily living tasks, 

engaging with friends and in school activities, or preparing for college, careers, and 

independent living.

Link to Additional Information: https://ies.ed.gov/ncee/pubs/20174016/

Pathways to Career or College

Upward Bound at 50: Reporting on Implementation Practices Today

Study Purpose:

First established in 1965, Upward Bound is the oldest and largest of the federal college access 

programs targeted to low-income students and those who would represent the first-generation 

of college completers in their families. When the evaluation began in 2012, the program served 

more than 60,000 high school students at a cost of about $4,300 per youth. Upward Bound 

projects provide a wide array of academic and college transition support services, with seven 

types of services required by statute (“core” services). While much about the structure of 

Upward Bound and the services to be offered are prescribed in legislation, little is currently 

known about the intensity, duration, and mix of services provided by projects or about how they 

are delivered. Because of the importance of its mission, and the comprehensiveness and costs 

of its services, Upward Bound has long been of interest to policymakers. This report describes 

the approaches to providing program services, as reported by Upward Bound project directors, 

to help identify common (or uncommon but promising) practices that could inform program 

improvement studies that Congress requires the U.S. Department of Education to conduct.

Key Question(s) Addressed: 

• Within the core service areas of the program, where do projects focus their efforts?

• How are services delivered to Upward Bound participants?

• In what ways does the focus or delivery of services vary across different Upward Bound 

projects?

Design:

This descriptive report relies primarily on findings from a summer 2013 survey of all regular 

Upward Bound project directors. 

Actual Completion Date:

The report was released in November 2016.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 125

Key Findings: 

• In four of the seven core service areas—coursework, tutoring, college exposure, and college 

application assistance—there was a dominant approach (used by at least 50 percent of 

projects) to how projects focused their activities. 

• When, where, and how services were delivered differed across service areas. For example, 

tutoring and college entrance exam preparation services were most commonly delivered in 

groups, while academic advising, college application assistance, and financial aid 

assistance were typically provided one-on-one.

• Variation in the focus and delivery of services appears related to the urbanicity and type of 

institution (four-year, two-year, and non-higher education) that hosts the project but not to 

other project characteristics examined (e.g., project size, per-student funding, and whether 

the host institution was a Minority-Serving Institution).

Link to Additional Information: http://ies.ed.gov/ncee/pubs/20174005/

Other

Race to the Top: Implementation and Relationship to Student Outcomes

Study Purpose:

Beginning in 2010, Race to the Top (RTT) was a competitive grant program of the Department 

that funded general state competitions, state competitions focused on early learning, and district 

competitions focused on personalized learning. With funds from the Recovery Act, the general 

state competition awarded approximately $4 billion to states in support of comprehensive K–12 

education reform in several core areas, including teachers and leaders, standards and 

assessments, data systems, and school turnaround. This study examined the implementation of 

RTT and its relationship to student outcomes, focusing on the initial general state competition 

for RTT that began in 2010.

Key Question(s) Addressed:

• Which policies and practices promoted by the RTT program did RTT states report using, and 

how do they compare to the policies and practices that non-RTT states reported using? 

• Was receipt of an RTT grant related to improvement in student outcomes?

Design:

The RTT sample included all 50 states and DC. Data from interviews with all states and DC 

informed the first evaluation question. The second evaluation question was examined by 

comparing state-level trends in NAEP scores before and after the RTT competition, for states 

that were awarded an RTT grant to states that applied for but were not awarded an RTT grant.

Actual Completion Date:

The final report for RTT was released in October 2016.

Key Findings from the Final Report:

• In four of six areas examined, 2010 RTT grantees reported using more policies and 

practices promoted by RTT than states that did not receive a grant: The four areas with 

differences were standards and assessments, teachers and leaders, school turnaround, and 

charter schools. The other two areas were state capacity and data systems.

• 2011 RTT grantees reported using more policies and practices promoted by RTT than 

states that did not receive a grant in one area, which was teachers and leaders.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 126

• Across all states, use of RTT-promoted policies and practices were highest in the data 

systems area and lowest in the teachers and leaders’ area: states reported using 76 percent 

of the 8 RTT-promoted practices examined in data systems, but only 26 percent of the 39 

practices in teachers and leaders.

• The relationship between RTT and student outcomes was not clear: Trends in student 

outcomes could be interpreted as providing evidence that RTT had a positive effect, a 

negative effect, or no effect.

Link to Additional Information: https://ies.ed.gov/ncee/pubs/20174001/

School Improvement Grants (SIG): Implementation and Effectiveness

Study Purpose:

In FY 2009, $3.5 billion ($546 million in regular appropriations and $3 billion from the Recovery 

Act) were allocated to the SIG program authorized by Title I of the ESEA. SIG intended to 

support rapid improvement in the nation’s persistently lowest-achieving schools, and funds were 

distributed to states by formula based on Title I allocations. States then competitively awarded 

funds to districts applying on behalf of their eligible schools. Schools receiving FY 2009 SIG 

funds were required over the three-year grant period to implement one of four prescriptive 

intervention models: turnaround, transformation, closure, or restart. This study examined the 

implementation and impacts of SIG, focusing on the first cohort of SIG schools implementing 

intervention models beginning in the 2010–11 school year.

Key Question(s) Addressed:

• Did SIG-funded schools use the improvement or turnaround strategies promoted by the four 

SIG intervention models, and how did they compare to strategies in schools not 

implementing a SIG-funded intervention model? How did states and districts support such 

efforts? 

• Did receipt of SIG funding to implement a school intervention model have an impact on 

outcomes for low-performing schools? 

• Was implementation of the four school intervention models related to improvement in 

outcomes for low-performing schools?

Design:

The SIG sample included about 500 schools in 60 districts from 22 states. This sample was 

purposely selected to support a regression discontinuity design to address the fourth evaluation 

question, exploiting cutoff rules that states used to identify their persistently lowest-achieving 

schools as eligible for SIG to implement one of the four intervention models. Data from state 

and district interviews, as well as school surveys from the SIG sample, were used to inform the 

third and fifth evaluation questions. Student- and school-level achievement data were also 

collected from administrative records up to the 2012–13 school year to inform the fourth and fifth 

evaluation questions.

Actual Completion Date:

The final report for SIG was released in January 2017.

Key Findings from the Final Report:

• SIG schools implementing one of the four models (transformation, turnaround, restart, or 

closure) reported using more practices than other schools: SIG schools reported using an 

average of 23 out of 35 practices, whereas other schools reported using 20 practices.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 127

• Across all schools, use of SIG-promoted practices was highest in the area of comprehensive 

instructional reform strategies and lowest in the area of operational flexibility and support: 

Schools reported using 89 percent of the eight SIG-promoted practices examined in the 

comprehensive instructional reform strategies area, but only 43 percent of the two practices 

in operational flexibility and support (the other two areas examined were increasing teacher 

and principal effectiveness, and increasing learning time and creating community-oriented 

schools).

• Implementing any of the four SIG models had no significant impacts on math or reading test 

scores, high school graduation, or college enrollment.

• In elementary grades, student achievement gains did not differ across the four SIG models. 

In secondary grades, the turnaround model was associated with larger achievement gains 

than the transformation model.

Link to Additional Information: https://ies.ed.gov/ncee/pubs/20174013/

Implementation of Title I and Title II-A Program Initiatives: Results from 2013-14

Study Purpose:

The Title I and Title II-A programs are a key part of the Elementary and Secondary Education 

Act (ESEA). These programs are intended to help provide all students with equal access to 

education by: providing financial assistance to schools and districts which have a high 

percentage of students from low-income families (Title I) and improving teacher and principal 

quality (Title II-A). Prior to the most recent reauthorization in December 2015, states were 

allowed to apply for waivers to certain No Child Left Behind Act requirements. A majority of 

states sought and received ESEA flexibility beginning in 2012. In exchange for waivers, they 

agreed to implement various reform principles, such as identifying and supporting schools with 

achievement gaps among student subgroups and implementing educator evaluation systems 

based on student achievement and multiple observations.

The first report presents implementation data from states, districts, schools, and teachers prior 

to ESSA, under the No Child Left Behind Act and ESEA flexibility during the 2013–2014 school 

year. A subsequent report will look at implementation after reauthorization during the 2017-2018 

school year using data from states and districts.

Key Question(s) Addressed:

• What content standards and high school graduation requirements are states adopting, and 

what materials and resources are provided to support implementation?

• What types of assessments do states and districts use, and what materials and resources 

are provided to support the implementation of assessments and use of assessment data?

• What elements are included in states' accountability systems? How do states and districts 

identify and reward their highest-performing schools; how do they identify and support their 

lowest-performing schools; and how do they offer differentiated support for those schools 

that are neither highest-performing nor lowest-performing?

• How do states and districts evaluate teacher and principal effectiveness and assess 

equitable distribution of teachers and principals, and what supports are provided to improve 

teacher and principal effectiveness?

• How has student achievement changed over time?

Design:

Data were collected from all 50 states and the District of Columbia, a nationally representative 

sample of districts and schools, and teachers within those schools through surveys in the 2013–
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 128

2014 school year. These data informed the first four research questions. Extant math and 

reading achievement data from state standardized achievement tests and the NAEP were used 

to inform the fifth research question.

Actual Completion Date:

The report was released in January 2017.

Key Findings from the First Report:

• Most states adopted, and most principals and teachers reported implementing state 

standards that focused on college–and career–readiness. All but one state had committed to 

having college- and career-ready standards in place by 2013–14. A majority of principals 

(67–69 percent, depending on subject) reported fully implementing state content standards, 

and most teachers reported receiving professional development relevant to state content 

standards (79 percent of teachers) and weekly use of aligned instructional activities (92 

percent of teachers).

• Many state assessments incorporated more sophisticated response formats to better assess 

students’ college- and career-readiness. In their reading/English language arts summative 

assessments, many states (24–36, depending on grade level) reported using extended 

constructed-response formats, a type of response format intended to assess higher-order 

thinking skills. Nineteen states used this response format in math assessments.

• States used ESEA flexibility to reset their accountability goals and to target a narrower set of 

schools for additional support. Forty-three states had received ESEA Flexibility for the 

2013–14 school year. The most common accountability goal adopted by states with ESEA 

Flexibility (28 of the 43 states) was reducing by half the percentage of students and 

subgroups not proficient in 6–8 years. States with ESEA Flexibility identified 5 percent of 

Title I schools as lowest performing and an additional 10 percent of Title I schools with 

substantial student achievement gaps, compared to non-Flexibility states that reported 

identifying 43 percent of Title I schools as lowest performing.

• Almost all states adopted new laws or regulations related to educator evaluation systems 

between 2009 and 2014, and most districts reported full or partial implementation in 2013–

14. Only four states had not adopted new teacher evaluation laws or regulations by 2014, 

and a majority (59 percent) of districts reported fully implementing, piloting, or partially 

implementing a new teacher evaluation system. However, few districts (18 percent) reported 

using evaluation system measures of student achievement growth and classroom practice 

consistent with emerging research.

• Proficiency rates on the NAEP slightly increased from 2005 to 2015, with the largest 

increases in 4th and 8th grades and smaller or no increases in 12th grade. Overall 

proficiency rates increased by statistically significant levels of 4–5 percentage points in 4th 

and 8th grade reading and math and by 2 percentage points in 12th grade reading. 

Statistically significant increases in proficiency were also evident for economically 

disadvantaged students in both subjects and across all three grades (by 4–7 percentage

points), and in the large majority of individual states (46–51 states, depending on grade and 

subject).

Link to Additional Information: https://ies.ed.gov/ncee/pubs/20174014/

Evaluation of NAEP Achievement Levels

Study Purpose:

Under the provisions of P.L. 107-279, the Secretary is required to provide for continuing review 

of the NAEP assessment. The legislation identifies the issues to be addressed in the reviews, 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 129

one of which includes the requirement to evaluate whether the NAEP achievement levels, 

established by the National Assessment Governing Board, are “reasonable, valid, reliable and 

informative to the public.” Section 303(e)(2)(C) of the Education Sciences Reform Act of 2002 

(P.L. 107-279) states that NAEP achievement levels shall be used on a trial basis until the 

Commissioner of the National Center for Education Statistics (NCES) determines, as a result of 

an evaluation under Sec. 303(f), that such levels are “reasonable, valid, and informative to the 

public.” This independent evaluation was intended to provide IES and the Commissioner of 

NCES with information necessary to inform the decision about whether the current trial status of 

the NAEP achievement levels can be removed or whether they should remain in trial status.

Key Objectives:

• Determine how “reasonable, valid, reliable and informative to the public” will be 

operationalized in this study.

• Identify the kinds of objective data and research findings that will be examined.

• Review and analyze extant information related to the study’s purpose.

• Gather other objective information from relevant experts and stakeholders, without creating 

burden for the public through new, large-scale data collection.

• Organize, summarize, and present the findings from the evaluation in a written report, 

including a summary that is accessible for nontechnical audiences, discussing the 

strengths/weaknesses and gaps in knowledge in relation to the evaluation criteria.

• Provide, prior to release of the study report, for an independent external review of that report 

for comprehensiveness, objectivity, and freedom from bias.

Design:

This study focused on the achievement levels used in reporting NAEP results for the reading 

and mathematics assessments in grades 4, 8, and 12. Specifically, the study reviewed 

developments over the past decade in the ways achievement levels for NAEP are set and used 

and will evaluate whether the resulting achievement levels are “reasonable, valid, reliable, and 

informative to the public.” The study relied on an independent committee of experts with a broad 

range of expertise related to assessment, statistics, social science, and education policy. The 

project is receiving oversight from the Board on Testing and Assessment and the Committee on 

National Statistics of the National Research Council.

Actual Completion Date: 

The report from this study was released final form in April 2017 and may be downloaded as a 

PDF or read online at https://www.nap.edu/catalog/23409/evaluation-of-the-achievement-levelsfor-mathematics-and-reading-on-the-national-assessment-of-educational-progress. 

Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/other_naep.asp

Policy and Program Studies Service

A Study of Practices to Improve Learning in Kindergarten Through Third Grade 

Study Purpose:

This study summarized how (a) aligning preschool through third grade education and (b) 

differentiating instruction for children in kindergarten and first grade may build on the positive 

effects of preschool and help students in kindergarten through third grade (K–3) make cognitive, 

social-emotional, and academic gains. In addition, the study conducted case studies of five 

programs that may build on the positive effects of preschool by using policies, programs, and 

practices for alignment and differentiation. 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 130

Key Question(s) Addressed in the Literature Review:

• What approaches does the research and theoretical literature suggest for aligning preschool 

through third grade (P–3) education, and what is the quality of the research studies? 

• What are the findings from studies of differentiated instruction on children in kindergarten 

and first grade, and what is the quality of these studies?

Key Question(s) Addressed in the Case Studies:

• What approaches did the five programs use to implement P–3 alignment?

• In programs that implemented differentiated instruction, what approaches did staff use?

Design:

The literature review covered two topics: (1) preschool and K–3 alignment, and (2) differentiated 

instruction in kindergarten and first grade. The case studies examined five sites that 

implemented P–3 alignment or differentiating instruction in kindergarten and first grade. The 

research team interviewed principals, teachers, evaluators, and funders to understand 

programs’ characteristics, challenges and solutions, and the sustainability of the programs.

Actual Completion Date:

The literature review was released in August 2016, and the case study report was released in 

December 2016.

Key Findings from the Literature Review:

• Nearly all qualitative studies and policy and theory articles on P–3 alignment suggest 

aligning standards, curriculum, instruction, assessments, and environments across 

preschool and grades K–3.

• P–3 alignment could be supported by establishing similar teacher education and training 

requirements and developing longitudinal data systems that integrate preschool and 

K–12 data.

• Of the 17 quantitative studies of differentiated instruction, one Randomized Control Trial of 

the Individualized Student Instruction With Assessment to Instruction intervention 

demonstrated positive results on reading outcomes and had the potential to meet the criteria 

for strong causal evidence.

• Qualitative studies of differentiated instruction indicate that opportunities for peer 

collaboration and guidance by mentors may be helpful to improve teacher practice related to 

differentiation.

• Key findings from the case studies:

o All five programs aligned instruction across grades by coordinating standards, curricula, 

instructional practices and professional development. 

o Common elements of these programs included the use of professional learning 

communities, coaches, parent engagement, and play-based or student-initiated 

learning. 

o All five programs reported using strategies to accommodate students’ different skill 

levels, including modifying assignments, adapting learning materials, providing different 

levels of support, or using small-group instruction.

Links to Additional Information:

• Literature review: http://www2.ed.gov/rschstat/eval/disadv/p-3-alignment-differentiatedinstruction/report.pdf. 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 131

• Case studies: http://www2.ed.gov/rschstat/eval/implementing-early-strategies/report.pdf. 

Study of Emerging Teacher Evaluation Systems

Study Purpose:

This study will provide descriptive information on the design and early implementation of teacher 

evaluation systems in eight school districts. The findings are intended to help other districts and 

states learn from the experiences of eight districts featured in the study, and apply the design 

and implementation lessons to their own work as it relates to teacher evaluation and support. 

Key Question(s) Addressed:

• What key priorities and measures informed the design of the new teacher evaluation 

systems?

• What steps did the districts take prior to full implementation to test the system and prepare 

teachers and staff to implement it?

• How did the districts structure and conduct the classroom observation component of their 

teacher evaluation systems?

• How did the districts analyze student performance and other data to evaluate teacher 

performance?

• How did the districts use, or plan to use, teacher evaluation results to make personnel 

decisions? To what extent were professional development and career advancement 

decisions tied to evaluation results?

• What administrative structures did districts use to support their new teacher evaluation 

systems?

• What are the perceived early effects of the teacher evaluation systems on the professional 

practices of teachers, principals, and district administrators?

Design:

This descriptive study relied on interviews with key district administrators, principals, teachers, 

and representatives of community stakeholder groups, from eight districts, who were involved in 

the development and early implementation of the respective districts’ teacher evaluation system. 

Given the limited sample, the findings cannot be generalized to other districts.

Actual Completion Date:

The final report was released in November 2016.

Key Findings: 

• Teachers and central office staff generally agreed that the foremost goal of the teacher 

evaluation system was to improve instruction.

• Teacher and principal input during the design and/or pilot test phase strongly influenced 

decisions regarding system modifications in six districts, according to district administrators. 

• Classroom observations varied in frequency, duration, and degree of formality in all eight 

districts. In addition, principals reported challenges in finding time to conduct teacher 

observations.

• Six districts used multiple approaches for measuring teacher impact on student 

performance, including individual and/or school-level value-added models.

• Districts used teacher evaluation results for a range of purposes, including targeted 

professional development and support, career ladders and performance pay, and in some 

instances, redeployment or release of teachers identified as ineffective.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 132

• The majority of districts created relatively simple, streamlined structures to administer their 

teacher evaluation systems.

• Teachers reported that they believed that the classroom observations and feedback helped 

them become better teachers.

Link to Additional Information:

http://www2.ed.gov/about/offices/list/opepd/ppss/reports.html#tq.

State Efforts to Promote Equitable Access to Effective Teachers

Study Purpose:

This report provides a broad overview of state efforts, as of the 2011–12 school year, to monitor 

equitable access to qualified and effective teachers among schools; develop and adopt multiple 

measures of teacher performance to rate teachers among at least three performance levels; 

and implement targeted strategies for promoting equitable access to qualified and effective 

teachers in schools serving high proportions of poor and/or minority students. The report 

examines the use of measures of teacher qualifications and teacher performance in the 

implementation of these strategies.

Key Question(s) Addressed: 

• What measures did states use to monitor equitable access to qualified and effective 

teachers among schools? 

• To what extent were states developing or using multiple measures of teacher performance 

to rate teachers among at least three performance levels? 

• What strategies did states use to promote equitable access to qualified and effective 

teachers in schools serving high proportions of poor and/or minority students?

Design:

This report is based on telephone interviews with officials in SEAs in all states, the District of 

Columbia, and Puerto Rico. Study staff also reviewed extant sources including equity plans, 

Consolidated State Performance Reports, reports from federal monitoring visits, and other 

information on state websites. The interviews were conducted between August 2011 and 

January 2012, and the study reflects state efforts that were underway at that time, and predates 

implementation of the fall 2011 ESEA Flexibility initiative as well as the recent state equity plans 

submitted under the Excellent Educators for All initiative.

Actual Completion Date:

The final report was released in January 2017.

Key Findings: 

• In 2011–12, states most commonly monitored equitable access to qualified and effective 

teachers among schools using measures of teacher qualifications.

• Four states reported using measures of teacher performance—student achievement growth 

and/or measures of teacher practice—to monitor equitable access to qualified and effective 

teachers among schools, and nine states used teacher performance measures to monitor 

the quality of the teacher workforce overall.

• In two of the four states that reported using teacher performance measures to monitor 

equitable access among schools, officials reported seeing larger inequities than were 

previously detected using measures of teacher qualifications alone.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 133

• Six states reported that they had adopted multiple measures of teacher performance and 

were using them to rate teachers among at least three performance levels in 2011–12, and 

38 states indicated that there were in the process of developing such measures.

• Offering monetary incentives was the most common strategy that states reported using in 

disadvantaged schools to promote equitable access to qualified and effective teachers 

among schools (24 states).

• Other state-reported strategies that were directed specifically at disadvantaged schools for 

promoting equitable access were specialized professional development (14 states) and 

teacher recruitment and preparation programs (14 states).

Link to Additional Information:

http://www2.ed.gov/about/offices/list/opepd/ppss/reports.html#tq. 

Feasibility Study on Improving the Quality of School-Level Expenditure Data

Study Purpose:

This study explores the feasibility of improving the collection of school-level expenditure data by 

examining the nature and quality of school-level fiscal data collection in five states and four 

school districts that had developed their own systems for collecting and reporting school-level 

expenditures: Florida, Hawaii, Ohio, Rhode Island, Texas, Baltimore City, Hillsborough County, 

Houston, and Los Angeles.

Key Question(s) Addressed: 

• In states and districts that have developed systems to report expenditures at the school 

level, what types of personnel and nonpersonnel expenditures are included in the schoollevel data? 

• To what extent do the sites track actual expenditures to individual schools versus allocating 

or prorating expenditures to schools using formulas?

• How consistent are school-level expenditure data obtained from these systems with similar 

data from other sources? How do the funding amounts attributed to individual schools based 

on formula allocations compare to those based on tracking actual expenditures?

• What lessons can other states and districts learn from these sites if they wish to implement 

systems for reporting accurate and reliable data on school-level expenditures?

Design:

The study included (1) surveys and interviews of officials to understand the process of collecting 

school-level expenditure data and (2) collection and analysis of school-level spending data to 

examine data quality issues. The study examined three aspects of data quality: the 

comprehensiveness of school-level spending data, consistency with other data sources, and the 

relative accuracy of allocating expenditures to schools by formula (rather than tracking actual 

expenditures for each school).

Actual Completion Date:

The final report was released in January 2017.

Key Findings: 

• Study sites reported that they attributed most categories of spending to the school level, 

including salaries for teachers, administrators, and other support staff as well as 

nonpersonnel items, such as textbooks, instructional materials, furniture and equipment, and 

computers and software.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 134

• The school districts and states in this study attributed an average of three-quarters of 

operational expenditures to individual schools, demonstrating that it is feasible to link a 

significant share of spending to the school level.

• Most of the expenditures that the study sites attributed to schools were directly tracked to 

schools (85 percent) rather than simply being allocated by formula (8 percent).

• Comparisons between the site-reported school-level expenditures and other data sources 

showed a relatively high degree of consistency for salary expenditures, but nonpersonnel 

expenditure data were much less consistent. 

• Allocating expenditures to schools by formula (e.g., based on total salaries or staff) 

appeared relatively accurate for health benefits and less accurate for pension benefits, pupil 

support staff, and instructional support staff.

• Instituting a system for collecting school-level expenditure data typically required new 

hardware and software (eight sites), changes to charts of accounts (six sites), and staff 

training (eight sites).

• Advice that interviewees offered for others aiming to implement school-level expenditure 

data systems was to get stakeholders involved, communicate clearly and frequently, and 

think long-term about future data needs.

Link to Additional Information:

https://www2.ed.gov/about/offices/list/opepd/ppss/reports.html#title. 

Study of Experiences and Needs of Rural Education Achievement Program 

(REAP) Grantees and Subgrantees

Study Purpose: This descriptive study examined how grantees and subgrantees use REAP 

funds provided through the Small, Rural School Achievement (SRSA) and Rural and LowIncome Schools (RLIS) programs–on their own or in combination with other federal funds–as 

well as to explore technical assistance needs related to REAP. 

Key Question(s) Addressed:

• What role do states play in supporting the Department’s SRSA and RLIS eligibility and 

award determination process?

• How do districts use their SRSA or RLIS funds?

• To what extent do SRSA-eligible districts use REAP Flex and for what purposes?

• What recommendations do states and districts have for improving the operation of the 

SRSA and RLIS programs? 

Design:

The study consisted of: 1) a survey of a sample of approximately 1,000 SRSA grantees and 

RLIS subgrantee districts; 2) telephone interviews with a sample of 30 SRSA grantees and RLIS 

subgrantees; and 3) telephone interviews with REAP coordinators in all states receiving REAP 

funds. Data collection began in winter 2015 and was completed by spring 2016. At the state 

level, the study included interviews with all state REAP coordinators about state goals and 

priorities, the planning process for use of RLIS funds, the eligibility process for districts, 

management and distribution of SRSA and RLIS funds, and recommendations for the program. 

At the school district level, the study included an online survey of a nationally representative 

sample of REAP coordinators about the REAP eligibility determination process, administration 

of REAP, and challenges and technical assistance needs related to REAP, as well as telephone 

interviews with a subsample of 30 districts. The interviews included questions about program 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 135

administration, technical assistance needs, and recommendations for changing the REAP 

program to better meet the needs of rural districts. 

Actual Completion Date:

The report was completed in December 2016.

Key Findings:

• States supported the Department in determining REAP eligibility by providing district-level 

data and reviewing the accuracy of Department-provided data.

• All 43 states with RLIS-eligible districts chose to make subgrants to districts on the basis of 

a funding formula rather than on a competitive basis, and 28 of these states based the 

subgrant amount entirely on average daily attendance.

• Districts most frequently used SRSA and RLIS funds to improve or expand access to 

technology (71 percent of SRSA districts and 71 percent of RLIS districts) and to provide 

educator professional development (45 percent of SRSA districts and 58 percent of RLIS 

districts).

• Forty-six percent of SRSA district coordinators reported exercising REAP-Flex; of these, 82 

percent reported that they used funds eligible for REAP-Flex to maintain a stable level of 

funding for ongoing activities.

• The majority of both district and state REAP coordinators were highly satisfied with REAP as 

a whole. However, they provided recommendations for improvement to REAP in three 

categories: (1) improved timelines for eligibility and award determination, (2) more 

information on allowable uses of funds and REAP-Flex, and (3) revised eligibility criteria.

Link to Additional Information: 

https://www2.ed.gov/about/offices/list/opepd/ppss/reports.html#rural. 

Summary of Performance Evaluations Expected During FY 

2018 and FY 2019 

National Center for Education Evaluation and Regional Assistance

Teacher Effectiveness

Impact Evaluation of Teacher and Leader Evaluation Systems

Study Purpose:

The study is designed to examine the implementation of a package of performance evaluation 

system components and the impact of their use for formative purposes. These are components 

that states and districts might elect to include in their evaluation systems, with support from Title 

II Part A funds under the ESSA. The components include measures of student achievement 

growth, classroom practices, and principal leadership.

Key Question(s) Addressed:

• What is the impact of the performance evaluation system on teachers’ classroom practices 

and on student achievement? 

• What are districts’ and educators’ experiences with implementation? 

Design:

Within each of eight districts, approximately 15 elementary and middle schools were randomly 

assigned to receive the study’s measures of student achievement growth, classroom practices, 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 136

and principal leadership during 2012–13 and 2013–14 (treatment group) or to participate only in 

the district’s usual performance evaluation system during the same time period. In treatment 

schools, each year, teachers received four rounds of classroom observations and feedback 

sessions and information about their students’ achievement growth in math and/or reading using 

a statistical model known as value-added. Principals in treatment schools received feedback on 

their leadership practices twice per year. Study data collection included surveys of teachers and 

principals, observations of teachers’ classroom practices, and collection of student records data. 

The study also looks at the extent to which the measures were implemented as intended and 

whether the ratings from the measures reliably distinguish educator performance.

Estimated or Actual Completion Date:

A report on first year implementation findings was released in November 2016. A final report on 

impacts on educator practices and student achievement as well as implementation during the 

study’s second year was released in December 2017. 

Key Findings from the First Report: 

• The study’s performance measures were implemented generally as planned. Teachers and 

principals received multiple rounds of ratings and feedback on their practices. However, 

fewer principals and teachers accessed their student growth reports than the study 

intended. 

• Both classroom observation and student growth measures differentiated teacher 

performance, although observation scores were mostly at the upper end of the scale. 

Overall, observation scores varied across teachers, and both value-added scores and 

average classroom observation scores over the year had sufficient reliability to capture 

performance differences among some teachers. 

• The principal leadership measure differentiated performance, but principal self-ratings, 

teachers’ ratings of the principal, and the principal’s supervisor’s ratings of the principal 

often differed.

• Both teachers and principals in treatment schools reported receiving more feedback on their 

performance than did their counterparts. For example, teachers and principals in treatment 

schools reported spending more total time in performance feedback sessions across the 

year than teachers and principals in the control schools. 

Link to Additional Information:

https://ies.ed.gov/ncee/projects/evaluation/tq_performance.asp.

Impact Evaluation of the TIF

Study Purpose:

The purpose of the TIF program is to develop and implement performance-based compensation 

systems (PBCSs) for teachers, principals, and other personnel in high-need schools. Research 

indicates that high-quality teachers are critical to raising student achievement in low-performing 

schools, but schools most in need often have difficulty in attracting and retaining high-quality 

teachers. Performance pay is a policy promoted by the TIF program to improve the quality of 

teachers in high-need schools. This evaluation studies performance pay that provides 

substantial and differentiated bonus pay to high-performing teachers in low-performing schools 

with high-need students.

Key Question(s) Addressed:

• What are the characteristics of all TIF grantee districts and their PBCSs? What 

implementation experiences and challenges did TIF districts encounter?
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 137

• How do teachers and principals in schools that did or did not offer pay-for-performance 

bonuses compare on key dimensions, including their understanding of TIF program 

features, exposure to TIF activities, allocation of time, and attitudes toward teaching and the 

TIF program?

• What is the impact of pay-for-performance bonuses on students’ achievement on state 

assessments of math and reading? 

• How do pay-for-performance bonuses affect educator mobility, including whether mobility 

differs by educator effectiveness?

Design:

Study schools were randomly assigned within a grant to either implement all components of the 

PBCS or the PBCS with a 1 percent across-the-board bonus in place of the differentiated 

effectiveness incentive component of the PBCS. Data collection included a grantee survey, a 

survey of teachers and principals, teacher and principal school assignment records, student 

record information (such as student demographics and student test scores), and grantee 

interviews.

Estimated or Actual Completion Date:

The first report was released in September 2014. The second report was released in September 

2015. The third report was released in August 2016. The fourth and final report, which updated 

implementation information and impacts, was released in December 2017.

Key Findings from the Third Report:

The main findings among all TIF districts with 2010 awards are:

• Similar to the previous two years, most districts (88 percent) implemented at least three of 

the four required program components for teachers.

• By the third year, reported implementation challenges decreased with no more than one-fifth 

of TIF districts reporting any major challenges.

• For the subset of 10 districts that agreed to participate in a random assignment study, key 

findings on the effect of pay-for-performance on educators include the following: 

• After three years of TIF implementation, average student achievement remained 1 to 

2 percentile points higher in schools that offered pay-for-performance bonuses than in 

schools that did not. This difference was equivalent to a gain of about four additional weeks 

of learning.

• At least half of the evaluation districts each year met the grant guidance for awarding 

differentiated performance bonuses for teachers. However, in each year, no more than 2 of 

the 10 districts awarded bonuses for teachers that were substantial or challenging to earn.

• Teachers’ understanding of performance measures continued to improve, but only about 60 

percent of teachers correctly reported being eligible for a performance bonus. In addition, 

teachers believed that the maximum bonus they could earn was no more than two-fifths the

size of the actual maximum bonus that districts awarded, a finding similar to previous years.

Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/tq_incentive.asp. 

Impact Evaluation of Support for Principals

Study Purpose:

Title II, Part A, the Improving Teacher State Formula Grants program, is the primary federal 

funding under ESEA to support high-quality educators. The program targets high-poverty 

districts and funds a broad array of allowable activities for principals and teachers, such as 

support for certification, teacher mentoring and induction, intensive professional development, 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 138

recruitment, retention, and merit-based teacher and principal pay strategies as well as class 

size reduction. Principals, through a collective focus on instructional and organizational 

leadership and human capital management, have the potential to greatly influence the quality of 

instruction. However, there is limited evidence about the effectiveness of principal professional 

development programs and their ability to improve principals’ leadership skills and school 

quality. This evaluation studies professional development for principals and thus provides an 

important source of information for this program. 

Key Question(s) Addressed:

• What are the professional development experiences of principals?

• What are the initial impacts on school climate and educator behaviors of providing principals 

structured and intensive professional development?

• What are the impacts on teacher retention, the effectiveness of instructional staff, and 

student achievement of providing principals with structured and intensive professional 

development?

Design:

The study team randomly assigned within 10 districts a total of 100 elementary schools to a 

treatment or control group. Treatment group principals will be offered intensive professional 

development provided by the University of Washington’s Center for Educational Leadership 

during the 2015–16 and 2016–17 school years. The Center for Educational Leadership was 

competitively selected to provide the professional development focused on in this study. The 

professional development includes a heavy emphasis on instructional leadership activities, 

including support in conducting school walkthroughs and classroom observations with 

constructive feedback to facilitate teacher growth focused on improving student achievement. 

Control group principals received supports normally offered by the district. Data collection 

included: information about the professional development delivered and experienced by the 

participating principals; teacher and principal surveys and periodic logs of principal daily 

activities to document intermediate outcomes, such as principal behaviors and school climate; 

and administrative records to document student outcomes (e.g., achievement, behavior, 

attendance) and teacher outcomes (e.g., retention of effective teachers, quality of newly hired 

teachers). 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 139

Estimated or Actual Completion Date:

The report, which will focus on implementation and intermediate outcomes over two years 

(2015-16 and 2016-17), is expected in spring 2019.

Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/tq_principals.asp. 

Study of Teacher Preparation Experiences and Early Teacher Effectiveness

Study Purpose:

A primary focus of the ESSA’s Title II Part A is on the improvement of teacher quality. Little 

research exists, however, to inform how best to prepare teachers for the classroom. This study 

provides descriptive information on the preparation experiences of a large sample of novice 

teachers. It also will examine whether the instructional skills that teachers learn about and have 

opportunities to practice in their preparation programs are associated with teachers’ 

effectiveness once they are in the classroom.

Key Question(s) Addressed:

• What experiences do novice teachers report receiving as part of their preparation program?

• What are the relationships between teacher-reported preparation experiences and novice 

teachers’ effectiveness in improving student achievement?

Design:

Approximately 3,200 novice language arts and/or math teachers from grades 4 through 6 

participated in the study. In spring 2015, the teachers responded to a survey focused on their 

preparation experiences related to instructional skills that have been shown to be associated 

with teacher value-added scores. For each instructional skill, the teachers were asked about the 

nature of their learning experiences, including how they learned (e.g., through coursework or 

through practice in K-12 classroom) and the extent to which they found the experience useful. 

The study will also compute value-added scores for teachers, based on students’ state math 

and English language arts tests, and examine the relationships between teacher preparation 

experiences and teacher value-added scores. 

Estimated or Actual Completion Date:

A report describing teachers’ preparation experiences and the relationship between experiences 

and teacher value-added scores is expected by fall 2018.

Link to Additional Information:

http://ies.ed.gov/ncee/projects/evaluation/tq_teacherprep_early.asp.

Impact Evaluation of Professional Development for Teachers on Data-Driven 

Instruction

Study Purpose:

Title II, Part A, the Improving Teacher Quality State Formula Grants program, is the primary 

federal funding under the ESEA to improve teacher quality. The program targets high-poverty 

districts and funds a broad array of allowable activities including professional development for 

teachers. Teachers and schools have access to ongoing data from student performance and 

assessments that provide information about students' academic needs. This study evaluates the 

effectiveness of professional development for teachers to use such data to guide their 

classroom instruction. 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 140

Key Question(s) Addressed:

• What are the effects on student achievement of providing professional development to 

principals and teachers that focuses on using individual student academic information 

already available to school staff to guide instruction? 

• What are the effects of providing this professional development on teachers’ and principals’ 

use of individual student academic information to inform instructional support, planning, and 

practice?

Design:

The study is a randomized controlled trial with a sample of 104 schools in 12 districts. In each 

district, schools were randomly assigned to treatment (professional development) or to control 

(business as usual) status. Data collection consists of a teacher survey, a principal survey, and 

student administrative records, including student state standardized achievement test scores.

Estimated or Actual Completion Date:

The report is expected in summer 2018.

Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/tq_datadriven.asp. 

School Choice

Evaluation of the DC Opportunity Scholarship Program: Impacts Two Years After 

Application and Final Report

Study Purpose: 

The April 2011 SOAR Act provided for a five-year continuation of a school choice program for 

low-income residents of Washington, DC. The program, still titled the Opportunity Scholarship 

Program (OSP), now provides annual scholarships of $8,000 (for grades K–8) or $12,000 (for 

grades 9–12), plus a cumulative inflation adjustment, to enable low-income students to attend 

private schools in DC in lieu of the public schools already available to them. The new law also 

mandated another independent, rigorous evaluation of the effectiveness of the program.

Key Question(s) Addressed:

• What is the impact of the OSP on student academic achievement and other measures of 

student success, overall and for subgroups of students identified in the statute as high 

priority?

• What effect does the program have on student and parent perceptions of school safety and 

satisfaction, and on parents’ involvement in education of their children?

• Does the program change students’ instructional environments and opportunities?

Design:

The evaluation will primarily compare outcomes of approximately 1,800 student applicants 

randomly assigned by lottery to either receive a scholarship or not receive a scholarship. 

Lotteries of program applicants were conducted in spring 2012 (cohort 1), spring 2013 (cohort 

2), and spring 2014 (cohort 3). Data were collected for three follow-up years for each of the 

cohorts and, for students in both the scholarship and nonscholarship groups. The contractor 

administered academic assessments, and conducted student, parent, and principal surveys 

each spring (spring 2013–16). Prior to the first impact report, descriptive reports based on 

application forms parents filled out when they applied to the OSP, principal surveys, and extant 

data were released.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 141

Estimated or Actual Completion Date:

The second and final impact reports are planned for early spring 2018 and late fall 2018.

Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/choice_soar.asp.

Parent Information and School Choice Evaluation

Study Purpose:

School choice is among the most visible K-12 education policy trends to emerge over the last 

two decades. For school choice to be effective, it seems critical that parents are able to navigate 

school choice systems and process large amounts of complex information about schools and 

application procedures to make informed choices. However, few studies have rigorously 

examined school choice information strategies or attempted to identify effective methods of 

information presentation. This evaluation is designed to address this significant gap in the 

literature. 

Key Question(s) Addressed:

• Which formats make school choice information displays easiest to understand and use? 

(e.g., Is it better to show school performance data with numbers, graphs, or icons?)

• How does the amount of information displayed affect understanding and use? (e.g., How 

much information about school performance should be shown? Should displays include 

district averages or parent survey data?) 

• How should school choice information displays be organized? (e.g., How should schools be 

ordered, when a list of options is shown?)

Design:

A low–cost, quick turn-around “lab” based experiment was carried out where about 3,500 lowincome parents of school-aged children participated online. Parents were randomly assigned to 

view one of 72 versions of a school choice information display and then answered survey 

questions about their understanding of the information, ability to use the information, and which 

schools they would select based on the information they were provided. Responses to these 

survey questions will be compared across the strategies tested (e.g., data shown with numbers, 

graphs, or icons) to determine which ones are most promising.

Estimated or Actual Completion Date:

A short user-friendly guide, based on the evaluation’s findings, that will help school districts and 

other providers of school choice information design or refine their own parent information 

materials is expected in spring 2018.

Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/choice_parent.asp.

Students with Disabilities

National Longitudinal Transition Study (NLTS) 2012

Study Purpose:

Supporting youth’s successful transition to adult life is a priority under the Individuals with 

Disabilities Education Act (IDEA). IDEA provides federal funds to states and through them to 

school districts to assist in providing special education and related services. States and school 

districts must make a FAPE available to all eligible children with disabilities. FAPE includes the 

provision of special education and related services provided in conformity with an Individualized 

Education Program. IDEA requires that the first IEP to be in effect when a child turns 16 include 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 142

postsecondary goals and transition services needed to help youth with disabilities complete high 

school prepared to achieve appropriate post-school outcomes, such as postsecondary 

education, jobs, and independent living. The National Longitudinal Transition Study 2012 (NLTS 

2012) is the third in a series examining the characteristics, school experiences, and post-high 

school outcomes of a nationally representative sample of youth with disabilities. NLTS 2012 

focuses on youth ages 13 to 21 (in December 2011), but also includes a small sample of 

students without disabilities to enable direct comparisons of students with and without an IEP. 

This study is part of the congressionally mandated National Assessment of the IDEA and is 

supported with funds authorized under Section 664 of IDEA.

Key Question(s) Addressed:

• How do the personal, family, and school characteristics and experiences of youth with 

disabilities differ from those of youth not served under IDEA? (Volume 1; March 2017)

• How do the characteristics and experiences of youth vary across disability groups? (Volume 

2; March 2017) 

• How have the characteristics and experiences of youth with disabilities changed over time? 

• To what extent do youth with disabilities make progress through high school compared to 

other youth?

• Are youth with disabilities achieving the post-high school outcomes envisioned by IDEA, and 

how do their college, training, and employment rates compare to those of other youth? 

• How do these high school and postsecondary outcomes vary with student characteristics?

Design:

This descriptive study includes a nationally representative sample drawn from 432 school 

districts and special schools randomly sampled in 2011 and students randomly sampled within 

those districts. Survey data were collected in 2012-2013 from approximately 12,000 in-school 

youth and their parents, of which about 10,000 are students with IEPs representing each of the 

federal disability categories. The surveys asked about the background characteristics of 

secondary school youth and the schools they attend, their health, functional abilities, and 

engagement in school, the academic supports they receive, and their expectations for and steps 

to achieve transitions beyond high school. High school records and postsecondary enrollment 

information from the National Student Clearinghouse are being collected (from 2017-2019) to 

follow the sample of students through high school and beyond. This administrative data will be 

linked with the 2012-2013 survey data to examine high school course-taking and completion, 

and youth’s experiences with college, training, and employment.

Estimated or Actual Completion Date:

Two volumes of a report describing the survey results were released in March 2017. The third 

volume of this report, examining trends over time by comparing the NLTS 2012 survey results 

with those from two earlier NLTS surveys, is expected in 2018. 

Key Findings (from the March 2017 volumes): 

• Youth with an IEP are more likely than their peers to be socioeconomically disadvantaged. 

Youth with an IEP are 12 percentage points more likely to live in low-income households 

and are less likely to have parents who are employed or have a college education. Among 

disability groups, youth with intellectual disability and youth with emotional disturbance are 

more socioeconomically disadvantaged and more likely to attend a lower-performing school 

than youth with an IEP overall.

• The vast majority of youth with and without an IEP feel positive about school, but those with 

an IEP experience bullying and are suspended at higher rates. Like their peers, more than 

80 percent of youth in special education report that they are happy with school and with 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 143

school staff. However, not only do youth with an IEP more commonly experience some 

types of bullying (e.g., being teased or called names) but, according to parent reports, they 

are more than twice as likely to be suspended or expelled from school.

• Youth with an IEP are more likely than other youth to struggle academically, yet less likely to 

receive some forms of school-based support. Half of all youth with an IEP report they have 

trouble with their classes, about 15 percentage points more than reported by their peers. 

However, they are less likely to report receiving school-based academic help before or after 

regular hours, although their parents more commonly help with homework and attend a 

parent-teacher conference. 

• Youth with an IEP lag their peers in planning and taking steps to obtain postsecondary 

education and jobs. Substantially fewer youth with an IEP expect to enroll in postsecondary 

education or training, compared to youth without an IEP. Reflecting these gaps, youth in 

special education are almost half as likely as their peers to report taking college entrance 

and placement tests. Forty percent report having recent paid work experience while in high 

school, compared with 50 percent of youth without an IEP. 

• Youth with autism, deaf-blindness, intellectual disability, multiple disabilities, and orthopedic 

impairments are most at-risk for not transitioning successfully beyond high school. Youth in 

these groups are less likely than all youth with an IEP to have key characteristics and 

experiences linked to success after high school, such as performing typical daily living tasks, 

engaging with friends and in school activities, or preparing for college, careers, and 

independent living.

Link to Additional Information: 

https://ies.ed.gov/ncee/projects/evaluation/disabilities_nlts2012.asp. 

Preschool Special Education Programs and Practices Supporting Children with 

Disabilities

Study Purpose:

Most recently reauthorized in 2004, IDEA provides funding to states to support special 

education and related services for children and youth with disabilities, including young children 

ages 3–5 years old. Currently there is limited information available on the curricula and 

interventions being used across states to support young children with disabilities. Phase I of the 

Evaluation of Preschool Special Education Practices is being conducted to address the primary 

objective of assessing the feasibility of conducting a large-scale impact study of curricula or 

interventions promoting the literacy, language, and/or social-emotional skills of preschool-age 

children with disabilities. A secondary objective of the Phase I study is to provide nationally 

representative descriptive information about preschool special education programs and the 

specific curricula or interventions being delivered to preschool children with disabilities. 

Key Question(s) Addressed:

• Which curricula and interventions are used nationally for preschool children with 

disabilities to promote learning of language, literacy, and social emotional skills? 

• How are states and school districts structuring programs to serve children ages 3 

through 5 with disabilities?

• What staff resources are available to support the instruction of children ages 3 through 5 

with disabilities?
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 144

Design:

The study collected survey data from state and district grantees of IDEA funds to obtain 

nationally representative information on the programs, services, curricula and interventions 

available to children ages 3–5 identified for special education services. The survey sample 

included a nationally representative sample of 1,200 school district preschool special education 

coordinators and state Section 619 coordinators in all 50 states and DC. If feasible to conduct, 

preparations for the impact study will occur under Phase I. 

Estimated or Actual Completion Date:

A descriptive report on the survey findings is expected in fall 2018. 

Link to Additional Information:

https://ies.ed.gov/ncee/projects/evaluation/disabilities_preschool.asp. 

National Evaluation of the IDEA Technical Assistance and Dissemination 

Program: Final Report

Study Purpose:

IDEA, which was most recently reauthorized in 2004, provides funds to assist states and local 

educational agencies in making a free appropriate public education available to all eligible 

children with disabilities. Funded at $12.6 billion in FY 2010, IDEA supports early intervention 

services for infants and toddlers with disabilities (birth through age 2) and special education and 

related services for children with disabilities (ages 3 through 21). IDEA permits local educational 

agencies to use a portion of their IDEA funds to develop and implement coordinated early 

intervening services (CEIS) for students who are not currently identified as needing special 

education or related services, but who need additional academic and behavioral support to 

succeed in a general education environment. The use of IDEA funds for CEIS is voluntary, 

except when an LEA is identified with significant disproportionality. IES is conducting studies 

under Section 664 of IDEA to assess the implementation and effectiveness of key programs and 

services supported under the law.

As specified in IDEA Part D, the Technical Assistance and Dissemination (TA&D) program is to 

provide technical assistance, support model demonstration projects, disseminate useful 

information, and implement activities that are supported by scientifically based research to meet 

the needs of children with disabilities. The National Evaluation of the IDEA TA&D program is 

designed to describe the products and services provided by the TA&D program grantees, state 

and local needs for technical assistance, and the role that the TA&D program plays in meeting 

these needs and supporting implementation of IDEA. The State Deaf-Blind Projects are part of 

the TA&D Program and are the focus of the evaluation’s final report.

Key Question(s) Addressed:

• What technical assistance and dissemination activities do State Deaf-Blind Projects provide 

and how does this vary across the states?

• How do State Deaf-Blind Projects collaborate with other organizations in their state, with 

other technical assistance providers, and across the network of State Deaf-Blind Projects?

• What are the needs for technical assistance among direct service providers who work with 

children and youth with deaf-blindness? 

• How satisfied are direct service providers with services received from the State Deaf-Blind 

Projects?
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 145

Design:

Data collection for the interim report included administering surveys to the national and regional 

level TA&D program grantees, all state IDEA Part B and Part C administrators, and a sample of 

state-level special education program staff. Data were collected between November 2012 and 

March 2013. State-level administrators and staff reported on their receipt of technical assistance 

from TA&D program grantees during the 2010–11 school year. Grantees reported on their goals 

and activities from the beginning of their current grant through the interview date. The funding 

period for the centers included in the interim report varied, with the earliest end date in 2012 and 

the latest in 2014. Data collection for the final report included administering surveys to the 48 

state level deaf-blind projects awarded grants in 2013 and to a sample of local level special and 

general education providers currently working with deaf-blind children. 

Estimated or Actual Completion Date:

The final report is expected in early 2018.

Key Findings from the Interim Report:

• TA&D centers most commonly reported providing technical assistance on the topics of 

”parent and family involvement” and “data systems and use of data for improvement.” States 

identified (a) “General Supervision/Monitoring,” (b) ”early childhood transition,” (c) ”financing 

of services/financing for special education” and (d) ”Response to Intervention” as the topics 

for which they had the greatest need for technical assistance in 2010–11.

• Many TA&D centers provide technical assistance on similar topics. For example, 14 states 

received "high intensity" technical assistance (i.e., frequent training or consultation) on the 

same topic from 5 different centers.

• State staff rated the majority of technical assistance experiences they had with TA&D 

centers as "very satisfactory" (71 percent). On average, customers receiving high intensity 

technical assistance were significantly more satisfied than those receiving lower intensity 

(i.e., infrequent training and consultation or web-only support). Satisfaction did vary to some 

degree depending on the special education topic being addressed.

Link to Additional Information:

http://ies.ed.gov/ncee/projects/evaluation/disabilities_idea2004.asp.

Study of Early Intervention and Special Education Services and Personnel

Study Purpose:

IDEA, which was most recently reauthorized in 2004, provides federal funds to states and 

through them to school districts to assist in making a free appropriate public education available 

to all ell eligible children with disabilities. Funded at $12.5 billion in FY 2015, IDEA supports 

early intervention services for infants and toddlers with disabilities (from birth through age 2) 

(IDEA Part C) and special education and related services to children with disabilities (ages 3 

through 21) (IDEA Part B). IDEA permits LEAs to use a portion of their IDEA funds to develop 

and implement coordinated early intervening services for students not currently identified as 

needing special education or related services but who need additional academic and behavioral 

support to succeed in a general education environment. The use of IDEA funds for CEIS is 

voluntary on the part of the LEA, except when an LEA is identified with significant 

disproportionality. Under Section 664 of IDEA, IES is conducting studies to assess the 

implementation and effectiveness of key programs and services supported under the law. This 

study is supporting the analysis of extant data to examine early intervention and special 

education service delivery and the personnel providing services.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 146

Key Question(s) Addressed:

• How has the delivery of early intervention services changed over time, and how do these 

services differ for subgroups defined by age, race/ethnicity, and state?

• How have special education services changed over time, and how do these services differ 

for subgroups defined by age, race/ethnicity, disability category, and state? 

• How has the distribution of personnel providing special education services changed over 

time? 

Design:

This study includes new descriptive analysis of extant data available in public use or restricted 

formats. Among the data sources that are being used are cross-sectional Section 618 data 

submitted by states to the Department and the NCES Schools and Staffing Survey.

Estimated or Actual Completion Date:

The report is expected in early 2018. 

Link to Additional Information:

http://ies.ed.gov/ncee/projects/evaluation/disabilities_persserv.asp.

Impact Evaluation of Training in Multi-Tiered Systems of Support for Behavior 

(MTSS-B)

Study Purpose:

Training school staff in supporting the behavior of all students is becoming increasingly 

attractive to districts and schools as a vehicle for school improvement. Implementation of multitiered systems of support for behavior (MTSS-B) is an approach to improving school and 

classroom climate as well as student outcomes. MTSS-B is a multi-tiered, systematic framework 

for teaching and reinforcing behavior for all students as well as for providing additional support 

to those who need it. Over a third of U.S. districts report implementing multi-tiered systems of 

behavior support at the elementary school level. Recent small-scale studies have shown the 

promise of MTSS-B. This evaluation occurs under the National Assessment of the Individuals 

with Disabilities Education Act (IDEA), which permits school districts to use a portion of their 

IDEA funds to provide coordinated early intervening services to students who are not currently 

identified as needing special education or related services, but who need additional academic 

and behavioral support to succeed in a general education environment, such as MTSS-B. 

Key Question(s) Addressed:

• What MTSS-B training and support activities were provided? What MTSS-B activities 

occurred in the schools receiving MTSS-B Training? How do these MTSS-B activities differ 

from those in schools that do not receive the training?

• What is the impact on school staff practices, school climate and student outcomes of 

providing training in the MTSS-B framework plus universal (Tier I) positive behavior supports 

and a targeted (Tier II) intervention?

• What are the impacts for relevant subgroups (e.g., at-risk students)?

Design:

This is a randomized trial of the impact of training in MTSS-B on school climate, school staff 

practice, and student outcomes. The contractor, with assistance and input from the Department 

and in consultation with a panel of experts, competitively selected a MTSS-B training provider, 

the Center for Social Behavior Support, which is a collaboration between the Illinois-Midwest 

Positive Behavior Interventions and Supports network at the School Association for Special 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 147

Education in DuPage, Illinois and the Positive Behavior Interventions and Supports network 

Regional Training and Technical Assistance Center at Sheppard Pratt, in Maryland. 

Approximately 90 elementary schools were randomly assigned to either training in MTSS-B 

including universal supports (Tier I) plus targeted interventions for at-risk students (Tier II) or a 

business-as-usual control group. Treatment schools are receiving training in MTSS-B prior to 

and across two school-years, 2015–2016 (Tier I) and 2016–2017 (Tiers I and II), and implement 

MTSS-B across these two years. Data collection will include a staff survey, teacher ratings of 

student behavior, classroom observations, site visits, and student records data. 

Estimated or Actual Completion Date:

The report is expected in 2019.

Link to Additional Information: 

http://ies.ed.gov/ncee/projects/evaluation/disabilities_MTSSB.asp. 

Pathways to Career or College

Evaluation of the Pell Grant Experiments Under the Experimental Sites Initiative

Study Purpose:

Federal Pell grants are considered the foundation of higher education financial aid for lowincome students. However, under current rules, otherwise income-eligible students who already 

have a bachelor’s degree (BA) or who want to enroll in short-term (less than 15 weeks and 600 

hours) programs are restricted from obtaining these grants. Given unemployment rates above 

8.5 percent in 2011, and reports of unfilled openings for skilled jobs in some occupations, 

postsecondary institutions called for expanding Pell grants to help fill the skill training gap for 

low-income workers. In response, the Office of Federal Student Aid, under the Experimental 

Sites Initiative authorized by section 487A(b) of the Higher Education Act of 1965, is conducting 

demonstrations to test the impacts of eliminating the BA restriction (Experiment #1) and 

significantly lowering the minimum clock hours/duration restriction (Experiment #2) for students 

interested in vocational training in high-demand fields. The IES designed and is overseeing a 

rigorous evaluation of these experiments.

Key Question(s) Addressed: 

• Does expanding Pell grant eligibility to include income-eligible students with a bachelor's 

degree and/or to cover shorter-term programs improve access to job training?

• Does expanding Pell grant eligibility to these groups affect financial aid receipt and/or 

student debt?

• Do these two Pell grant experiments improve persistence and completion rates?

• Is there any evidence of an impact on employment and earnings?

Design:

Close to 50 higher education institutions that chose to participate have identified nearly 3,000 

students eligible for the experiments between the 2012-2013 and 2016-2017 financial aid award 

years. Students were randomly assigned to receive a Pell grant or not to receive a Pell grant in 

their financial aid package. Student administrative data from participating higher education 

institutions will be collected in the fall of 2017 and 2018 to allow for the maximum number of 

study participants to complete their programs. Data collection will also include student 

administrative records on financial aid receipt from the Office of Federal Student Aid and, 

potentially, earnings data from the Social Security Administration.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 148

Estimated or Actual Completion Date:

The report for the study is expected in fall 2019.

Link to Additional Information: 

https://ies.ed.gov/ncee/projects/evaluation/pathways_pell.asp. 

Enhancing Advising to Improve College Fit in Upward Bound

Study Purpose:

Growing concern over college enrollment and completion rates has heightened interest in 

strategies to improve the outcomes of low-income students, including those in the Department’s 

high-profile college access program Upward Bound. One aspect of this concern is that many 

low-income high school students fail to enroll in colleges that are well aligned with their 

qualifications, talents, and needs. Research suggests that this kind of “undermatching” or lack of 

college fit can have longer-term consequences for completion and later earnings. This study 

evaluates the effectiveness of a low-cost enhancement to current college advising approaches 

in Upward Bound that is designed to improve college fit and persistence. The approach includes 

professional development for Upward Bound advisors and user-friendly packets of information 

for students that demonstrate the value of considering a range of institutional indicators in their 

college application, search, and acceptance process.

Key Question(s) Addressed: 

• Can an enhanced college advising approach improve upon what Upward Bound grantees 

are already doing, i.e., does it have positive effects on the number of colleges to which 

Upward Bound participants apply, the quality/selectivity of the colleges in which they enroll, 

and their persistence? 

• In what types of grantees is this approach most effective and with what types of students?

Design:

About 200 Upward Bound grantees that volunteered were randomly assigned in spring 2015 so 

that half received the professional development and packets to begin using with their rising 

2015-16 seniors (the treatment group) and half did not receive the enhancements during the 

study period (the control group). A survey administered in spring 2016 to seniors in both sets of 

projects collected information about their college planning, including how many and to which 

colleges they applied. The study will also examine those seniors’ later enrollment and 

persistence in college using administrative records.

Estimated or Actual Completion Date:

The first report will examine the impacts of the enhanced advising on early indicators of college 

going (number of college applications, selectivity of the colleges to which students applied, 

FAFSA completion by March 15) and is expected to be released in spring 2018. A second report 

examining impacts on college enrollment and fit is expected in fall 2018.

Link to Additional Information: 

http://ies.ed.gov/ncee/projects/evaluation/pathways_upward.asp. 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 149

The Effectiveness of Text Messaging to Support College Transition of GEAR UP 

Students

Study Purpose:

The Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP) is one 

of the Department’s college access programs, funding states or local partnerships of districts 

and postsecondary institutions to serve students in high need schools beginning in 7th grade. 

The 2008 amendments to the Higher Education Act allowed GEAR UP grantees to serve 

participating students beyond high school graduation and into a first year of college, when they 

might be dispersing to a variety of college campuses. This demonstration evaluates a low-cost 

way to provide these services, building on emerging evidence that customized text messages 

can help students overcome logistical and behavioral challenges that might otherwise derail 

their college matriculation and persistence into sophomore year. The messages include 

reminders and information relating to college registration, course selection, financial aid award 

and renewal, meeting with college advisors and faculty, and tuition payments, and provide a 

way to access real-time support from GEAR Up counselors.

Key Question(s) Addressed: 

• Do the text-based reminder strategies improve GEAR UP students’ rates of college 

enrollment and persistence?

• For which types of students is the strategy more or less beneficial?

Design:

Across about 80 GEAR UP high schools, approximately 6,000 seniors in school years 2015–16 

or 2016–17 were randomly assigned to receive either the college transition services grantees 

originally proposed in their applications or those regular transition services plus the customized 

reminders and support through text messages. Reminders and support began at the end of 

students’ high school senior year and continue into the spring of their expected first year of 

college. The study team administered a survey before students were randomly assigned in 

order to collect information on students’ experiences with college advising and their intended 

college (so that the messages can be tailored to individual schools’ deadlines and 

requirements). College enrollment and persistence, as well as FAFSA renewal, will be 

measured using administrative records.

Estimated or Actual Completion Date:

The first report examining impacts on initial college enrollment is expected in early 2019.

Link to Additional Information: 

http://ies.ed.gov/ncee/projects/evaluation/pathways_gearup.asp. 

Other

Implementation Study of State Supports under Title I for Reducing School 

Dropouts

Study Purpose:

Title I, Part A of the ESSA requires states to support local educational agencies in providing 

effective transitions of students at all levels of schooling, especially middle grades and high 

school, to decrease the risk of students dropping out (Section 1111(g)(1)(D)). Title IX, Section 

9208 of ESSA requires an evaluation of these efforts. This report will describe the 

implementation of dropout prevention strategies at the state and district levels and report recent 

dropout rate trends. 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 150

Key Question(s) Addressed:

• How do states and districts identify students at risk of dropping out? 

• What strategies do districts use to help students transition from elementary to middle school 

and from middle to high school? What services or options do districts offer to students at risk 

of dropping out? How do states support these efforts?

• What are recent trends in dropout rates at the national and state levels?

Design:

Data will be collected from all 50 states and the District of Columbia, and a nationally 

representative sample of school districts through surveys during the 2017–18 school year to 

inform the first two research questions. Extant data in dropout rates will also be collected to 

inform the third research question.

Estimated or Actual Completion Date:

An evaluation brief describing findings is expected in summer 2019.

Evaluation of Investing in Innovation

Study Purpose:

The Investing in Innovation (i3) Fund was established by the American Recovery and 

Reinvestment Act of 2009 to provide funds to local educational agencies and nonprofits with a 

record of improving student achievement. The program awards competitive grants to implement 

and evaluate educational interventions to improve student academic outcomes.

The i3 program awarded three different tiers of grants. The amount of funding awarded was 

aligned with the strength of the prior evidence supporting the proposed intervention as well as 

the proposed implementation scale. The smallest, or “Development,” grants supported 

developing and testing interventions with limited or no prior evidence. Interventions with 

moderate evidence of effectiveness could receive a larger “Validation” grant, to implement and 

test the intervention in a broader population or in new contexts. The largest, or “Scale-up,” 

grants supported interventions with strong prior evidence of effectiveness to be implemented 

and tested on a much larger scale. 

The i3 program funded seven cohorts of grantees from 2010 through 2016. The program has 

awarded more than $1.4 billion to 172 grantees across the three tiers. One of the i3 program's 

key priorities is evaluating the impact of the funded interventions. Each grantee was required to 

fund an independent evaluation using local evaluators. To maximize what could be learned from 

these evaluations, the Department is conducting an Evaluation of i3, which (1) provides 

comprehensive technical assistance to local evaluators for all grantees, and (2) summarizes 

both the quality of and findings from the evaluations.

Key Question(s) Addressed:

• Were the i3 evaluations strong?

• Did the i3 evaluations find the interventions to be implemented with adequate fidelity?

• Did the i3 evaluations find that the interventions improved student academic outcomes?

Design:

This study is descriptive and includes technical assistance to support the i3 local evaluators. In 

addition to providing technical assistance, the contractor will characterize whether or not the 

evaluations meet several criteria used to define a strong study and provide a report for the 

program office each year about the number of grantees meeting the i3 evaluation-related 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 151

Government Performance and Results Act measures. The contractor will also summarize the 

results of the i3 evaluations.

Estimated or Actual Completion Date:

The first report, which will focus on findings for the 67 i3 evaluations completed by May 2017, is 

expected in spring 2018. 

Link to Additional Information: 

http://ies.ed.gov/ncee/projects/evaluation/assistance_ita.asp. 

Progress and Challenges in Developing Tiered Quality Rating and Improvement 

Systems (TQRIS) in the Round 1 Race to the Top-Early Learning Challenge (ELC) 

States

Study Purpose:

The Race to the Top–Early Learning Challenge Program (RTT-ELC) aims to improve early 

learning and development programs so children enter kindergarten ready to succeed. The 

program awarded $1 billion in four-year grants to 20 states in three rounds between 2011 and 

2013. The program promotes reform in five key areas: (1) state systems; (2) high-quality, 

accountable programs; (3) early learning and development outcomes for children; (4) workforce; 

and (5) measurement. The second area focuses on the design and implementation of TQRIS 

that can provide parents and other stakeholders with information on the quality of early learning 

programs. To better understand the TQRIS that RTT-ELC grantees developed, this descriptive 

study will: (1) examine the structure and characteristics of grantee states' TQRIS, (2) analyze 

state administrative data to examine patterns in the participation and distribution of programs in 

TQRIS, and (3) synthesize findings from grantee states with completed TQRIS validation 

studies. This study focuses on the nine Round 1 states, which received over $500 million in 

2011.

Key Question(s) Addressed: 

• How are TQRIS structured and implemented in the nine Round 1 RTT-ELC states?

• How are TQRIS ratings defined, collected, and generated in the nine Round 1 RTT-ELC 

states?

• What are the patterns in TQRIS participation, the distribution of programs and children 

across the rating levels and the movement of programs and children across the rating levels 

since 2011?

• How do TQRIS characteristics (include structure, policies, and practices) relate to program 

movement up the rating levels and achievement of the highest rating level?

• What have completed RTT-ELC state validation studies found about the relationship 

between TQRIS tiers and program quality and the relationship between children's 

development and TQRIS ratings?

Design:

This descriptive study collected various data from the nine Round 1 RTT-ELC grantee states. In 

fall 2014, the study collected and conducted a targeted review of documents describing the 

structure of TQRIS, including component measures and the quality indicators used to evaluate 

preschool programs, and how these are combined to generate overall ratings. In fall 2014 

through winter 2015, the study also conducted interviews with state administrators to confirm 

and clarify the information obtained from documents and gather information that could not be 

obtained from the document reviews.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 152

To address the third and fourth research questions above, the study collected administrative 

data from the Round 1 RTT-ELC grantee states. The study will answer the last question by 

conducting a systematic review of up to nine completed RTT-ELC grantee state validation 

studies available in 2017.

Estimated or Actual Completion Date:

A report addressing the first two questions is expected in fall 2017. Evaluation briefs to address 

the third, fourth, and fifth questions are expected in fall 2018.

Link to Additional Information: 

http://ies.ed.gov/ncee/projects/evaluation/other_rtt.asp. 

Evaluation of the Comprehensive Technical Assistance Centers, FY 2012 

Grantees

Study Purpose:

The Comprehensive Technical Assistance Centers is a federally funded program currently 

authorized under the Educational Technical Assistance Act of 2002. The Department awarded 

five-year grants in FY 2012 to 22 Comprehensive Technical Assistance Centers with the 

purpose to help state education agencies build their capacity to implement state-level initiatives 

and to support district- and school-level initiatives that improve educational outcomes for all 

students, close achievement gaps, and improve the quality of instruction. The FY 2014 

appropriation for the Centers was $48.4 million. This study will inform the Department of 

Education, the Comprehensive Center program, and the larger field about the design, 

implementation, and outcomes of the Centers’ work.

Key Question(s) Addressed:

• Program Design: How did the Centers design their work? How did Centers define capacity 

building? Did their definitions change over time? If so, how? What theories of action did 

Centers use to guide their general capacity-building work? Did the theories change over 

time? If so, how? How did Centers assess the needs of their constituencies?

• Program Implementation: How did the Centers operate? What strategies did Centers employ 

to achieve their outcomes? To what extent did Centers implement technical assistance to 

their constituencies as planned? To what extent and how did Centers collaborate with each 

other?

• Program Outcomes: What was the result of the Centers’ work? To what extent did Centers 

achieve their goals and objectives?

Design:

This evaluation is a multiyear descriptive study examining the Centers’ programs. Data on the 

Centers’ activities and outcomes will be collected during the FY 2015, FY 2016, and FY 2017 

program years. The evaluation will describe how the individual Centers intend to build SEA 

capacity (their theories of action) and document what types of activities they actually conduct to 

build capacity. The evaluation plans to focus on Center projects in two priority areas: great 

teachers and leaders and early learning. Data collection will include: (1) the Centers’ 

management plans and technical assistance activity data; (2) interviews with staff from each 

Center; (3) interviews with technical assistance recipients; (4) a survey of Center staff; and (5) a 

survey of technical assistance recipients. This approach will yield a diverse set of data that can 

be analyzed and summarized using qualitative research methods and simple quantitative 

tabulations. Additionally, a small set of project profiles will be put together using the data to 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 153

illustrate in more depth the particular strategies that Centers may use to support capacity 

building and achieve planned outcomes.

Estimated or Actual Completion Date:

A report describing findings is expected in fall 2018.

Link to Additional Information: 

http://ies.ed.gov/ncee/projects/evaluation/other_techcenters12.asp. 

Impact Evaluation of Parent Messaging Strategies on Student Attendance

Study Purpose:

Student attendance is a strong predictor of student success, even in the early school years. In 

some communities, a quarter of all students in kindergarten through third grade are chronically 

absent, and this is most prevalent among low-income students. Under the recently reauthorized 

Elementary and Secondary Education Act (ESSA) many states plan to hold schools 

accountable for reducing chronic absenteeism. One potential low-cost intervention that schools 

are increasingly trying involves text messaging parents to provide relevant tips and motivation to 

improve their child’s attendance. This evaluation is designed to provide evidence on the 

effectiveness of such an intervention for students attending low-performing elementary schools. 

A novel aspect of the intervention is that it is adaptive. Initially, parents will be treated with “lighttouch” messaging in the fall, and those that do not appear to be responsive will subsequently 

receive more intensive messaging in the spring.

Key Question(s) Addressed:

• What is the impact on student attendance of using text messaging to provide parents with 

basic information related to their child’s attendance? Does it matter if messages are 

positively or negatively framed?

• For parents who do not respond to the low-intensity messaging strategies, which intensified 

strategy works better to improve attendance, one that includes direct outreach from school 

staff or one that uses automated methods to improve motivation and behavioral skills?

• Do the combinations of fall and spring messaging strategies (i.e., the adaptive interventions) 

have effects on end-of-year attendance and achievement when compared to each other and 

to business-as-usual attendance strategies?

• How is the messaging intervention implemented, and what are its costs?

Design:

The evaluation will use a variant of a typical random assignment design, called a sequential, 

multiple assignment, randomized trial. This design will measure the effectiveness of different 

initial messaging strategies, and examine if there are any benefits to following up with more 

intensive strategies for parents who do not respond initially. Data will be collected to examine 

both the implementation of the intervention and the impact of the intervention on student-level 

outcomes, such as attendance and achievement. Specifically, information gathered from the 

text-messaging vendor’s platform, a brief log that school staff will complete to document their 

parent outreach activities, interviews with district staff, and a survey of parents will be used to 

assess how well the intervention was implemented and to examine its costs. Information 

gathered from extant district records, including student absences and test scores, will be used 

to examine the impact of different messaging strategies.

Estimated or Actual Completion Date:

A report describing findings is expected in 2019.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 154

Link to Additional Information: 

https://ies.ed.gov/ncee/projects/evaluation/other_messaging.asp. 

Policy and Program Studies Service

Teacher Education Assistance for College and Higher Education Grant (TEACH) 

Study

Study Purpose:

The TEACH Grant program provides grants up to $4,000 a year to students who are completing 

coursework needed to begin teaching. If a recipient does not complete four years of service in a 

high-need subject at a high-need school within eight years after completing their coursework, 

their grant funds are converted to a direct unsubsidized loan. This study examines how 

institutions support and inform students who are eligible for TEACH Grants. In addition, the 

study examines why some participants do not meet TEACH Grant service requirements. Lastly, 

the study examines factors associated with grant recipients meeting and not meeting service 

requirements.

Key Question(s) Addressed: 

• How are TEACH Grants administered in institutions of higher education? 

• Why do some TEACH Grant recipients fail to meet program service requirements?

• What are the factors associated with TEACH Grant recipients meeting and not meeting the 

grant service requirements?

Design:

The study includes surveys of 1) 479 institutions that administered at least 10 TEACH Grants in 

the 2014–15 award year and 2) a sample of 500 TEACH Grant recipients. In addition, the study 

includes case studies of six institutions and an analysis of administrative data. 

Estimated Completion Date:

The final report is scheduled for completion by the spring of 2018. 

National Survey on High School Strategies Designed to Help At-Risk Students 

Graduate

Study Purpose:

This nationally representative survey of high school administrators is examining strategies that 

high schools use to reduce students’ likelihood of dropping out of high school and to increase 

their likelihood of attaining a high school credential. The survey seeks information on the 

prevalence of high school graduation strategies, the students who participate in them, and how 

high schools deliver services or interventions as part of that strategy. The Department will 

release a set of issue briefs based on the survey data to describe the prevalence and 

characteristics of dropout prevention strategies for at-risk youth and will compare high schools 

with high and low graduation rates, among other school characteristics.

Key Question(s) Addressed: 

• What strategies are high schools implementing to help students stay in school and 

graduate? Do these strategies vary for high schools with high or low graduation rates?

• How many students are served through each of these strategies? Are the strategies focused 

on particular student populations?
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 155

• How do schools deliver services or interventions for each of the strategies? What specific 

services are provided, and who provides the services? 

Design:

The study is conducting a web-based survey of a nationally representative sample of 

approximately 2,000 high school administrators.

Estimated Completion Date:

Issue briefs are scheduled for completion between winter 2016–17 and winter 2017-18. 

Implementation Study of the Turnaround School Leaders Program 

Study Purpose:

This study will examine the implementation of the Turnaround School Leaders Program (TSLP). 

This study seeks to generate information to help policymakers and practitioners who struggle 

with the challenges of developing leaders to turn around low-performing schools and to add to 

the field’s general body of knowledge about developing turnaround leadership pipelines.

Key Question(s) Addressed: 

• How do TSLP projects identify, develop, and support leaders for low-performing schools?

• What role do project partners play in implementing projects and helping grantees to achieve 

project goals?

• How have grantees modified projects to adapt to challenges or meet the demands of 

changing circumstances?

• How are grantees measuring the success of their TSLP projects, and do early outcome data 

show promising results?

• How have the turnaround school leaders’ grants contributed to developing a sustainable, 

long-term pipeline of leaders for turnaround schools?

Design:

The study will include surveys of 12 TSLP cohort 1 grantees and case studies of seven TSLP 

cohort 1 grantees, including each grantees’ partners; an analysis of extant data, including 

grantee applications, early outcomes data, and other relevant project-specific data.

Estimated Completion Date:

The final report is scheduled for completion in spring 2018. 

Study of School Climate Transformation Grants

Study Purpose:

The study will describe how states and school districts that participate in the School Climate 

Transformation Grant (SCTG) program are coordinating services and supports with certain 

other related federal programs administered by the Departments of Health and Human Services 

and Justice. 

Key Question(s) Addressed:

• What does coordination between SCTG and Project AWARE grantees look like? 

• What does coordination between SCTG and U.S. Department of Justice grantees look like?

• What did grantees say about the value of coordination?

• What were the challenges and lessons learned?
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 156

Design:

The study will conduct telephone interviews with representatives from 38 state and local 

grantees to explore the ways in which grantees coordinate services, the benefits that grantees 

have experienced from program coordination, and challenges and lessons learned. 

Estimated Completion Date:

The report is scheduled for completion in spring 2018. 

Study of Title I Schoolwide and Targeted Assistance Programs

Study Purpose:

The study will examine how Title I schoolwide programs use the schoolwide flexibility to design 

services and strategies to address the needs of low-achieving students and subgroups, and 

how such strategies compare to approaches used in targeted assistance programs. The study 

will include interviews and analysis of extant data in approximately 30 Title I schools, including 

both schoolwide and targeted assistance programs, as well as surveys of principals and district 

administrators for a representative sample of Title I schools.

Key Question(s) Addressed: 

• How do schoolwide and targeted assistance programs use Title I funds to improve student 

achievement, particularly for low-achieving subgroups?

• How do districts and schools make decisions about how to use Title I funds in schoolwide 

programs and targeted assistance programs?

• To what extent do schoolwide programs commingle Title I funds with other funds or 

coordinate the use of Title I funds with other funds?

Design:

The study will include both in-depth case studies and surveys of a nationally representative 

sample of 350 districts and 1,400 schools. The case studies will include approximately 35 Title I 

schools including both schoolwide and targeted assistance programs. Data collection for the 

case studies will include site visits and interviews with two to four staff members in each school 

and approximately two staff members in each district, as well as extant documents and data, 

including Title I budgets and plans.

Estimated Completion Date:

The report is scheduled for completion in spring 2018.

Study of Digital Learning Resources for Instructing English Learners

Study Purpose:

This study will examine the use of digital learning resources (DLRs) to support the English 

language acquisition and academic achievement of English learners in K–12 education. The 

study will explore the range of DLRs that are available for use with English learner students, 

examine how districts and schools select and use these apps, and consult an expert panel of 

technology developers, practitioners, and education researchers for ideas on ways to improve 

the design and use of apps to support learning for English learner students. The study will 

culminate in a final report that presents findings from the study, as well as two short fieldfocused toolkits or guides for educators and technology developers that present key information 

from the study in a manner that will be accessible and useful for those audiences.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 157

Key Question(s) Addressed: 

• How do districts and teachers identify and select DLRs in general? How do districts and 

teachers identify and select DLRs specifically to support English learner students? 

• What are the types and characteristics of DLRs that districts most commonly report as used 

to support English learners? What are the types and characteristics of DLRs that teachers 

most commonly report that they use in instructing and structuring learning activities for their 

English learner students?

• How do teachers of English learner students use DLRs in the instruction of English learner 

students? 

• To what extent do teachers receive professional development or other supports for effective 

use of DLRs for instruction? Which professional development approaches do teachers 

report to be most helpful in supporting their use of DLRs in instruction?

• What are barriers to and supports for (1) the use of DLRs in instruction of English learner 

students and (2) the use of DLRs by students at home? How can districts, schools, and DLR 

developers address these?

• How do districts and teachers define and measure the success of their use of technology to 

support English learner students?

• How could developers and practitioners improve the usefulness of DLRs for instructing 

English learner students?

Design:

The study will explore the range of such apps that are available for use with English learner 

students, examine how districts and schools select and use these apps, and consult an expert 

panel of technology developers, practitioners, and education researchers for ideas on ways to 

improve the design and use of apps to support learning for English learner students. It will rely 

on six key components: (1) a literature review; (2) market research on existing DLRs for K–12 

instruction; (3) survey of school districts; (4) survey of teachers; (5) case studies; and (6) an 

expert panel of technology developers, practitioners, and education researchers.

Estimated Completion Date:

The report is scheduled for completion in spring 2018.

Study of the Title III Native American and Alaska Native Children in Schools 

Program

Study Purpose:

The study will examine services and strategies for English learners in the Native American, 

Alaska Native, Native Hawaiian, and Pacific Islander communities that are supported through 

the Native American and Alaska Native Children in Schools (NAM) program. The study includes 

case study of 25 NAM grantees, and examines: (1) the types of services and strategies that 

NAM funds support to address the instructional needs of these communities and develop 

student proficiency in both English and (optionally) native languages; (2) how grantees plan, 

implement, and evaluate their respective projects; (3) how grantees coordinate and prioritize the 

use of NAM funds in relation to other federal, state, and local resources; and (4) how grantees 

use data and evidence to inform program implementation and meet the Department’s reporting 

requirements.

Key Question(s) Addressed:

• How does the NAM program support grantees in providing services to Native American, 

Alaska Native, Native Hawaiian, and Pacific Islander children, especially through teacher 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 158

training and professional development, curriculum development, parent engagement, and 

English (especially academic English) and native language instruction? How do NAM 

grantees incorporate technology to support and/or preserve heritage languages through 

dual-language or other forms of English language instruction? How many students are 

served through each of these strategies? Are the strategies focused on particular student 

populations?

• What challenges do NAM grantees face in providing funded services, and what steps have 

grantees taken to overcome these challenges? To what extent does the Department or 

other external entities provide support to overcome these challenges? What lessons have 

the grantees learned?

• How do schools deliver services or interventions for each of the strategies? What specific 

services are provided, and who provides the services?

• What are the roles and responsibilities among (as applicable) tribal entities, public schools, 

local education districts, and state agencies in implementing NAM grants and meeting 

federal reporting requirements?

• What are NAM grant stakeholders’ perceptions of community participation and student 

engagement in language instruction and other educational programs?

Design:

The study will include site visits to NAM grantees to obtain detailed information regarding how 

they are implementing the NAM program and meeting the needs of their students and 

communities, as well as telephone surveys or interviews of local grant coordinators and state 

directors of Indian education to inform the case study site visits. The study will also utilize extant 

data, including grant applications.

Estimated Completion Date:

The report is scheduled for completion in fall 2018.

Evaluation of the Title I, Part D Neglected or Delinquent Program

Study Purpose:

The study will examine the implementation of educational programs for children and youth in 

residential facilities and correctional institutions funded under Title I, Part D of ESEA. The study 

will be informed by surveys of state grantees and local subgrantees to examine the types of 

services and strategies that Part D funds support, how state and local agencies assist students 

in transitioning back to schools, how state correctional facilities implement institution-wide Part 

D projects, and how grantees assess the educational outcomes of participating students.

Key Question(s) Addressed: 

• What types of services and strategies do Title I, Part D funds support for children and youth 

in correctional and child welfare settings?

• How do correctional facilities and child welfare agencies assist students in transitioning back 

to districts and schools, including those outside their jurisdictions?

• How do state correctional facilities plan and implement institution-wide Part D projects?

• How do grantees assess the educational outcomes of students participating in 

Part D-funded educational programs?

Design:

The study will include a review of extant data, a review of literature related to programs for 

neglected and delinquent youth, surveys of state and local coordinators of Title I-Part D funded 

programs, and site visits to state agencies, school districts, correctional institutions, and child 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 159

welfare facilities to obtain more detailed information on how grantees and subgrantees are 

implementing the Part D programs and how they are meeting the needs of their students. 

Estimated Completion Date:

The report is scheduled for completion in spring 2018.

Evaluation of the Migrant Education Program

Study Purpose:

This study will examine how states, districts, and schools are providing instructional supports 

and assessing highly mobile migratory students, as well as examine state plans for 

implementing the new accountability requirements (as they pertain to migratory students) under 

the ESSA. The Migrant Education Program is a state program, giving states flexibility in how 

they allocate funds to serve migratory students, allowing the local entities that serve migratory 

students to be both local educational agencies and local operating agencies. 

Key Question(s) Addressed: 

• How do state and local grantees respond to federal requirements for serving migratory 

children?

• What services are provided to migratory students?

• How do Migrant Education Program grantees collaborate with other programs and agencies 

to address the needs of highly mobile students?

• How do migrant programs support students in earning high school diplomas and 

equivalency, and preparing for postsecondary education and the workforce? 

Design:

The study will include survey of states, a nationally representative sample of district programs 

serving migrant children, site visits to state and local grantees. The contractor shall prepare a 

final report that integrates findings from the surveys and case studies as well as from an extant 

data analysis.

Estimated Completion Date:

The report is scheduled for completion in fall 2018.

Evaluation of the Indian Education LEA Grants Program

Study Purpose:

This study will examine the implementation of the Indian Education Local Education Agency 

(LEA) Grants Program funded under Title VI of the ESEA, as amended by the ESSA. The study 

will document the scope of activities funded by the Indian Education LEA Grants Program and 

will examine the LEA and tribal-level implementation of the grants. Specifically, the study will 

examine the processes used to identify and count eligible children and how grantees establish 

LEA program priorities and implement grant-funded services.

Key Question(s) Addressed: 

• What strategies do grantees use to identify and count program-eligible children?

• How do grantees plan services?

• What services do Indian Education LEA Grants fund?

• How do grantees assess the project outcomes of students participating in Indian Education 

LEA Grants Program-funded educational programs? 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 160

Design:

This study will consist of four key components: (1) analysis of extant data including APRs and 

Electronic Application System for Indian Education data; (2) review of relevant literature; (3) 

survey of 1,300 coordinators of Indian Education LEA grant programs; and (4) case studies of 

nine districts.

Estimated Completion Date:

The report is scheduled for completion in winter 2018-19. 

Study of Weighted Student Funding (WSF) Systems

Study Purpose:

This study is examining WSF and school-based budgeting (SBB) systems, which are methods 

for providing funds to schools based on the numbers and, in the case of WSF, types of students 

they served. The study will examine how districts have implemented SBB and WSF systems for 

allocating funds to schools and how these districts and their schools compare with districts 

using traditional systems for allocating school resources.

Key Question(s) Addressed:

• How are resources allocated to schools in districts with SBB or WSF systems compared with 

districts with more traditional resource allocation practices?

• In what ways do schools have autonomy and control over resource allocation decisions, and 

how does this vary between districts with SBB or WSF and other districts?

• How has the implementation of WSF systems affected the distribution of funding provided to 

schools?

• What challenges did districts and schools experience in implementing SBB and WSF, and 

how did they respond to those challenges?

Design:

To obtain detailed information about the implementation, benefits, and challenges of WSF 

systems, this study will conduct case studies in nine districts using such systems, including site 

visits that include in-person interviews and collection and analysis of extant documents and data 

such as school budgets and planning documents. In addition, the study will administer surveys 

to a nationally representative survey of districts and schools to enable the study to ground the 

case study data in the larger context of WSF, SBB, and traditional methods of resource 

allocation.

Estimated Completion Date:

The report is scheduled for completion in winter 2018–19.
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 161

Appendix F: Glossary of Acronyms and Abbreviations

Acronym Description

AARTS Audit Accountability and Resolution Tracking System 

ACGR Adjusted Cohort Graduation Rate

AASA American Association of School Administrators

AP Advanced Placement 

APG Agency Priority Goal 

APP Annual Performance Plan 

APR Annual Performance Report 

AY Academic Year 

BA Bachelor of Arts

CAMS Case and Activity Management System

CAP Goals Cross-Agency Priority Goals 

CAR Consolidated Annual Report

CCD Common Core of Data

CDQR Coordinated Data Quality Reviews

CMO Charter Management Organization

CMS Case Management System

CoSN Consortium for School Networking 

CPS Current Population Survey 

CRDC Civil Rights Data Collection 

CSAM Cyber Security Assessment Management

CSP Charter School Program

CSPR Consolidated State Performance Reports

CTE Career and Technical Education 

CTS Case Tracking System

DAP Digital Analytics Program 

DLRs Digital Learning Resources 

DM/RM Document/Record Management 

DQC Data Quality Campaign

DRT Data Request Team 

DST Data Strategy Team 

ED U.S. Department of Education 

EDGAR Education Department General Administrative Regulations 

EDGB EDFacts Data Governance Board

ELA English Language Acquisition

ELC TA Early Learning Challenge Technical Assistance Program 

ERIC Education Resources Information Center 

ERM Enterprise Risk Management

ESEA Elementary and Secondary Education Act of 1965 

ESH Education Superhighway

ESSA Every Student Succeeds Act 
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 162

Acronym Description

EVS Employee Viewpoint Survey 

FAC Federal Audit Clearing House

FDSL Federal Direct Student Loan 

FERPA Family Educational Rights and Privacy Act 

FEVS Federal Employee Viewpoint Survey 

FFEL Federal Family Education Loan 

FISMA Federal Information Security Modernization Act of 2014

FITARA Federal Information Technology Acquisition Reform Act

FPCO Family Policy Compliance Office 

FPPS Federal Personnel/Payroll System

FY Fiscal Year 

GEAR UP Gaining Early Awareness and Readiness for Undergraduate Programs

GLBA Gramm-Leach-Bliley Act

GSA General Services Administration 

HBCUs Historically Black Colleges and Universities 

HE Higher Education

HEA Higher Education Act of 1965 

HEAL Health Education Assistance Loans 

HR Human Resources 

HSI-STEM Hispanic Serving Institution STEM Articulation Program 

i3 Investing in Innovation Program 

IDEA Individuals with Disabilities Education Act 

IEP Individualized Education Program

IES Institute of Education Sciences 

IHE Institutions of Higher Education 

IPEDS Integrated Postsecondary Education Data System 

IT Information Technology 

KEA Kindergarten Entry Assessment 

LEA Local Educational Agency 

LOA Local Operating Agency 

MTSS-B Multi-tiered Systems of Support for Behavior

NA Not Applicable 

NAEP National Assessment of Educational Progress 

NAM Native American and Alaska Native Children in Schools Program

NCES National Center for Educational Statistics 

NIEER National Institute for Early Education Research 

NLTS National Longitudinal Transition Study

NRS National Reporting System

NSLDS National Student Loan Data System

NTID National Technical Institute for the Deaf
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 163

Acronym Description

OCFO Office of the Chief Financial Officer 

OCIO Office of the Chief Information Officer 

OCR Office for Civil Rights 

OCTAE Office of Career, Technical, and Adult Education 

OELA Office of English Language Acquisition 

OESE Office of Elementary and Secondary Education 

OET Office of Educational Technology 

OGC Office of the General Counsel 

OHR Office of Human Resources 

OIG Office of Inspector General 

OII Office of Innovation and Improvement 

OM Office of Management 

OMB Office of Management and Budget 

OPE Office of Postsecondary Education 

OPEPD Office of Planning, Evaluation, and Policy Development 

OPM Office of Personnel Management 

OSEP Office of Special Education Programs 

OSERS Office of Special Education and Rehabilitative Services 

OSP Opportunity Scholarship Program 

P3 Performance Partnership Pilots for Disconnected Youth

PBCS Performance-Based Compensation System 

PBIS Positive Behavior Interventions and Supports

POC Principal Operating Component 

PPIRS Past Performance Information Retrieval System 

PPSS Policy and Program Studies Service 

PSC Partner Support Center

PTAC Procurement Technical Assistance Centers

REAP Rural Education Achievement Program 

RLIS Rural and Low-Income Schools Program 

RSA Rehabilitation Services Administration

RTT Race to the Top 

RTT-ELC Race to the Top—Early Learning Challenge Program

SAA Student Aid Administration

SASS Schools and Staffing Survey 

SBB School-Based Budgeting 

SCTG School Climate Transformation Grant Program 

SEA State Educational Agency 

SECOPS Security Operations Management

SFA Student Financial Assistance 

SIG School Improvement Grant 

SIP Strengthening Institutions Program 

SLDS Statewide Longitudinal Data System
APPENDICES

FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 164

Acronym Description

SRSA Small, Rural School Achievement Program 

STEM Science, Technology, Engineering, and Mathematics 

SY School Year 

TA&D Technical Assistance and Dissemination 

TBD To Be Determined 

TEACH Teacher Education Assistance for College and Higher Education Grant 

TIF Teacher Incentive Fund 

TQRIS Tiered Quality Rating and Improvement System 

TSLP Turnaround School Leaders Program

USAC Universal Service Administrative Company

WSF Weighted Student Funding

WWC What Works Clearinghouse
OUR MISSION IS TO PROMOTE STUDENT ACHIEVEMENT AND PREPARATION FOR

GLOBAL COMPETITIVENESS BY FOSTERING EDUCATIONAL EXCELLENCE AND 

ENSURING EQUAL ACCESS. 

WWW.ED.GOV